]> cat aescling's git repositories - mastodon.git/commitdiff
Disallow robots from indexing /interact/ (#10666)
authorThibG <thib@sitedethib.com>
Wed, 1 May 2019 22:10:19 +0000 (00:10 +0200)
committerEugen Rochko <eugen@zeonfederated.com>
Wed, 1 May 2019 22:10:19 +0000 (00:10 +0200)
This does not provide any new information and may just triple the number
of crawled pages

public/robots.txt

index d93648beee28c91a74d07b46c260a382daf61080..771bf2160b4e3bdc65981d3680ef1ae78925f79d 100644 (file)
@@ -2,3 +2,4 @@
 
 User-agent: *
 Disallow: /media_proxy/
+Disallow: /interact/