Robots.txt Crawl-delay
-
Hi,
I noticed my robots.txt includes the following line:
Crawl-delay: 3600
I would like to know is there is a reason for this to happen, since this is not something in common with all my wordpress sites.
Thank you in advance,
The blog I need help with is theoutlierwalk.com.
Hi,
This is something you would need to ask in the general WordPress forum since it is not theme related.
The topic ‘Robots.txt Crawl-delay’ is closed to new replies.