-
@GNUmanth It says “crawl allowed”, so it’s not a
robots.txtproblem. Must be something at the CDN level then. They have some tips to debug: support.cloudflare.com/hc/en-us/articles/200169806-Troubleshooting-crawl-errors.
robots.txt problem. Must be something at the CDN level then. They have some tips to debug: support.cloudflare.com/hc/en-us/articles/200169806-Troubleshooting-crawl-errors.