See whether a specific URL is allowed or blocked, and exactly which line of the file decides it, for whichever crawler you care about.
Reads the public robots.txt, nothing else. No signup. We do not save the addresses you check, and the site runs standard analytics like most sites. A free tool from Hawk Academy.
