Check the crawlability of an URL


This tool checks the crawlability of an URL against robots.txt, metatags, x-robots and canonical urls.