Parse sitemap urls in robots.txt files.
Catch overflowerror in robots.txt crawl-delay
Remove crawl-delay stuff
Support checking Sitemap: URLs in robots.txt files.