Fix robots.txt and sitemap problems.