Update robots.txt and sitemap.xml.