Updating robots.txt, updating MW sitemaps and how to generate them noted in robots file