Updating robots.txt for allow and sitemap