removes robotstxt