Modified robots.txt to allow crawling