Update robots.txt to allow Google to crawl the website except for sensitive folders