For robots to go and access the website, and crawl.