Add a robots.txt to mitigate bot crawling.