Adds robots.txt and disallows CloudFlare-generated directory.