DEVELOPER-835 enable robots.txt for prod
Merge pull request #403 from pmuir/DEVELOPER-835

DEVELOPER-828 enable robots.txt for prod