Adds robots.text in preparation for sitemap.
Merge pull request #372 from 18F/feature/adds-robots

Adds robots.text in preparation for sitemap.