Send a robots.txt directive to disallow indexing

We'd like to get people comfortable pushing this app to anywhere they
want, including platform-as-a-service and software-as-a-service providers.

This will help us to prototype things and experiment more freely.

But we don't want those copies of the app to be indexed by search engines,
so send a robots.txt file.

This duplicates the robots.txt file we send from Puppet [1], but also applies
to services beyond our control.

This does not affect GOV.UK, because the only parts of the site that are routed
to this application are below /performance.

[1]: https://github.com/alphagov/pp-puppet/blob/jenkins-performanceplatform_puppet-616/modules/performanceplatform/manifests/proxy_vhost.pp#L170-179
Set X-Robots-Tag HTTP header conditionally

We don't want unpublished dashboards to be showing up in search
engine results.

https://developers.google.com/webmasters/control-crawl-index/docs/robots_meta_tag
Merge pull request #613 from alphagov/send-robots-txt-response

Send a robots.txt directive to disallow indexing
Merge pull request #614 from alphagov/72166404-unpublished-dashboard-warning

Add big red warning to unpublished dashboards