Emulate robots.txt from Nginx to disallow all
Merge pull request #317 from ministryofjustice/feature/robots.txt

Emulate robots.txt from Nginx to disallow all