[#58498312] Test for /robots.txt

Since it's quite important to our site and we wouldn't want to break it
again. Technically this belongs to static, not frontend, but static doesn't
respond to `HEAD /` and there aren't any other fixed name assets.

https://github.gds/gds/puppet/pull/996
https://www.pivotaltracker.com/story/show/58498312
Merge pull request #50 from alphagov/check_static_robots_txt

[#58498312] Test for /robots.txt