[#73455918] Set user agent for crawler

Set the crawler's HTTP user agent to "GOV.UK Crawler" and add a test for
this.
If the need is to identify where it's running from, would something like the hostname of the machine running it be helpful instead of a version?