Add a block all robots to all vhosts by default

We don't want any of our internal vhosts being indexed by robots.
Merge pull request #399 from alphagov/add-robots-to-vhosts

Add a block all robots to all vhosts by default