contint: deny webspider from accessing Jenkins

Some web spiders have the bad habit to come hit our Jenkins installation
which might sometime cause it to end up slightly overloaded.  This patch
is inspired by Gerrit blacklisting already in use.

Change-Id: Ia689694df3f704e31fce29f8bc767e9899cf5225
Merge "contint: deny webspider from accessing Jenkins" into production