Whitespace cleanup of availability check code before edit.
Retry an endpoint if it timed out.

A timeout does not return a response so there is no point in checking the
response code! Check for an empty response and re-try.
Tidy up availability check code.

Make it more ruby-like.
Reduce availability check timeout to 10 seconds.

20 seconds is a long time, especially if there are a lot of services timing out.
Add more status codes to availability checking code.

Now treats 401 as success. This is a little contentious but it is hard to argue
that a 401 is straight failure as it has responded and that is what we are
checking for.

403 is failure as the server is refusing to return what it was asked to return.

We give 501 another chance in the same way we treat 500 as some servers just
break on HEAD but do work on GET.