Handle the case where too many retries are attempted, by returning
a 503 reply.

Fixes #1401