Clear timeouts when the Upstream closes for any reason- not just an error.