The other relevant `TimeoutError`s should inherit from `TimeoutError`, so it should work as-is, right?

With the exception of `TimeoutStateError` which inherits from `HTTPError`, but that shouldn't be retried anyways (it only gets raised when something in the code is broken, such as incorrectly mangling the Timeout object's internal state).