Fixes #2133

Simplify querying for timeouts, and introduce cleaning-up abilities to guarantee we don't skip any timeouts
Merge pull request #2234 from Particular/hotfix-4.6.2

Fixes #2133: Stop losing timeouts