Really fix the stupid timers finally

I was operating under sensible assumptions when I added those asserts.
Unfortunately, the code wasn't.