Revert "There's no harm in losing the wakeup."

This reverts commit 17ae70dcc5bbdaebc09502880e92941cf75e002a.
Don't go to sleep forever if we miss a wakeup.