Turn off the timer if there's nothing to wait for