Only stop the timer if a timeout is set