In case too long has passed, check for <= 0 and timeout immediately if needed.