Only sleep if necessary