Don't sleep if we don't need to

According to #5670, we added a sleep to let a startup_state run cleanly. This
only needs to happen if we have a startup state, otherwise let's keep our foot
on the gas.