There is no need to kill it completely, just step it down

This reduces the crazy amount of time the failover tests require