Do our best to stop OVS before we upgrade it

Change-Id: I5f9bdd9005d0e107e8e823e62b3a4e205eb00338