Post bug 21446 this test became racy since it is supposed to kill the queue and not have it restart - so let's kill the supervisor first.