Centralize vault cleanup

Further attempts at getting rid of intermittent vault test failures.