graceful teardown of cache

@icewind1991 @DeepDiver1975 another place that, when failing, stops further test execution. 3 loc changed, give it a quick review.