remove the cached tests after every run so we don't continue to run tests that we remove
conditional hackery