Huge speedup for testsuite categories
This ensures that we won't try to do cleanups of test cases that we are skipping
e.g. this brings down the time required to run the cmdline category on my machine from ~70s to ~30s