Have regressions clean up after themselves
Add better cleanup for regressions

TODO: Make this more automatic