Changed test runner to reliably clean up.

I can see this behavior not being totally desirable in all cases. This is the automated case though, so for now, lets go with this.