Do not delete last_test_dir with file handle open

Previous behavior does not work on windows.