Properly clean up temporary files and directories during tests

Eventually we should probably move those helper methods out of FileSystemTestsBase so that other tests can use them.