Safely remove the temp directory.