Allow remove_temp_files to accept a directory