Only kill by file if the file exists