Add support for removing the mp3 files

If preserve files are disabled, this will now
remove the mp3 files as well.