Package: demucs3
Demucs Music Source SeparationInfoThis repository is fork ofdemux. The only reason for it to have independent module installed from pip for demucs v3 version.pip install demucs3DataThis is the 3rd release of Demucs (v3), featuring hybrid source separation.For the waveform only Demucs (v2):Go this commit.
If you are experiencing issues and want the old Demucs back, please fill an issue, and then you can get back to the v2 withgit checkout v2.We provide an implementation of Hybrid Demucs for music source separation, trained
both on theMusDB HQdataset, and with internal extra training data.
They can separate drums, bass and vocals from the rest and achieved the first rank
at the 2021Sony Music DemiXing Challenge (MDX)Demucs is based on U-Net convolutional architecture inspired byWave-U-Net.
The most recent version features hybrid spectrogram/waveform separation,
along with compressed residual branches, local attention and singular value regularization.
Checkout our paperHybrid Spectrogram and Waveform Source Separationfor more details. As far as we know, Demucs is currently the only model supporting true
end-to-end hybrid model training with shared information between the domains,
as opposed to post-training model blending.When trained only on MusDB HQ, Hybrid Demucs achieved a SDR of 7.33 on the MDX test set,
and 8.11 dB with 200 extra training tracks. It is particularly efficient for
drums and bass extraction, althoughKUIELAB-MDX-Netperforms better for
vocals and other accompaniments.Important news if you are already using DemucsSee therelease notesfor more details.12/11/2021: ReleasingDemucs v3with hybrid domain separation. Strong improvements
on all sources. This is the model that won Sony MDX challenge.11/05/2021: Adding support for MusDB-HQ and arbitrary wav set, for the MDX challenge. For more information
on joining the challenge with Demucs seethe Demucs MDX instructions28/04/2021:Demucs v2, with extra augmentation and DiffQ based quantization.EVERYTHING WILL BREAK, please restart from scratch following the instructions hereafter.
This version also adds overlap between prediction frames, with linear transition from one to the next,
which should prevent sudden changes at frame boundaries. Also, Demucs is now on PyPI, so for separation
only, installation is as easy aspip install demucs:)13/04/2020:Demucs released under MIT: We are happy to release Demucs under the MIT licence.
We hope that this will broaden the impact of this research to new applications.Comparison with other modelsWe provide hereafter a summary of the different metrics presented in the paper.
You can also compare Hybrid Demucs (v3),KUIELAB-MDX-Net,Spleeter, Open-Unmix, Demucs (v1), and Conv-Tasnet on one of my favorite
songs on mysoundcloud playlist.Comparison of accuracyOverall SDRis the mean of the SDR for each of the 4 sources,MOS Qualityis a rating from 1 to 5
of the naturalness and absence of artifacts given by human listeners (5 = no artifacts),MOS Contaminationis a rating from 1 to 5 with 5 being zero contamination by other sources. We refer the reader to ourpaper,
for more details.ModelDomainExtra data?Overall SDRMOS QualityMOS ContaminationWave-U-Netwaveformno3.2--Open-Unmixspectrogramno5.3--D3Netspectrogramno6.0--Conv-Tasnetwaveformno5.7-Demucs (v2)waveformno6.32.372.36ResUNetDecouple+spectrogramno6.7--KUIELAB-MDX-Nethybridno7.52.862.55Hybrid Demucs (v3)hybridno7.72.833.04MMDenseLSTMspectrogram804 songs6.0--D3Netspectrogram1.5k songs6.7--Spleeterspectrogram25k songs5.9--RequirementsYou will need at least Python 3.7. Seerequirements_minimal.txtfor requirements for separation only,
andenvironment-[cpu|cuda].yml(orrequirements.txt) if you want to train a new model.For Windows usersEverytime you seepython3, replace it withpython.exe. You should always run commands from the
Anaconda console.For musiciansIf you just want to use Demucs to separate tracks, you can install it withpython3-mpip-UinstalldemucsAdvanced OS support are provided on the following page,you must read the page for your OS before posting an issues:If you are using Windows:Windows support.If you are using MAC OS X:Mac OS X support.If you are using Linux:Linux support.For machine learning scientistsIf you have anaconda installed, you can run from the root of this repository:condaenvupdate-fenvironment-cpu.yml# if you don't have GPUscondaenvupdate-fenvironment-cuda.yml# if you have GPUscondaactivatedemucs
pipinstall-e.This will create ademucsenvironment with all the dependencies installed.You will also need to installsoundstretch/soundtouch: on Mac OSX you can dobrew install sound-touch,
and on Ubuntusudo apt-get install soundstretch. This is used for the
pitch/tempo augmentation.Running in DockerThanks to @xserrat, there is now a Docker image definition ready for using Demucs. This can ensure all libraries are correctly installed without interfering with the host OS. See his repoDocker Facebook Demucsfor more information.Running from ColabI made a Colab to easily separate track with Demucs. Note that
transfer speeds with Colab are a bit slow for large media files,
but it will allow you to use Demucs without installing anything.Demucs on Google ColabWeb Demo(Possibly broken with the update, need to investigate) Integrated toHuggingface SpaceswithGradio. See demo:Separating tracksIn order to try Demucs, you can just run from any folder (as long as you properly installed it)demucsPATH_TO_AUDIO_FILE_1[PATH_TO_AUDIO_FILE_2...]# for Demucs# If you used `pip install --user` you might need to replace demucs with python3 -m demucspython3-mdemucs--mp3--mp3-bitrateBITRATEPATH_TO_AUDIO_FILE_1# output files saved as MP3# If your filename contain spaces don't forget to quote it !!!demucs"my music/my favorite track.mp3"# You can select different models with `-n` mdx_q is the quantized model, smaller but maybe a bit less accurate.demucs-nmdx_qmyfile.mp3If you have a GPU, but you run out of memory, please add-d cputo the command line. See the section hereafter for more details on the memory requirements for GPU acceleration.Separated tracks are stored in theseparated/MODEL_NAME/TRACK_NAMEfolder. There you will find four stereo wav files sampled at 44.1 kHz:drums.wav,bass.wav,other.wav,vocals.wav(or.mp3if you used the--mp3option).All audio formats supported bytorchaudiocan be processed (i.e. wav, mp3, flac, ogg/vorbis on Linux/Mac OS X etc.). On Windows,torchaudiohas limited support, so we rely onffmpeg, which should support pretty much anything.
Audio is resampled on the fly if necessary.
The output will be a wave file, either in int16 format or float32 (if--float32is passed).
You can pass--mp3to save as mp3 instead, and set the bitrate with--mp3-bitrate(default is 320kbps).Other pre-trained models can be selected with the-nflag.
The list of pre-trained models is:mdx: trained only on MusDB HQ, winning model on track A at theMDXchallenge.mdx_extra: trained with extra training data (including MusDB test set), ranked 2nd on the track B
of theMDXchallenge.mdx_q,mdx_extra_q: quantized version of the previous models. Smaller download and storage
but quality can be slightly worse.mdx_extra_qis the default model used.SIG: whereSIGis a single model from themodel zoo.The--shifts=SHIFTSperforms multiple predictions with random shifts (a.k.a theshift trick) of the input and average them. This makes predictionSHIFTStimes
slower. Don't use it unless you have a GPU.The--overlapoption controls the amount of overlap between prediction windows (for Demucs one window is 10 seconds).
Default is 0.25 (i.e. 25%) which is probably fine.Memory requirements for GPU accelerationIf you want to use GPU acceleration, you will need at least 8GB of RAM on your GPU fordemucs. Sorry, the code for demucs is not super optimized for memory! If you do not have enough memory on your GPU, simply add-d cputo the command line to use the CPU. With Demucs, processing time should be roughly equal to 1.5 times the duration of the track.Training DemucsIf you want to train (Hybrid) Demucs, please follow thetraining doc.MDX Challenge reproductionIn order to reproduce the results from the Track A and Track B submissions, checkout theMDX Hybrid Demucs submission repo.How to cite@inproceedings{defossez2021hybrid,
  title={Hybrid Spectrogram and Waveform Source Separation},
  author={D{\'e}fossez, Alexandre},
  booktitle={Proceedings of the ISMIR 2021 Workshop on Music Source Separation},
  year={2021}
}LicenseDemucs is released under the MIT license as found in theLICENSEfile.