kickvn/htdemucs-onnx

HTDemucs (Demucs v4) - ONNX

ONNX export of Meta's HTDemucs v4 (Hybrid Transformer Demucs) from facebookresearch/demucs, using the weights from adefossez/HTDemucs. It separates music into 4 stems: drums, bass, other, vocals.

ONNX export by KICK; core network only, STFT/iSTFT done outside.

Files

• htdemucs.onnx - fp32 (~174 MB)
• htdemucs_fp16.onnx - fp16 weights, fp32 inputs/outputs (~88 MB)
• htdemucs_ft_vocals.onnx - htdemucs_ft vocals specialist, fp32 (~174 MB), see below

Inputs / outputs

Segment length 343980 samples (7.8 s at 44.1 kHz), 336 STFT frames (n_fft 4096, hop 1024, 2048 bins, complex-as-channels).

Name	Shape
in mix	[1, 2, 343980]
in spec	[1, 336, 4, 2048]
out out_spec	[1, 336, 4, 4, 2048]
out out_wave	[1, 4, 2, 343980]

Stem order: drums, bass, other, vocals. The mix must be normalised (mean/std) before the call, and the STFT, iSTFT and overlap-add must be done outside the model.

Export script: tools/aligner-candidates/export_demucs.py.

htdemucs_ft (vocals specialist)

htdemucs_ft_vocals.onnx is the vocals model of Meta's fine-tuned bag htdemucs_ft (demucs 4.1 pretrained: 4 HTDemucs models, each fine-tuned for one stem; bag weights [0, 0, 0, 1] for vocals). Only this one model is exported, so speed equals plain htdemucs. License MIT, original models by Meta / Alexandre Defossez.

• Same inputs/outputs and normalisation as htdemucs.onnx; use only the vocals stem (index 3), ignore the other three.
• Accompaniment = mix - vocals (stereo), computed outside the model.
• Parity vs PyTorch apply_model on a 30 s clip (ONNX Runtime DirectML): vocals SDR 120 dB, max abs diff 7.6e-7.
• Export: tools/aligner-candidates/export_demucs.py <out_dir> <clip.wav> htdemucs_ft.

License and credits

MIT. Original model by Alexandre Defossez / Meta AI. The weights and code are under the MIT license of facebookresearch/demucs.

Citation

@inproceedings{rouard2022hybrid,
  title={Hybrid Transformers for Music Source Separation},
  author={Rouard, Simon and Massa, Francisco and D{\'e}fossez, Alexandre},
  booktitle={ICASSP 23},
  year={2023}
}