schism-audio/htdemucs-ft-coreml

HTDemucs FT (bag of 4) — Core ML

The fine-tuned HTDemucs bag (htdemucs_ft): four cores, one per stem, one-hot combined — stem k comes from Core{k}. Best-quality 4-stem Demucs, for Core ML on Apple devices. ~4× the compute of plain htdemucs.

Converted from the same verified reference used by the schism-mlx MLX ports. Two variants per model:

File	Precision	Compute units	segment residual
HTDemucs_Core{0-3}_fp16.mlpackage	FLOAT16	ALL (ANE)	−49.5 … −64.5 dB
HTDemucs_Core{0-3}_fp32.mlpackage	FLOAT32	CPU+GPU	−114.4 … −128.0 dB

Verified on-device-equivalently via coremltools on an M5 Max, against the reference implementation on real audio. fp16 is ANE-eligible and recommended for iPhone / iPad; fp32 is the tight-parity fallback.

Download

.mlpackage bundles must be materialized as real files — the Core ML compiler rejects the symlinks that a default snapshot_download creates in the Hugging Face cache:

from huggingface_hub import snapshot_download
path = snapshot_download("schism-audio/htdemucs-ft-coreml", local_dir="./htdemucs-ft-coreml")

(or hf download schism-audio/htdemucs-ft-coreml --local-dir ./htdemucs-ft-coreml). Swift hosts downloading files directly are unaffected.

I/O contract

• input mag: (1, 4, 2048, 336) float32 — complex-as-channels STFT of one 7.8 s / 343980-sample segment ([c0_re, c0_im, c1_re, c1_im]), n_fft 4096, hop 1024, normalized 1/sqrt(n_fft), demucs framing (see test vectors note)
• input mix: (1, 2, 343980) float32 — the segment waveform
• output spec_out: (1, S, 4, 2048, 336) — CaC mask spectrogram → complex → iSTFT on host
• output time_out: (1, S, 2, 343980) — time-branch waveform; final stems = iSTFT(spec_out) + time_out
• full tracks: overlapping segments (25% overlap), triangular blending, centered padding pulled from surrounding audio
• bag combine: run all four cores per segment; take stem k from core k

DSP frontend (host-side)

The Core ML graph contains the network only. The host implements the audio frontend and must match schism_mlx.audio numerically — test_vectors_*.npz in this repo holds deterministic input/output pairs (float32; match within ~1e-4 relative to be interchangeable with what these models were verified against). test_vectors_pipeline.npz additionally pins the full-track orchestration around the model call (chunk grid, blending, normalization) via a deterministic mock model — scaled-down sizes, same algorithm, generated by the shipped schism-mlx separate() / windowing code. A validated Swift implementation (Accelerate; modules SchismDSP and SchismPipeline) is available at schism-audio/schism-dsp, tested against these exact vectors.

License

MIT, inherited from facebookresearch/demucs (Rouard, Massa, Défossez — Meta AI). Core ML conversion by schism-audio.