TrevorJS/htdemucs-CoreAI

htdemucs on Core AI

Meta's Hybrid Transformer Demucs (htdemucs: drums, bass, other, vocals) converted to a Core AI model for macOS 27 on Apple Silicon. It is the drum, bass and other separator in slurper, a stem-splitting command-line tool.

Graph

htdemucs_fp32.aimodel, float32, one function main, fixed shapes for one 7.8 s segment at 44.1 kHz:

	Name	Shape	Contents
in	mix	[1, 2, 343980]	stereo audio
in	spec	[1, 4, 2048, 336]	HTDemucs._magnitude(HTDemucs._spec(mix)): left real, left imaginary, right real, right imaginary
out	time	[1, 8, 343980]	time branch, 4 sources × 2 channels
out	freq	[1, 16, 2048, 336]	frequency branch, 4 sources × 2 channels × real/imaginary

The graph is HTDemucs.forward from its normalization to just before _mask; both outputs are denormalized. The complex STFT stays on the host, so a caller:

1. computes spec as demucs does: reflect-pad by 1536 samples plus the remainder of the last hop, take a centered, normalized STFT (4096-sample periodic Hann, hop 1024), and keep bins 0..<2048 of frames 2..<338;
2. runs main;
3. inverts each source and channel of freq with HTDemucs._ispec (zero Nyquist bin, two zero frames each side, normalized inverse STFT, trim) and adds time.

For whole songs, split into 7.8 s segments overlapping by a quarter with triangular crossfades, as demucs's apply_model does. Source order is drums, bass, other, vocals.

float16 overflows to NaN in the frequency branch. Running two inferences at once on one loaded model corrupted the outputs, so run segments one at a time.

Verification

• A synthetic segment through Core AI on the GPU against PyTorch HTDemucs.forward: 114 dB (drums), 129 dB (bass), 114 dB (other), 100 dB (vocals) SDR.
• A 135 s song through slurper's Swift host against PyTorch apply_model (no shifts, overlap 0.25): 113-120 dB SDR per stem.

Conversion

scripts/convert_htdemucs.py in slurper: demucs 4.1.0, torch 2.13.0, coreai-torch 0.4.2 (coreai-core 1.0.0b2). It exports the core with torch.export, converts it with TorchConverter, and checks the Core AI output against PyTorch before saving.

License

MIT, as are the htdemucs weights in facebookresearch/demucs.