TrevorJS/MelBandRoformer-Vocal-CoreML

Kim Mel-Band RoFormer (vocals) on Core ML

KimberleyJSN/melbandroformer (MIT, 228 M parameters) as a float16 Core ML .mlpackage, the vocal model behind slurper. Runs on the GPU on macOS 15 and later. The instrumental is the mix minus the vocals.

Graph

One 8 s chunk at 44.1 kHz (352 800 samples), with the STFT and inverse STFT folded into the graph as constant DFT matmuls, the periodic Hann window included:

name	shape	meaning
frames (input)	[1, 2, 801, 2048]	unwindowed 2048-sample frames at hop 441 of the chunk reflect-padded by 1024 on each side
recon (output)	[1, 2, 801, 2048]	windowed inverse-DFT frames of the vocals

The host overlap-adds recon at hop 441, divides by the summed squared window, and drops the 1024-sample pad. Chunks across a song overlap by half with linear crossfades. Sources/SlurperKit/VocalSeparator.swift in slurper is a reference host.

Conversion

scripts/convert_melband_roformer.py in slurper, after the coreai-model-zoo recipe: the band-average scatter becomes a constant matmul, the complex mask multiply becomes real arithmetic, rotary angles become constants, and attention goes through scaled_dot_product_attention. torch.jit.trace, then coremltools 9.0 to an ML program with float16 precision. Against the PyTorch model on the golden chunk: the re-authored float32 graph at cosine 0.9999984, the Core ML float16 model at cosine 0.999946 (39.6 dB SDR).

Files

• mbr_fp16.mlpackage/Manifest.json
• mbr_fp16.mlpackage/Data/com.apple.CoreML/model.mlmodel
• mbr_fp16.mlpackage/Data/com.apple.CoreML/weights/weight.bin (490 MB)
• golden_raw.f32: an 8 s stereo excerpt with vocals, channel-major float32, from the Core AI conversion's repository
• golden_vocals.f32: PyTorch's vocals for it, same layout, for host self-tests

Attribution

Mel-Band RoFormer by Ju-Chiang Wang, Wei-Tsung Lu and Minz Won (ByteDance). Checkpoint by Kimberley Jensen, trained with ZFTurbo's training code on lucidrains's BS-RoFormer implementation. Export recipe from coreai-model-zoo (Daisuke Majima, BSD-3-Clause). This conversion is MIT.