unifolia/muscriptor-small-onnx

MuScriptor small — ONNX conversion

An ONNX Runtime conversion of MuScriptor/muscriptor-small, the 103M-parameter multi-instrument music transcription model by Kyutai × Mirelo × IRCAM. The weights are theirs and unmodified apart from the conversion; all credit for the model belongs to them.

• Original model and paper: https://github.com/muscriptor/muscriptor, arXiv:2607.08168
• Licence: CC BY-NC 4.0 — non-commercial use only, inherited from the original weights.

It exists so that Lepton Hum, a desktop practice tool for musicians, can run the model through ONNX Runtime without shipping PyTorch. Anyone else running ONNX Runtime can use it the same way, under the same non-commercial terms.

Files

file	what it is
muscriptor-small-cond.onnx	conditioner: audio → prefix embeddings
muscriptor-small-lm.onnx	the decoder-only transformer, KV cache in and out

muscriptor-small-cond.onnx

audio	f32 [1, 80000] — one 5-second chunk at 16 kHz, zero-padded
instrument	i64 [1, K] — instrument class ids (group + 2; 1 = unconditional)
dataset	i64 [1, 1] — always 1, the null class
→ prefix	f32 [1, 501 + 1 + K, 768]

The spectrogram is inside this graph on purpose. The checkpoint carries its own STFT window and mel filterbank as buffers, and they are stored quantized — about 2.4e-4 away from a freshly computed periodic Hann window and htk filterbank. That rounding raises the window's sidelobe floor by roughly 70 dB, and since the model reads log-magnitude features, bins that the textbook formulas put at the log(1e-6) floor sit near −4 for this model. Recomputing the front end therefore feeds it a spectrum it was not trained on, so the conversion keeps the checkpoint's own.

muscriptor-small-lm.onnx

tokens	i64 [1, S]
prefix	f32 [1, P, 768] — the conditioner's output; P = 0 after the first step
past_k, past_v	f32 [14, 1, T_past, 12, 64]
→ logits	f32 [1, 1393] — last position only
→ present_k, present_v	the cache, grown by this step

Positions are not an input: everything in the cache precedes the call, so T_past is where the new tokens start. The graph is branch-free, so one exported model serves both the prefill and single-token decoding.

Greedy decoding, MT3 token vocabulary (1393 tokens), 5-second chunks with a tie prologue carrying notes across chunk boundaries — see the original repository for the decoding rules.

Fidelity

Checked against the reference PyTorch implementation on the same 3.7-minute mix, both given identical 16 kHz audio: drums 1103 notes vs 1103 (exact), bass 522 vs 514, guitar 1730 vs 1845. Greedy decoding turns the ~1e-5 logit differences between ONNX Runtime and PyTorch into an occasional flipped argmax that then cascades, so agreement within a few percent per instrument is what parity looks like here.

Layer-level checks from the conversion script: conditioner max|Δ| 7.7e-3 (the STFT's numerics), prefill logits 2.7e-5, three decode steps ≤ 5.4e-5.

Reproducing

scripts/export_muscriptor_onnx.py in the Lepton Hum repository. It fetches the original gated weights (accept the licence on the model page first), exports both graphs, and asserts parity against the reference modules before writing anything.