musetric/chordmini-onnx

ChordMini ChordNet (2E1D) — classifier + CQT plan (ONNX / WebGPU)

ONNX export of the ChordMini chord recognizer (ChordNet "2E1D", 170-class large vocabulary), packaged for the musetric packages/ai runtime (onnxruntime-web on WebGPU).

The graph is the classifier only: it takes log-CQT feature windows and returns per-frame chord logits. Feature extraction is deliberately not baked in — the host computes a recursive constant-Q transform on WebGPU and hands the result over as a GPU buffer, so no features cross back to the CPU. This is not a drop-in audio -> chords model.

mono PCM @ 22050 Hz              (arithmetic-mean downmix — see Limitations)
  -> WebGPU recursive CQT        -> log(|CQT| + 1e-6) features [T, 144]
  -> pad/window                  -> [W, 108, 144]
  -> chordnet.onnx               -> logits [W, 108, 170]
  -> WebGPU smoothing + argmax   -> chord indices [T]

cqt-plan.bin ships with the model because it defines the features the graph expects: the octave schedule, the sparse per-octave FFT basis and the resampling FIR, baked from librosa 0.11.0. Model and plan are a matched pair — a release therefore carries a hashable feature-extraction contract instead of an implicit one.

Normalization ((x - mean) / (std + 1e-8)) is inside the graph. CQT, windowing, smoothing and argmax stay in the host so their GPU buffers stay reusable.

Intended uses & limitations

Intended:

• Chord recognition over music, as a stage in an audio pipeline.
• Client/edge inference via WebGPU through onnxruntime-web.

Out of scope:

• Standalone use without a host that computes librosa-equivalent log-CQT features, windows them to 108 frames, and applies smoothing + argmax to the logits (see musetric packages/ai and packages/cqt).
• Use in other training frameworks — this is an inference-only export.

Limitations:

• The 108-frame window and 144 CQT bins are fixed model contract; only the window count W is dynamic. Inputs shorter than 108 frames must be padded to one window and trimmed back.
• The features must be librosa-equivalent. Substituting a different CQT is not free: an nnAudio CQT1992v2 stand-in correlates at ~0.998 yet still costs ~1.2% of frames end to end. Use the shipped plan.
• The model is gain-sensitive. It was trained on librosa.load's arithmetic mean downmix (L+R)/2. ffmpeg -ac 1 uses an energy-preserving rematrix (L+R)/sqrt(2), i.e. a factor of √2, which log(|CQT| + 1e-6) turns into a constant log(√2) = 0.347 offset on every feature — after std = 1.719 a uniform +0.20 shift, enough to flip frames near a decision boundary. Downmix as the arithmetic mean.
• Its idx_to_chord checkpoint map differs from the reference runner's idx2voca_chord() on 70 of 170 indices, in enharmonic spelling only (Db:min vs C#:min). config.json ships the runner's vocabulary.
• Training-data provenance of the upstream checkpoint is not documented here.

How to use

The session runs the classifier; the host supplies features and consumes logits.

import * as ort from 'onnxruntime-web/webgpu';
import { createCqt, verifyCqtPlanArtifact } from '@musetric/cqt/gpu';

const session = await ort.InferenceSession.create('chordnet.onnx', {
  executionProviders: ['webgpu'],
  preferredOutputLocation: { logits: 'gpu-buffer' },
});
const device = await ort.env.webgpu.device;

// cqt-plan.bin; verifies the payload against the SHA-256 it carries.
const plan = await verifyCqtPlanArtifact(new Uint8Array(planBytes));
const cqt = createCqt(device).get({ input: pcm, output: features, sampleCount, plan });
// cqt.run(encoder) writes log features [T, 144]; pad T up to a multiple of 108.

const input = ort.Tensor.fromGpuBuffer(paddedFeatures, {
  dataType: 'float32',
  dims: [windowCount, 108, 144],
});
const { logits } = await session.run({ features: input });
// logits: float32 [W, 108, 170] -> uniform 9-frame smoothing -> argmax -> indices

See the musetric packages/ai host code for the full CQT, smoothing/argmax and segment-grouping pipeline.

Files

File	Size	SHA256
chordnet.onnx	9,604,664 B	9a6570bf611cdc3f2c36286307af46fb94927fe7f6a2bc22a87c0ebf5f6c082e
config.json	3,009 B	1f26c11ebea51ec08f12e813eb213a729fa0ecc407ac7632dfdc7bad67e65aa4
cqt-plan.bin	23,896 B	c31f0a6fd2d582d753be6628b5daecdee58acba53cba93b2bc2b5c75dee2ba48
cqt-plan.manifest.json	1,721 B	522b178e4f6e8ae5b6bf63b8e2f1a615fe2398592e27f7d9e3e219810081019f

config.json records the I/O contract, checkpoint normalization, the CQT configuration and the 170-label vocabulary. cqt-plan.manifest.json records the plan's generator, configuration and payload hash.

Signature — float32 weights, opset ai.onnx 17:

Tensor	Type	Shape	Meaning
features (in)	float32	[W, 108, 144]	unnormalized log(|CQT| + 1e-6) windows; 108 frames, 144 bins
logits (out)	float32	[W, 108, 170]	per-frame chord logits, before smoothing and argmax

CQT plan — librosa 0.11.0, sr=22050, hop=2048, fmin=C1, n_bins=144, bins_per_octave=24, norm=1, sparsity=0.01, window='hann', scale=True, pad_mode='constant'; 6 octaves after one early downsample, 512-point FFT per octave, resampler kaiser-lowpass-255-cutoff-0.48-beta-12.

Validation

This export + the WebGPU CQT vs the PyTorch + librosa.cqt reference runner:

Metric	Value
per-frame chord agreement (20 instrumental stems)	1.0000
exported logits vs Torch ChordNet, identical inputs	max abs error < 1e-4
degenerate outputs	0

Agreement is exact because the only approximation was removed. The predecessor artifact baked the whole pipeline into one graph with nnAudio CQT1992v2 in place of librosa.cqt; that stand-in was the entire remaining gap (mean 0.9883, worst 0.9410) and cost 70% of inference time and 37.8 of 47.4 MB. Reproducing librosa's recursive per-octave transform on WebGPU fixed accuracy and size at once.

Validate on the material fed in production — the instrumental stem. Agreement measured on audio where the reference emits a near-constant label (for example an isolated vocal, where "no chord" is correct on ~99% of frames) carries no information: a stub returning that label scores just as well. Re-run the parity gate on the exact published bytes before relying on it.

Source & lineage

Code license and weight license are separate; ONNX conversion does not change the weight license. Documented only as far as it is verifiable.

• Architecture: ChordNet "2E1D" — frequency encoder + time encoder + decoder, a small transformer (~2.3 M parameters).
• Reference implementation and weights: ptnghia-j/ChordMini, MIT (per its LICENSE). Upstream publishes no Hugging Face repo, so the weights come from the GitHub repository rather than the Hub.
• Checkpoint: checkpoints/2e1d_model_best.pth — 27,523,646 B, git blob b61f6b3a02cc42b87afa38392f80d185a49f719a — fetched at export time from raw.githubusercontent.com. That URL tracks main and upstream publishes no tagged release, so the fetch follows a moving branch; the blob hash above identifies what this export actually used.
• Vendored code: the inference subset lives under musetric_toolkit/chords_audio/chordmini in musetric-toolkit; see its thirdPartyNotices.md.
• Export tooling: scripts/onnx/chordmini in musetric-toolkit.
• Host runtime: packages/cqt (the CQT) and packages/ai (the session and the smoothing/argmax passes) in musetric.

This export preserves the upstream MIT license; we do not claim authorship of the original weights.

License

MIT, inherited from the upstream weights.