miditizer v1 — polyphonic piano transcription

miditizer transcribes solo-piano audio (wav, mp3, flac) into notes (pitch, onset, offset, velocity) and writes MIDI and JSON. The model is an original architecture trained from scratch on MAESTRO v3. It ships with its full training recipe, its evaluation protocol, and the error characterization behind every number reported here.

Checkpoint: miditizer-v1.pt (385 MB), sha256 fc5a037dcbaab3705d4d8c0ee4193c452f9b7280984f3a07ae705f2b550ff62f

Results

MAESTRO v3 validation (137 held-out pieces), sustain-extended label convention, frozen harness v1.1.0. "note-F1@0.2" = a note counts as correct when pitch matches, onset is within 50 ms, and the offset is within max(20 % of the reference note's duration, 50 ms) (mir_eval-style matching, comparable to Transkun's reporting convention). "onset-F1" uses the same 50 ms tolerance without the offset condition.

metric	value
note-F1@0.2	0.7999 (95 % CI [0.787, 0.813], 137-piece bootstrap)
onset-F1	0.9213
frame-F1	0.9039

Ablations (each row is one controlled change, measured as a paired per-piece difference with a 10,000-resample bootstrap CI on the same 137 pieces; ¢ = hundredths of note-F1@0.2):

change	Δ note-F1@0.2	95 % CI
focal → ASL onset loss	+1.33¢	[+1.09, +1.58]
SemiCRF loss weight 0.5 → 2.0 (the released setting)	+1.48¢	[+1.28, +1.69], 126/137 pieces improve
SemiCRF loss weight 2.0 → 4.0	+0.28¢	[+0.14, +0.43]

The structured-loss weight saturates above 2.0; the released setting sits near the optimum of that curve.

Characterization (measured on the released checkpoint):

• Recipe re-execution (n=1: one fresh seed and execution path, identical recipe): Δ = −0.03¢, CI spans zero.
• Decode operating point: a 9-point onset-logit bias sweep puts the shipped θ=0 at the top of the grid; decode-scale sweeps find no setting beating the shipped one by more than bootstrap noise (best alternative +0.015¢).
• Known deficit: soft-note onsets (the quietest velocity third) score onset-F1 0.626 vs 0.921 overall, and 77 % of "too long" offset errors sit on reference notes shorter than 0.35 s. Loss-side interventions on the soft-note slice were measured and did not move it; we classify this deficit as architectural.

Context for these numbers: on comparable conventions, published systems include Kong 2020 (0.8247, raw note-off), Hawthorne 2021 seq2seq (0.8394, sustain-extended), and Transkun (0.9348 on the MAESTRO test split, sustain-extended; ISMIR 2024, arXiv:2404.09466 — our numbers are validation).

Architecture (19.26M parameters)

• Front-end: 229-bin log-mel spectrogram, 62.5 fps, 16 kHz mono.
• Trunk: 5-block CNN funnel (64→512 ch, residual, stride-2 frequency) into an 8-layer / 8-head Transformer encoder (d_model 384, dff 1536, RoPE positional encoding).
• Heads: onset / frame / offset / release (binary logits) + velocity (sigmoid-scaled 1–127).
• Decoding: Semi-CRF segment model — Viterbi over note segments with a learned duration prior, onset/frame/offset probabilities as potentials (offset scaled 0.5), sliding 10 s windows at 50 % overlap (max-stitching, mean velocity), dedup and duration floors. The SemiCRF is both a training loss (its gradients shape the heads) and the decoder at inference.

Training

• Data: MAESTRO v3.0.0 train split (962 pieces, ~159 h of the dataset's ~199 h; Hawthorne et al., 2019). Model selection used the validation split only; the released checkpoint has never been evaluated on the test split.
• Recipe: ASL onset loss (γ−=2, m=0.1), focal frame loss, SemiCRF structured offset loss (weight 2.0), masked-L1 velocity; frequency-only SpecAugment; 45 epochs cosine schedule, batch 16 × 10 s windows, seed 20260728; one RTX 5090, ~14 GPU-hours per run (training and in-training validation).
• Provenance: every training run in the program is logged (config hash, code commit, metrics); the released checkpoint is epoch 39 of the training run whose MLflow id is embedded in the file (internal run 8354c373…; not in this repository).

Intended use

Offline transcription of solo piano recordings — musicology, practice tooling, dataset labeling, creative MIDI manipulation. Evaluation is MAESTRO-only (Disklavier pianos in controlled competition conditions); expect degradation on non-piano instruments, heavily produced mixes, or noisy field recordings. This is an offline system; streaming/causal use is future work.

Data

Trained exclusively on MAESTRO v3.0.0 (Hawthorne et al., 2019; CC-BY-NC-SA 4.0): ~200 h of paired piano audio and aligned MIDI from International Piano-e-Competition performances. See Intended use for scope notes.

License

CC-BY-NC-SA 4.0 — non-commercial use with attribution and share-alike. We apply the dataset's restrictions to the checkpoint as policy (whether trained weights legally inherit dataset licenses is unsettled; we take the conservative reading).

Limitations

• Quiet notes are under-detected (see Known deficit above); dense pedal textures accumulate offset errors on short notes.
• Sustain-pedal convention: labels and outputs reflect acoustic note sustain including pedal, not raw MIDI note-off. Comparisons against systems using a different convention are not apples-to-apples.
• Velocity is a relative loudness estimate, not a calibrated strike force.

Citation

@software{miditizer_v1_2026,
  title  = {miditizer v1: a piano transcriber with a RoPE Transformer
            trunk and SemiCRF-as-loss},
  author = {Krystian Lewandowski},
  year   = {2026},
  url    = {https://github.com/elewarr/miditizer},
  note   = {Note-F1@0.2 0.7999 / onset-F1 0.9213 on MAESTRO v3
            validation, sustain-extended convention},
}

Usage

pip install torch numpy librosa soundfile mido pretty_midi
git clone https://github.com/elewarr/miditizer && cd miditizer && pip install .
python examples/transcribe.py input.wav output.mid \
  --ckpt miditizer-v1.pt --device cuda --json-out output.json

The example is self-contained: it loads the checkpoint, runs the sliding-window + SemiCRF decode, and writes type-1 MIDI plus a JSON note list. On the released checkpoint it produces the same notes the frozen evaluation harness scored on held-out pieces.