open-fret-tab — tiny T5 for MIDI-to-guitar-tablature

Maps a MIDI note stream to a playable guitar fingering: input tokens (CAPO_*, TUNING_*, NO_*/NF_* on/off events, TS_* time shifts) map to output tokens (TAB_<string>_<fret> attacks, TS_* gaps, techniques). The model predicts where to play, never what — pitch and timing come from the input, and the reference decoder emits exactly one fingering slot per input attack.

Training info

• Architecture. T5 encoder-decoder (d_model 128, d_ff 1024, 3 + 3 layers, 4 heads) with a 6,631-token WordLevel vocabulary, trained from scratch (no pretrained init). ~14 MB in safetensors.
• Run. full-pos05-stretch05-3ep-20261110 (open-fret @ 20fa647), RTX 4090 fp16, seed 42, Optuna study full-pos05-stretch05-ce03-pitch1-smooth01-type03-v2. Full hyper-parameters, loss config and tokenizer hash are in training_info.json and in the checkpoint config.json (open_fret_loss, open_fret_token_metadata).
• Schedule. The dev-HPO winner (12 Optuna trials): lr $5.7\times10^{-4}$, dropout 0.144, batch 128 × grad-accum 2, 10k-example warmup, ×0.1 decay after 100k examples.
• Checkpoint status. Completed 3-epoch full run (step 9,891) — final checkpoint.
• Objective. $0.3\cdot(\mathrm{CE} + \Delta_{\mathrm{smooth}}) + 1.0\cdot\mathrm{pitch}_{\mathrm{cond}} + 0.3\cdot\mathrm{type} + 0.5\cdot\mathrm{position} + 0.5\cdot\mathrm{stretch}$ (span weight 0.0, disabled) with $\varepsilon = 0.1$ equivalence smoothing over same-pitch fingerings. Details: project docs on losses.
• Loss difference vs previous release (full-curated-3ep-20261109). Same corpus (loss-v4-curated-20261109), vocabulary (6,631), architecture, schedule and base weights ($0.3$ CE, $1.0$ conditional pitch, $0.1$ smoothing, $0.3$ event-type). New in this release:
  • Position term ($0.5$): mean $|E[\mathrm{fret}] - \mathrm{target}|$ over fretted targets (open strings excluded), where $E[\mathrm{fret}]$ is the expected fret from the per-string candidate TAB distribution.
  • Stretch term ($0.5$, margin 5): mean $\mathrm{relu}(|E[\mathrm{fret}]
    • \mathrm{localMean}_{\pm5}| - 5)$ over fretted notes in rows with at least 2 notes; the local mean is the ±5-note window mean of target frets in sequence order.
  • Span term ($0.0$, margin 4): per-onset expected hand-span hinge $\mathrm{relu}(\max E[\mathrm{fret}] - \min E[\mathrm{fret}] - 4)$, kept at zero (inactive) in this checkpoint.
  • Base terms unchanged: standard T5 CE over the full vocabulary; $\Delta_{\mathrm{smooth}}$ is the target-minus-alternative log-prob gap (target vs mean over plain same-pitch alternatives, technique-free TAB targets only); $\mathrm{pitch}{\mathrm{cond}} = -\log\sum{\mathrm{equiv}} p + \log\sum_{\mathrm{playable}} p$ (correct-pitch mass given playable TAB, so TS/EOS pressure stays in CE and the type loss); $\mathrm{type}$ is the macro-averaged 3-class (note / TS / EOS) log-sum-exp loss.
• Latest teacher-forced eval. eval_loss 3.30 (ce 2.18, pitch 1.72, position 1.13, stretch 0.024, span 0.0). These are training diagnostics — scalar values of different objectives are not comparable, and selection used a frozen five-window validation panel with anchored generation, playable viewers and listening. The held-out test split was cached but never used for tuning.

Training data (provenance)

• Source. vldsavelyev/guitar_tab at revision e6a6dbbccf7cad2e057770cae5ebf8257ec36382 (GuitarPro tablatures parsed with AlphaTab 1.3.0).
• Corpus. loss-v4-curated-20261109: 843,886 train / 105,394 val / 104,920 test windows, family-disjoint splits, vocab conditioning lists (zero [UNK]), group-level masking of same-onset/same-string reference conflicts (441k windows, 23.6M note tokens), train-only capo augmentation.
• Rights status (please read). The dataset card and loader declare no license, and the upstream converter repository carries none either, so training-data rights are not established by the published material. Public access is not a rights grant. The full evidence trail is docs/dataset_licensing.md. This model is published by the open-fret maintainer with that caveat stated; the mit repository license covers the code/config written for this release and does not by itself grant rights to the underlying music or training data. Commercial or redistribution use should resolve data rights first (clearer-licensed alternatives are tracked in issue #22).

How to use

from transformers import AutoTokenizer, T5ForConditionalGeneration

tok = AutoTokenizer.from_pretrained("mylonasc/open-fret-tab")
model = T5ForConditionalGeneration.from_pretrained("mylonasc/open-fret-tab")

# One attack: E4 on string 1 at tick 0, off at tick 240 (standard tuning)
src = "CAPO_0 TUNING_64_59_55_50_45_40 NO_64 TS_240 NF_64"
ids = tok(src, return_tensors="pt").input_ids
out = model.generate(ids, max_new_tokens=32,
                     num_beams=4, no_repeat_ngram_size=3, early_stopping=True)
print(tok.decode(out[0], skip_special_tokens=True))
# TAB_1_0 TS_240 TAB_1_0 TS_240  (open high-E string)

Tokens: TAB_<string>_<fret> (1–6, high string first; optional _<technique>), TS_<ticks> gaps (480 = one beat), [EOS] ends the sequence. For faithful decoding use the repo's anchored generator (scripts/inference.py), which enforces one slot per input attack, pitch-equivalent masking and distinct strings per onset instead of free generate.

Limitations

• Same-onset/same-string reference conflicts are masked in training, never learned; the strict renderer refuses to draw them.
• No isolated ablations yet — the combined objective has no same-decoder CE-control comparison on generated-note metrics.
• Bends/slides need contour modeling the pitch marginal does not provide; timing/occupancy are supervised only through CE/type, and playability is regularized only via the position/stretch terms above.
• Scalar eval losses are not comparable across objectives (previous release eval_loss 2.19 vs this release 3.30 reflects the added terms, not quality).