Junnuo/BPE-on-AMT-MIDI

BPE on AMT MIDI: checkpoints

SmolLM2-135M models for caption-conditioned MIDI generation, trained with and without byte-pair encoding (BPE) over AMT's five-token note representation. Code, training details and results: github.com/Junnuo3/BPE-on-AMT-MIDI.

Folder	Tokenizer	Best stage-2 step
no_bpe/stage2	no BPE (vocab11293)	35,200
bpe/stage2	BPE, 16,384 merges (vocab27677)	38,400

Both are final stage-2 (caption → MIDI) checkpoints, each the one with the lowest validation loss of its run, after stage-1 continued pretraining on MusicPile and GigaMIDI. amt_config.json points to the AMT tokenizer files in the GitHub repository, which the generation scripts need for decoding.

git clone https://github.com/Junnuo3/BPE-on-AMT-MIDI && cd BPE-on-AMT-MIDI
hf download Junnuo/BPE-on-AMT-MIDI --local-dir checkpoints
python scripts/generate_smollm2.py --checkpoint checkpoints/bpe/stage2 \
    --caption "A calm piano piece in C major with a slow tempo." --top-p 1.0 --top-k 40

Decoding settings selected on the validation split: BPE T = 1.0, top-p = 1.0, top-k = 40; no BPE T = 1.0, top-p = 1.0, no top-k.

The training recipe follows MIDI-LLM (Wu, Kim and Huang, 2025), and the note representation and anticipation augmentation follow the Anticipatory Music Transformer (Thickstun et al., 2024). The models were trained on Lakh MIDI, MidiCaps, GigaMIDI and MusicPile; check those datasets' terms before use.