---
tags:
- midi
- music
---

step650 model was trained in 48hr on an RTX3090, 
step1100 model initially experimentally trained to add tiny attenion (RWKV-4b), but then finetuned again to remove it. Final finetuning was done with ctx 4096.


Trained using GiantMIDI-Piano dataset, augmented 30x. See training history: https://wandb.ai/brianflakes/rwkv-midi

To run, use the rwkv pip package: https://pypi.org/project/rwkv/

To convert outputs to MIDI, use https://github.com/briansemrau/MIDI-LLM-tokenizer