metadata
license: gpl-3.0
Experiments on training 0.4B RWKV models around midi notation in a manner similar to this already existing midi model.
- RWKV v4neo based, 20 epoch: Loss of about 2.7ish
- WIP v6 pretrain that also sucks. Loss was around 2.3 to 2.5 but I'm guessing it ended up at 2.5, kind of sad but this can be used as a base I guess?