README.md

metadata

license: gpl-3.0

Experiments on training 0.4B RWKV models around midi notation in a manner similar to this already existing midi model.

• RWKV v4neo based, 20 epoch: Loss of about 2.7ish

• WIP v6 pretrain.