license: gpl-3.0 | |
Experiments on training 0.4B RWKV models around midi notation in a manner similar to [this already existing](https://huggingface.co/brianflakes/rwkv-midi-piano) midi model. | |
* RWKV v4neo based, 20 epoch: Loss of about 2.7ish | |
![image/png](https://cdn-uploads.huggingface.co/production/uploads/6352287eef8786433ecdb736/zPg9n76e40lEl-HzF7TvF.png) | |
* WIP v6 pretrain. |