skytnt

/

midi-model

like 27

@jukofyork Interesting thoughts. Thank you for sharing :)

I have a slightly different idea for music that may work better for LLMs. It's a very early WIP but check it out:
https://github.com/asigalov61/Tegridy-MIDI-Dataset/tree/master/Semantic-Music

If you have any thoughts or ideas similar to this, I would love to hear them and chat more :)

Alex

I did some more searching on and found this guy:

https://www.plymouth.ac.uk/staff/eduardo-miranda
https://en.m.wikipedia.org/wiki/Eduardo_Reck_Miranda

His early-2000s; papers, book chapters and PhD students' theses are a treasure trove of interesting stuff related to this (sadly he now seems to only be interested in "Quantum" stuff though...).

@jukofyork Interesting thoughts. Thank you for sharing :)

I have a slightly different idea for music that may work better for LLMs. It's a very early WIP but check it out:
https://github.com/asigalov61/Tegridy-MIDI-Dataset/tree/master/Semantic-Music

If you have any thoughts or ideas similar to this, I would love to hear them and chat more :)

Alex

Thanks! I stared it to have a look at later.

I don't know if you've read the two articles about chess linked here before:

https://huggingface.co/jukofyork/creative-writing-control-vectors-v3.0/discussions/4

but I think they are quite interesting and possibly useful for (genre) conditional music generation:

• I think the idea is that LLMs get trained on all sorts of different skill-levels of chess, so in general will try to pick a move that is sort of "average" for the dataset, and there are likely many orders if magnitude more crappy/amateur players' game histories in the training data than good players, so the "skill vector" he finds is really just a way of conditioning the output to be biased towards higher-skilled players (in a similar way to my creative writing control vectors conditioning the output to be more/less like 1 of the 8 traits I trained for).
• So for music specifically; this might let you condition the output to be biased towards a certain genre (like techno) without any real changes to the modeling approach (ie: find the "techno" direction and then apply it to this exact model in this repo to filter out almost everything else).

I can't say for sure if it would work but I think there is quite a good chance, as I noticed you can "seed" the model in this repo with just the first handful of midi events for something like "Born Slippy" and it will be much more conditioned to output music in the "electronic" music genre and way less in the "chip tune" genre of late 90s games (which I assume, like crappy/amateur chess players in the chess data; is massively over represented in the midi datasets?).

@jukofyork Thank you for starring and thank you for your thoughts.

Yes, adjusting the vectors may very well work but the results would probably would only be "slightly better" (read - not significant) because like you said that its the data that is mostly crappy. Even with people, there are very few who could think truly creatively or critically, especially of oneself.

What I tried with music (and what you can try too with your models) is to train/fine-tune on the labeled data. The labels must be precise and dense though or it won't work.
I.e I wanted to teach the model to generate only good chords, so I labeled each train data chord with label (good/bad). The label was also prefixed so I could condition on it upon generation. This produced very nice results in terms or generating only good chords after training.

You can see how this was implemented here:
https://huggingface.co/spaces/asigalov61/Ultimate-Chords-Progressions-Transformer

Otherwise, if you want to play with vectors/neurons, it may also be a good idea but it would be very difficult because its a NN, not an algorithm or a code.

Hope any of this makes sense to you.

Alex

Thanks - I'll take a look!