Kyllene and MergeMonster
Undi, Ikari, I think it would be interesting for you (and thus, for us all) to try and test this Model :
https://huggingface.co/Nexesenex/TeeZee_Kyllene-Yi-34B-v1.1-iMat.GGUF
And compare it to the usual Yi models & merges, while focusing also on the merge technique used.
I don't know if you're familiar with Gryphe MergeMonster already, but I clearly see an improvement, natural output-wise, compared to the already good merges of BruceTheMoose on the Yi 34b segment. The clearing of any problematic gptism, llamaism, or yiism which was specified to MergeMonster is noticeable, and it's like the model is freed of these sequences which represent some form of "EOS chains of tokens" in many models, in the sense that they conclude many outputs, this ofc in an unwanted way.
The merge parameters and logs are in the repo : https://huggingface.co/TeeZee/Kyllene-34B-v1.1/tree/main
I was really impressed, and I think that this trimming practice of unwanted sequences during a merge should become the standard.
We tried mergemonster and had okay-ish results, maybe we will get back to it soon.