Just an idea

#1
by distantquant - opened

Have you thought about merging this with ShinojiResearch/Senku-70B-Full?

It may make a good model.

Lodrick_and_Lily org

Great idea, I will definitely try that. I was not aware of Senku yet. Swap merging with a light finetune perfectly fits our hypothesis about attempting to restore quantization error by using weights from other models.

I tried a few merges with MiquMaid x Quartet already, and they were disappointing.

senku beat an early version of GPT4 on eq-bench (https://eqbench.com/) so yeah it may have a good potential

Sign up or log in to comment