cognitivecomputations

/

dolphin-2.6-mistral-7b-dpo-laser

like 115

Thanks a lot for sharing the model :)
I've tested the gguf version by
kroonen/dolphin-2.6-mistral-7b-dpo-laser-GGUF.
It performs almost better on cpu (in terms of speed) than the LMCocktail-phi-2 version I'm testing right now, which has only 3b parameters.

I'm also very satisfied with the responses so far.

Would it be possible to share how to build these laser versions?
I would really like to see this on the current top performers on the llm leaderboard with 10b parameters, I bet you could speed them up to the speed of 7b models. And 3b models like phi-2 could be blazingly fast on any device.

Excited to see more of this ^^

Thanks again