Clarification needed and request for GGUF support for llama.cpp?
Dear VAGOsolutions
If possible it would be great if you can provide two short responses:
1.) Is you model based on Mixtral 8x7B v0.1 or v0.2? The later was released around the time your model came out and contains some bugfixes. I observed some issues with your model in providing unecessary user example output in some cases and there is the chance that this is related to this.
2.) Could you mabye provide a GGUF version for llama.cpp inference of your model? This should be a quick task for you and would drastically increase your user base.
Thanks a lot - any feedback is welcome,
Robert
Hi Robert,
The model was fine-tuned on the 0.1 version. Is there really a 0.2v of the Mixtral? I don't think so
You find GGUF Versions for most of our models in our organization card (Thanks to the bloke, who provided these quantizations)
best regards
Daryoush