TheBloke

/

Mixtral-8x7B-v0.1-GGUF

like 414

Did you make any progress here? I'm running into the same thing - only for ctransformers.
ctransformers can load the old mistral-7b-v0.1-gguf models, but not the new mixtral-8x7b-v0.1-gguf models, even if i modified the model_type to either one of 'gguf', 'llama'.

@eugeneie @hattran

mixtral is not supported by ctransformers yet. its a much different architecture then mistral or llama so you cant use it currently.

use llama cpp python or llama cpp(which ctransformers uses) to load the model instead since its much faster and supports mixtral