Can't load in LM studio
Just tried it with it LM studio and got this error loading it. Any solution?
"llama.cpp error: 'error loading model vocabulary: unknown pre-tokenizer type: 'mistral-bpe''"
Its not supported by llamacpp yet, so anything based on llamacpp such as KoboldCpp won't be able to run this.
Other than llamacpp and it's derivatives, what else supports gguf quants?
Quoting GTP4o :
"While the Mistral-Nemo-Instruct-2407-GGUF model is not currently supported by llama.cpp and hence cannot be run on LMStudio, you have several other options. Using the Hugging Face transformers library directly, converting the model for use with ONNX Runtime, leveraging cloud-based services like AWS SageMaker, or setting up a local Docker environment are all viable alternatives to run this model."
Disclaimer : I haven't tried any of the above options, though i'm inclined to try it with Docker and transformers.