Quantized models in GGUF
#4
pinned
by
MaziyarPanahi
- opened
Hi @v2ray
Thanks for converting and sharing this model, kudos! I am in the process of uploading GGUF models from 16bit all the way down to 2bit, for those with low resources:
https://huggingface.co/MaziyarPanahi/Mixtral-8x22B-v0.1-GGUF
owo
v2ray
changed discussion status to
closed
v2ray
pinned discussion
v2ray
changed discussion status to
open
Hello all,
In LMStudio 0.2.18 : "llama.cpp error: 'illegal split file: 4, model must be loaded with the first split'" for the Q4
Any tip to solve this ?
Thank you.
Does LM Studio supports this new loading split?
Does LM Studio supports this new loading split?
It does support, I can confirm 😊