Are the GGML files gpt4all compatible?
#2
by
Hoioi
- opened
Can i use these ggml files with gpt4all or koboldcpp?
You can use these with anything compatible with the newer version of llama.cpp, so yes to GPT4All, and idk for koboldcpp I haven’t used that.
That's nice. I was waiting for the q5_K_M version and since it wasn't being uploaded, I would like to know is it possible that you release the q6_k version too? Actually q6_k has more accuracy than q5 versions and of course it uses more RAM.
q5_K_M quantization is incompatible with this model, currently, I'm looking into a way to hack it together because the naming schema is wrong. I can try for q6_k, yes.
If the vocabulary size of the model is 32001,both q5_K_M and q6_k are not possible to be generated. Is the vocabulary size 32001?
Hoioi
changed discussion status to
closed