Are the GGML files gpt4all compatible?

by Hoioi - opened

Can i use these ggml files with gpt4all or koboldcpp?

You can use these with anything compatible with the newer version of llama.cpp, so yes to GPT4All, and idk for koboldcpp I haven’t used that.

That's nice. I was waiting for the q5_K_M version and since it wasn't being uploaded, I would like to know is it possible that you release the q6_k version too? Actually q6_k has more accuracy than q5 versions and of course it uses more RAM.

q5_K_M quantization is incompatible with this model, currently, I'm looking into a way to hack it together because the naming schema is wrong. I can try for q6_k, yes.

If the vocabulary size of the model is 32001,both q5_K_M and q6_k are not possible to be generated. Is the vocabulary size 32001?

Hoioi changed discussion status to closed

Sign up or log in to comment