Not working with KoboldCpp
I downloaded the TheBloke/stablecode-instruct-alpha-3b-GGML ggml v1 8_0.bin version and it did not work with KoboldCpp. I think something needs to be fixed in the model.
Can confirm, LM Studio gives me this
I thought KoboldCpp worked but I guess not. I will ask concedo about that.
LM Studio didn't have support (which I didn't realise), but it's been added in v0.1.13:
I can not load it either on GPT4all
KoboldCpp will soon have automatic support for this. But you can get it working now with:
--forceversion 405
@juanjgit sorry I don't have recent experience of GPT4All so I can't help there. I don't know if they support this model type or not. Please raise it to their team, telling them it's a GPT-NeoX model and asking them if they support that. If they don't now, I imagine they will add support at some point.
Why doesn't this work with Llama.cpp? Is it a different type of GGML?
Yes, it's a GPT-NeoX model, and llama.cpp only currently supports Lllama, Llama 2 and OpenLlama models.
Eventually we hope Llama.cpp will support many model types, but that won't be for at least another month or two.
Thanks for explaining, I've been to their repo and read a bit more about. All sounds super exciting and thanks for all your hard work on this.
TheBloke, do you ever sleep? If you do, I bet you're processing new models even in your dreams lol