Looks like the starchat-alpha-ggml-q4_1.bin is broken
The other models work, this one is messed up:
starcoder_model_load: unknown tensor '' in model file
I also did verify hashes and I'm 100% sure I downloaded the file correctly. It seems that one was quantized to a corrupt file, could you please reupload it?
Sure.
Looks like it's a bug in upstream https://github.com/ggerganov/ggml
Requantization on latest code from ggml creates the same file, git lfs doesn't see any changes.
I was using starcoder.cpp, I'll try pulling the ggml
But if it's a bug upstream, it's worth reporting, since you're running quantization would you mind filing an issue on the ggml repo
I will try if I can reproduce issue.
https://github.com/bigcode-project/starcoder.cpp/issues/11
Incompatibility between starcoder.cpp and upstream ggml. Try to use https://github.com/ggerganov/ggml/tree/master/examples/starcoder for inference while issue not solved.
For now koboldcpp supports starcoder gglm models.