latest llama.cpp using q5_0, q4_1 error: "is this really a GGML file?"
#5
by
Free-Radical
- opened
I am using latest llama.cpp (master-66874d4) and it only works for:
- ggml-vicuna-13b-4bit.bin
but NOT for:
- stable-vicuna-13B.ggmlv3.q5_0.bin
- stable-vicuna-13B.ggmlv3.q4_1.bin
Please check the sha256sum for the stable-vicuna-13B.ggmlv3.q5_0 and q4_1 files, or if in doubt download them again.
Those two files definitely work with the latest llama.cpp, so you likely have incomplete/corrupted downloads. (In fact the q5_0 file would work with older llama.cpp as well - only q4_0, q4_1 and q8_0 require the latest llama.cpp)
I just re-downloaded and tested q5_0 with llama.cpp compiled today and confirmed it works OK.
Ok , MY BAD, 🫢 very embarrassed, i forgot to recompile. YES THEY WORK!
Free-Radical
changed discussion status to
closed