newest llama.cpp seems to crash now
This is the newer model file you posted recently. This worked previously.
../llama.cpp/main --model ./models/ggml-vic13b-uncensored-q8_0.bin
main: build = 588 (ac7876a)
main: seed = 1684995027
llama.cpp: loading model from ./models/ggml-vic13b-uncensored-q8_0.bin
libc++abi: terminating due to uncaught exception of type std::runtime_error: unexpectedly reached end of file
zsh: abort ../llama.cpp/main --model ./models/ggml-vic13b-uncensored-q8_0.bin
Indeed, q4, 4_1, and 8 quantization formats have had breaking changes. Will update affected files asap
Thank you so much, I really like this model
Having a ggml F16 version of the files would be nice, than we could also requantize when new changes are coming.
Yeah, good idea. Will upload in a sec
Indeed, q4, 4_1, and 8 quantization formats have had breaking changes. Will update affected files asap
difference between q4, q4_1, q4_2 etc.
also: q4 = 4bits, q8 = 8bits is that right?
Correct. This is an obsolete model btw