How do you run ggml-model-q4_1.bin in new llama.cpp?
hi, i tried to run it but it failed to load the model. May be I am doing something wrong. How do you run?
$ ./main -m ./models/alpaca/30b/ggml-alpaca-30b-q4_1.bin --color -f ./prompts/alpaca.txt -ins --n_parts 1
main: seed = 1680368465
llama_model_load: loading model from './models/alpaca/30b/ggml-alpaca-30b-q4_1.bin' - please wait ...
llama_model_load: n_vocab = 32000
llama_model_load: n_ctx = 512
llama_model_load: n_embd = 6656
llama_model_load: n_mult = 256
llama_model_load: n_head = 52
llama_model_load: n_layer = 60
llama_model_load: n_rot = 128
llama_model_load: f16 = 3
llama_model_load: n_ff = 17920
llama_model_load: n_parts = 1
llama_model_load: type = 3
llama_model_load: failed to mmap './models/alpaca/30b/ggml-alpaca-30b-q4_1.bin'
llama_init_from_file: failed to load model
main: error: failed to load model './models/alpaca/30b/ggml-alpaca-30b-q4_1.bin'
Oh... Did you use the very latest llama.cpp? When I tested it, it was fine. I'll check again later.
Yes. Have just downloaded and built it.
I just realized, the model is called
ggml-model-q4_1.bin
You put
ggml-alpaca-30b-q4_1.bin
Unless you renamed it
I renamed it
Having trouble as well. Getting "bad magic" with current main.
4_0.bin was probably my favorite model so far, I was excited to see the update as I was completely unable to use their convert unversioned ggml to ggml tool. Guessing because it's 4bit? IDK, but couldn't make it work.
Here's what I got.
./main -m ./models/30b/ggml-model-q4_1.bin
main: seed = 1680402127
llama_model_load: loading model from './models/30b/ggml-model-q4_1.bin' - please wait ...
llama_model_load: invalid model file './models/30b/ggml-model-q4_1.bin' (bad magic)
llama_init_from_file: failed to load model
main: error: failed to load model './models/30b/ggml-model-q4_1.bin'
Bad magic is usually because you're using alpaca.cpp (outdated) or the older version of llama.cpp. Try recloning llama.cpp.
I'm seeing the same "bad magic" error on the latest llama.cpp commit.
Solved: in my case, git lfs was corrupted
Solved: in my case, git lfs was corrupted
Yay I'm glad it worked!