llama.cpp can not load this gguf
#1
by
keelsike
- opened
error loading model: unknown model architecture: 'qwen2'
llama_load_model_from_file: failed to load model
llama_init_from_gpt_params: error: failed to load model 'models/qwen1_5-14b-chat-q5_k_m.gguf'
This architecture was added there only in January,
so update your llama.cpp.
I am already successfully using llama.cpp with this Q5_K_M gguf.
BTW, the latest text-generation-webui is bundled with recent enough llama.cpp.
Thanks, its OK now.
JustinLin610
changed discussion status to
closed