TheBloke

/

Llama-2-7B-Chat-GGUF

like 378

from llama_cpp import Llama

llm = Llama(model_path="llama-2-7b-chat.Q2_K.gguf", n_ctx=512, n_batch=126)

Gives the error:

gguf_init_from_file: invalid magic number 4f44213c
error loading model: llama_model_loader: failed to load model from llama-2-7b-chat.Q2_K.gguf

llama_load_model_from_file: failed to load model

I changed from the format GGML to GGUF and thought it would resolve the error but it did not.

I have the llama-2-7b-chat.Q2_K.gguf file completely downloaded and can also access it (path is correct).

Any idea?