TheBloke/Llama-2-70B-Chat-GGUF does not appear to have a file named pytorch_model.bin, tf_model.h5, model.ckpt or flax_model.msgpack.
#6
by
senthilknzanz
- opened
llama_model = pipeline("text-generation", model="TheBloke/Llama-2-70B-Chat-GGUF", token=access_token)
for the above code, I get the following error:
OSError: TheBloke/Llama-2-70B-Chat-GGUF does not appear to have a file named pytorch_model.bin, tf_model.h5, model.ckpt or flax_model.msgpack.
Facing same error using Pipeline
@swathiKonakanchi
this does not work with huggingface transformers pipeline since it doesnt support gguf quanted models
use llama cpp python or ctransformers as it supports gguf models.