Failed to create LLM 'falcon
#2
by
asmacats
- opened
Hello,
i'am trying to run fthis below code
from ctransformers import AutoModelForCausalLM
Set gpu_layers to the number of layers to offload to GPU. Set to 0 if no GPU acceleration is available on your system.
llm = AutoModelForCausalLM.from_pretrained("TheBloke/alfred-40B-1023-GGUF", model_file="alfred-40b-1023.Q4_K_M.gguf", model_type="falcon", gpu_layers=50)
print(llm("AI is going to"))
i have got this error
RuntimeError: Failed to create LLM 'falcon' from '/home/jupyter-atrabels/.cache/huggingface/hub/models--TheBloke--alfred-40B-1023-GGUF/blobs/33236825912d58e06fdb4f3e83ca30f222b7305957231aa9a247bed58f7c90a0'.
Any help be to resolve this problem? have i to install specific version of ctransformers ?