Error during basic model loading
#2
by
goodromka
- opened
I got this error when i ran your basic script. Have you seen this problem? I tried this on my server as well as on google colab.
OSError: libcudart.so.12: cannot open shared object file: No such file or directory
my script is :
from ctransformers import AutoModelForCausalLM
llm = AutoModelForCausalLM.from_pretrained("TheBloke/CodeLlama-7B-GGUF", model_file="codellama-7b.Q4_0.gguf", model_type="llama", gpu_layers=50)
print(llm("AI is going to"))
Have you tried the most recent version of the ctransformers package?
Yeah, I've tried. Finally i decided to use langchain for inference.
goodromka
changed discussion status to
closed