Running the model using "pip install auto-gptq" still results in "CUDA extension not installed"
#3
by
mvetter
- opened
I was following Marc Sun's instructions to get this running, which say to use "pip install auto-gptq".
However when loading the model using AutoModelForCausalLM.from_pretrained() (in my case in Jupyter), I still get the "CUDA extension not installed" warning. Twice actually.
The model does run and produce output though, so I'm not sure if there's actually an issue here.