libbitsandbytes_cpu.so undefined symbol: cget_col_row_stats
Hi, there. Does anyone have problem with loading this model:
the code:
import torch
from transformers import GenerationConfig, LlamaTokenizer, LlamaForCausalLM
tokenizer = LlamaTokenizer.from_pretrained("chainyo/alpaca-lora-7b")
model = LlamaForCausalLM.from_pretrained(
"chainyo/alpaca-lora-7b",
load_in_8bit=True,
torch_dtype=torch.float16,
device_map="sequential",
)
the console logs:
CUDA SETUP: Highest compute capability among GPUs detected: 7.5
CUDA SETUP: Detected CUDA version 117
CUDA SETUP: Loading binary /mnt/data/renyanda/code/starwhale/example/LLM/alpaca7b/venv/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cpu.so...
Could you try to reinstall PyTorch in your working environment?
Something like:
pip install --force-reinstall --index-url https://download.pytorch.org/whl/nightly/cu118
I think I faced this problem when PyTorch wasn't installed with CUDA, or I didn't have PyTorch in my environment.
Could you try to reinstall PyTorch in your working environment?
Something like:
pip install --force-reinstall --index-url https://download.pytorch.org/whl/nightly/cu118I think I faced this problem when PyTorch wasn't installed with CUDA, or I didn't have PyTorch in my environment.
Thank you for the answer!pip install --force-reinstall --index-url https://download.pytorch.org/whl/nightly/cu118 couldn't solve this problem in my case.
I finally managed to solve this problem by changing my python environment from venv to conda. And install cudatoolkit in conda.
The reason why it happens maybe that bitsandbytes could not find libcudart.so, so it chooses libbitsandbytes_cpu.so rather than libbitsandbytes_cudaxxx.so. libcudart.so could not be found in venv bin path while cudatoolkit could make it visiable in conda bin path.