openbmb/MiniCPM-Llama3-V-2_5-int4 · report for local try run

May 21

I'm trying to run the model locally. for both MAC M3 chip and windows 2080Ti
the same error occured:
"""
ImportError: Using bitsandbytes 8-bit quantization requires Accelerate: pip install accelerate and the latest version of bitsandbytes: pip install -i https://pypi.org/simple/ bitsandbytes
"""
I've tried to re-install the latest version. unfortunate, it's still happy.
I'm not sure how to fix this.

harshitface2003

May 29

I am facing the same error today, have you fixed it?

stargram

May 31

I encountered the same error, and I resolved it by following these steps:

Execute the command python -c 'import torch; print(torch.version.cuda)'. If it returns None, this indicates that you have installed the CPU-only version of PyTorch without the necessary CUDA runtime.
To rectify this, use the installation commands provided here, ensuring that the installation log confirms the selection of the appropriate binary.
Ensure that the torch version is updated from 2.1.2 to 2.1.2+cu121, and it should works.

Good luck.
Reference