Visual Question Answering
Transformers
Safetensors
internvl_chat
feature-extraction
custom_code
8-bit precision
bitsandbytes

How to use this model?

#1
by bingw5 - opened

I can see there is example code to run the model. But that's for original model. Do I need to modify any parameters or lines to run quantized model?

Owner

Ah, that example code is from the original model card.
The example code of how to run this model is very similar but pointed to this repo. See here: https://huggingface.co/failspy/InternVL-Chat-V1-5-8bit/blob/main/example_inference.py

Running that will be running the quantized model.

I ran into error:

TypeError(\"internvl_chat isn't supported yet.\")

Do you know what's the root cause?

Sign up or log in to comment