AIDC-AI
/

Ovis1.6-Gemma2-9B-GPTQ-Int4

Image-Text-to-Text

text-generation

4-bit precision

Model card Files Files and versions Community

TryingHard commited on 16 days ago

Commit

da09c52

•

1 Parent(s): d034c7a

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -66,7 +66,7 @@ from auto_gptq.modeling import OvisGemma2GPTQForCausalLM
 # load model
 load_device = "cuda:0" # customize load device
-model = OvisGemma2GPTQForCausalLM.from_pretrained(
     "AIDC-AI/Ovis1.6-Gemma2-9B-GPTQ-Int4",
     device=load_device,
     trust_remote_code=True

 # load model
 load_device = "cuda:0" # customize load device
+model = OvisGemma2GPTQForCausalLM.from_quantized(
     "AIDC-AI/Ovis1.6-Gemma2-9B-GPTQ-Int4",
     device=load_device,
     trust_remote_code=True