OPEA
/

deepseek-vl2-int4-sym-gptq-inc

4-bit precision

Model card Files Files and versions Community

cicdatopea commited on Jan 3

Commit

0b5077e

·

verified ·

1 Parent(s): 542f2a4

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -28,7 +28,7 @@ vl_gpt: DeepseekVLV2ForCausalLM = AutoModelForCausalLM.from_pretrained(
   model_path,
   trust_remote_code=True,
   device_map="auto",
-  torch_dtype="auto",
 )
 vl_gpt = vl_gpt.eval()

   model_path,
   trust_remote_code=True,
   device_map="auto",
+  torch_dtype=torch.float16,
 )
 vl_gpt = vl_gpt.eval()