ssuncheol
/

Phi-3-mini-128k-instruct-int4

Text Generation

text-generation-inference

Inference Endpoints

4-bit precision

Model card Files Files and versions Community

ssuncheol commited on Apr 27

Commit

acf884a

•

1 Parent(s): 3c7a39d

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -49,7 +49,7 @@ Here's an simplified code for quantization. In order to save memory in quantizat
 ```
 python main.py \
   --model_name "microsoft/Phi-3-mini-128k-instruct" \
-  --bits 8 \
   --group_size 128 \
   --train_bs 1 \
   --gradient_accumulate_steps 8 \

 ```
 python main.py \
   --model_name "microsoft/Phi-3-mini-128k-instruct" \
+  --bits 4 \
   --group_size 128 \
   --train_bs 1 \
   --gradient_accumulate_steps 8 \