Intel
/

Mistral-7B-v0.1-int4-inc

Text Generation

text-generation-inference

Inference Endpoints

4-bit precision

Model card Files Files and versions Community

wenhuach commited on Apr 28

Commit

9a45a0e

•

1 Parent(s): a8d0e05

Update README.md

Files changed (1) hide show

README.md +1 -7

README.md CHANGED Viewed

@@ -25,14 +25,8 @@ python3 main.py \
 --group_size 128 \
 --bits 4 \
 --iters 1000 \
---enable_minmax_tuning \
---enable_quanted_input \
 --deployment_device 'gpu' \
---scale_dtype 'fp32' \
---eval_bs 32 \
---output_dir "./tmp_autoround" \
---amp
 ```

 --group_size 128 \
 --bits 4 \
 --iters 1000 \
 --deployment_device 'gpu' \
+--output_dir "./tmp_autoround"
 ```