Intel
/

Qwen2-1.5B-Instuct-int4-inc

Text Generation

text-generation-inference

Inference Endpoints

4-bit precision

Model card Files Files and versions Community

lvkaokao commited on Jun 6

Commit

e440629

•

1 Parent(s): 7f7d841

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -42,7 +42,7 @@ lm_eval --model hf --model_args pretrained="Intel/Qwen2-1.5B-Instuct-int4-inc" -
-| Metric-1.5B-instruct | BF16   | W4G32 sym |
 | -------------------- | ------ | --------- |
 | Avg                  | 0.5518 | 0.5474    |
 | mmlu                 | 0.5565 | 0.5499    |

+| Metric-1.5B-instruct | BF16   | INT4 |
 | -------------------- | ------ | --------- |
 | Avg                  | 0.5518 | 0.5474    |
 | mmlu                 | 0.5565 | 0.5499    |