Intel
/

phi-2-int4-inc

Text Generation

text-generation-inference

Inference Endpoints

4-bit precision

Model card Files Files and versions Community

yintongl commited on May 21

Commit

60f2919

•

1 Parent(s): 559c322

Update README.md

Files changed (1) hide show

README.md +11 -12

README.md CHANGED Viewed

@@ -47,20 +47,19 @@ Since we encountered an issue evaluating this model with lm-eval, we opted to ev
-| Metric         | FP16   | INT4 qdq   |
 | -------------- | ------ | ------ |
-| Avg.           | 0.6155 | 0.6163 |
-| mmlu           | 0.5448 | 0.5417 |
-| lambada_openai | 0.6268 | 0.6225 |
-| hellaswag      | 0.5585 | 0.5498 |
-| winogrande     | 0.7530 | 0.7545 |
 | piqa           | 0.7867 | 0.7824 |
-| truthfulqa_mc1 | 0.3133 | 0.3060 |
-| openbookqa     | 0.4000 | 0.4100 |
-| boolq          | 0.8339 | 0.8327 |
-| rte            | 0.6245 | 0.6643 |
-| arc_easy       | 0.7997 | 0.7955 |
-| arc_challenge  | 0.5290 | 0.5196 |
 ### Reproduce the model

+| Metric         | BF16   | INT4 qdq   |
 | -------------- | ------ | ------ |
+| Avg.           | 0.6138 | 0.6115 |
+| mmlu           | 0.5325 | 0.5417 |
+| lambada_openai | 0.6276 | 0.6225 |
+| hellaswag      | 0.5584 | 0.5498 |
+| winogrande     | 0.7561 | 0.7545 |
 | piqa           | 0.7867 | 0.7824 |
+| truthfulqa_mc1 | 0.3146 | 0.3060 |
+| openbookqa     | 0.4020 | 0.4100 |
+| boolq          | 0.8330 | 0.8327 |
+| arc_easy       | 0.7992 | 0.7955 |
+| arc_challenge  | 0.5282 | 0.5196 |
 ### Reproduce the model