neuralmagic
/

Mixtral-8x7B-Instruct-v0.1-FP8

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

mgoin commited on Jun 12

Commit

54e6f9f

•

1 Parent(s): 28b8603

Update README.md

Files changed (1) hide show

README.md +3 -3

README.md CHANGED Viewed

@@ -19,12 +19,12 @@ Produced using [AutoFP8 with calibration samples from ultrachat](https://github.
 | :------------------: | :----------------------: | :------------------------------------------------: |
 | arc-c<br>25-shot     | 71.50                    | 71.08                                              |
 | hellaswag<br>10-shot | 87.53                    | 87.38                                              |
-| mmlu<br>5-shot       | 70.33                    | 68.81                                              |
 | truthfulqa<br>0-shot | 64.79                    | 64.20                                              |
 | winogrande<br>5-shot | 82.40                    | 82.40                                              |
 | gsm8k<br>5-shot      | 64.36                    | 64.06                                              |
-| **Average<br>Accuracy**  | **73.48**                    |              **72.99**                                     |
-| **Recovery**             | **100%**                     |              **99.33%**                                     |

 | :------------------: | :----------------------: | :------------------------------------------------: |
 | arc-c<br>25-shot     | 71.50                    | 71.08                                              |
 | hellaswag<br>10-shot | 87.53                    | 87.38                                              |
+| mmlu<br>5-shot       | 70.33                    | 70.00                                              |
 | truthfulqa<br>0-shot | 64.79                    | 64.20                                              |
 | winogrande<br>5-shot | 82.40                    | 82.40                                              |
 | gsm8k<br>5-shot      | 64.36                    | 64.06                                              |
+| **Average<br>Accuracy**  | **73.48**                    |              **73.19**                                     |
+| **Recovery**             | **100%**                     |              **99.61%**                                     |