Update README.md

README.md

@@ -1,21 +1,16 @@
 ---
 tags:
 - fp8
+- vllm
 ---
 
-Mixtral-8x7B-Instruct-v0.1 quantized to FP8 weights and activations, meant to be deployed in vLLM.
+# Mixtral-8x7B-Instruct-v0.1-FP8
 
-Accuracy on MMLU:
-```
-vllm (pretrained=nm-testing/Mixtral-8x7B-Instruct-v0.1-FP8), gen_kwargs: (None), limit: None, num_fewshot: 5, batch_size: 1
-|      Groups      |Version|Filter|n-shot|Metric|Value |   |Stderr|
-|------------------|-------|------|-----:|------|-----:|---|-----:|
-|mmlu              |N/A    |none  |     0|acc   |0.7008|±  |0.0036|
-| - humanities     |N/A    |none  |     5|acc   |0.6453|±  |0.0065|
-| - other          |N/A    |none  |     5|acc   |0.7692|±  |0.0072|
-| - social_sciences|N/A    |none  |     5|acc   |0.8083|±  |0.0070|
-| - stem           |N/A    |none  |     5|acc   |0.6115|±  |0.0083|
-```
+## Model Overview
+Mixtral-8x7B-Instruct-v0.1 quantized to FP8 weights and activations, ready for inference with vLLM >= 0.5.0.
+
+## Usage and Creation
+Produced using [AutoFP8 with calibration samples from ultrachat](https://github.com/neuralmagic/AutoFP8/blob/147fa4d9e1a90ef8a93f96fc7d9c33056ddc017a/example_dataset.py).
 
 Quantized using the script below:
 
@@ -313,4 +308,23 @@ if __name__ == "__main__":
 
         print("Exporting model with static weights and static activations")
         save_quantized_model(model, args.activation_scheme, args.save_dir)
-```
+```
+
+## Evaluation
+
+### Open LLM Leaderboard evaluation scores
+|                      | Mixtral-8x7B-Instruct-v0.1 | Mixtral-8x7B-Instruct-v0.1-FP8<br>(this model) |
+| :------------------: | :----------------------: | :------------------------------------------------: |
+| arc-c<br>25-shot     | 71.50                    | 70.05                                              |
+| hellaswag<br>10-shot | 87.53                    | 86.30                                              |
+| mmlu<br>5-shot       | 70.33                    | 68.81                                              |
+| truthfulqa<br>0-shot | 64.79                    | 63.69                                              |
+| winogrande<br>5-shot | 82.40                    | 81.69                                              |
+| gsm8k<br>5-shot      | 64.36                    | 59.82                                              |
+| **Average<br>Accuracy**  | **73.48**                    |              **71.72**                                     |
+| **Recovery**             | **100%**                     |              **97.60%**                                     |
+
+
+
+
+