Update README.md

README.md

@@ -4,11 +4,13 @@ tags:
 - vllm
 ---
 
-# Qwen2-72B-Instruct-FP8 
+# Qwen2-72B-Instruct-FP8
 
-Ready to use with `vllm>=0.5.0`.
+## Model Overview
+Qwen2-72B-Instruct quantized to FP8 weights and activations using per-tensor quantization, ready for inference with vLLM >= 0.5.0.
 
-Quantized with [AutoFP8](https://github.com/neuralmagic/autofp8) using the following script on 8xA100:
+## Usage and Creation
+Produced using [AutoFP8 with calibration samples from ultrachat](https://github.com/neuralmagic/AutoFP8/blob/147fa4d9e1a90ef8a93f96fc7d9c33056ddc017a/example_dataset.py).
 
 ```python
 from datasets import load_dataset
@@ -33,4 +35,19 @@ model = AutoFP8ForCausalLM.from_pretrained(
 )
 model.quantize(examples)
 model.save_quantized(quantized_model_dir)
-```
+```
+
+## Evaluation
+
+### Open LLM Leaderboard evaluation scores
+|                      | Qwen2-72B-Instruct       | Qwen2-72B-Instruct-FP8<br>(this model) |
+| :------------------: | :----------------------: | :------------------------------------------------: |
+| arc-c<br>25-shot     | 71.58                    | 72.09                                              |
+| hellaswag<br>10-shot | 86.94                    | 86.83                                              |
+| mmlu<br>5-shot       | xx.xx                    | 84.06                                              |
+| truthfulqa<br>0-shot | 66.94                    | 66.95                                              |
+| winogrande<br>5-shot | 82.79                    | 83.18                                              |
+| gsm8k<br>5-shot      | xx.xx                    | 88.93                                              |
+| **Average<br>Accuracy**  | **xx.xx**                    |              **80.34**                                     |
+| **Recovery**             | **100%**                     |              **xx.xx%**                                     |
+