Update README.md
Browse files
README.md
CHANGED
|
@@ -2,6 +2,18 @@
|
|
| 2 |
tags:
|
| 3 |
- fp8
|
| 4 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 5 |
Quantized using the script below:
|
| 6 |
|
| 7 |
Command:
|
|
|
|
| 2 |
tags:
|
| 3 |
- fp8
|
| 4 |
---
|
| 5 |
+
|
| 6 |
+
```
|
| 7 |
+
vllm (pretrained=nm-testing/Mixtral-8x7B-Instruct-v0.1-FP8), gen_kwargs: (None), limit: None, num_fewshot: 5, batch_size: 1
|
| 8 |
+
| Groups |Version|Filter|n-shot|Metric|Value | |Stderr|
|
| 9 |
+
|------------------|-------|------|-----:|------|-----:|---|-----:|
|
| 10 |
+
|mmlu |N/A |none | 0|acc |0.7008|± |0.0036|
|
| 11 |
+
| - humanities |N/A |none | 5|acc |0.6453|± |0.0065|
|
| 12 |
+
| - other |N/A |none | 5|acc |0.7692|± |0.0072|
|
| 13 |
+
| - social_sciences|N/A |none | 5|acc |0.8083|± |0.0070|
|
| 14 |
+
| - stem |N/A |none | 5|acc |0.6115|± |0.0083|
|
| 15 |
+
```
|
| 16 |
+
|
| 17 |
Quantized using the script below:
|
| 18 |
|
| 19 |
Command:
|