Update README.md
Browse files
README.md
CHANGED
|
@@ -32,6 +32,21 @@ Original model: https://huggingface.co/google/Qwen2-7B
|
|
| 32 |
| [Qwen2-7B-Q3_K_S.gguf](https://huggingface.co/fedric95/Qwen2-7B-GGUF/blob/main/Qwen2-7B-Q3_K_S.gguf) | Q3_K_S | 3.49GB | 9.3743 +/- 0.06023 |
|
| 33 |
| [Qwen2-7B-Q2_K.gguf](https://huggingface.co/fedric95/Qwen2-7B-GGUF/blob/main/Qwen2-7B-Q2_K.gguf) | Q2_K | 3.02GB | 10.5122 +/- 0.06850 |
|
| 34 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 35 |
## Downloading using huggingface-cli
|
| 36 |
|
| 37 |
First, make sure you have hugginface-cli installed:
|
|
|
|
| 32 |
| [Qwen2-7B-Q3_K_S.gguf](https://huggingface.co/fedric95/Qwen2-7B-GGUF/blob/main/Qwen2-7B-Q3_K_S.gguf) | Q3_K_S | 3.49GB | 9.3743 +/- 0.06023 |
|
| 33 |
| [Qwen2-7B-Q2_K.gguf](https://huggingface.co/fedric95/Qwen2-7B-GGUF/blob/main/Qwen2-7B-Q2_K.gguf) | Q2_K | 3.02GB | 10.5122 +/- 0.06850 |
|
| 34 |
|
| 35 |
+
## Benchmark Results
|
| 36 |
+
|
| 37 |
+
| Benchmark | Quant type | Metric |
|
| 38 |
+
| -------- | ---------- | --------- |
|
| 39 |
+
| WinoGrande (0-shot) | Q8_0 | |
|
| 40 |
+
| WinoGrande (0-shot) | Q4_K_M | |
|
| 41 |
+
| WinoGrande (0-shot) | Q3_K_M | |
|
| 42 |
+
| WinoGrande (0-shot) | Q3_K_S | |
|
| 43 |
+
| WinoGrande (0-shot) | Q2_K | |
|
| 44 |
+
| HellaSwag (0-shot) | Q8_0 | 78.00238996 |
|
| 45 |
+
| HellaSwag (0-shot) | Q4_K_M | 77.92272456 |
|
| 46 |
+
| HellaSwag (0-shot) | Q3_K_M | 76.97669787 |
|
| 47 |
+
| HellaSwag (0-shot) | Q3_K_S | 74.96514639 |
|
| 48 |
+
| HellaSwag (0-shot) | Q2_K | 72.71459869 |
|
| 49 |
+
|
| 50 |
## Downloading using huggingface-cli
|
| 51 |
|
| 52 |
First, make sure you have hugginface-cli installed:
|