Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
@@ -41,6 +41,7 @@ Below are the perplexity scores for the EXL2 models. A lower score is better.
|
|
41 |
| Quant Level | Perplexity Score | Repo |
|
42 |
|-------------|------------------|------|
|
43 |
| 6.0 | 4.7068 | [turboderp](https://huggingface.co/turboderp/command-r-plus-103B-exl2) |
|
|
|
44 |
| 5.0 | 4.7309 | [turboderp](https://huggingface.co/turboderp/command-r-plus-103B-exl2) |
|
45 |
| 4.5 | 4.8111 | [turboderp](https://huggingface.co/turboderp/command-r-plus-103B-exl2) |
|
46 |
| 4.25 | 4.8292 | [turboderp](https://huggingface.co/turboderp/command-r-plus-103B-exl2) |
|
@@ -49,9 +50,10 @@ Below are the perplexity scores for the EXL2 models. A lower score is better.
|
|
49 |
| 3.5 | 4.9592 | [turboderp](https://huggingface.co/turboderp/command-r-plus-103B-exl2) |
|
50 |
| 3.25 | 5.0631 | [turboderp](https://huggingface.co/turboderp/command-r-plus-103B-exl2) |
|
51 |
| 3.0 | 5.2050 | [turboderp](https://huggingface.co/turboderp/command-r-plus-103B-exl2) |
|
|
|
52 |
| 2.5 | 5.6681 | [turboderp](https://huggingface.co/turboderp/command-r-plus-103B-exl2) |
|
|
|
53 |
|
54 |
-
_TODO:_ Dracones quant perplexity scores being worked on.
|
55 |
|
56 |
## EQ Bench
|
57 |
|
@@ -60,10 +62,11 @@ Here are the EQ Bench scores for the EXL2 quants using Alpaca, ChatML, Command-R
|
|
60 |
| Quant Size | Alpaca | ChatML | Command-R | Command-R-Plus |
|
61 |
|------------|--------|--------|--------|--------|
|
62 |
| 6.0 | 70.77 | 62.58 | 75.81 | 74.95 |
|
|
|
63 |
| 5.0 | 69.51 | 63.94 | 74.92 | 75.28 |
|
64 |
|
65 |
|
66 |
-
|
67 |
|
68 |
### Command-R-Plus Template
|
69 |
|
|
|
41 |
| Quant Level | Perplexity Score | Repo |
|
42 |
|-------------|------------------|------|
|
43 |
| 6.0 | 4.7068 | [turboderp](https://huggingface.co/turboderp/command-r-plus-103B-exl2) |
|
44 |
+
| 5.5 | 4.7136 | Dracones |
|
45 |
| 5.0 | 4.7309 | [turboderp](https://huggingface.co/turboderp/command-r-plus-103B-exl2) |
|
46 |
| 4.5 | 4.8111 | [turboderp](https://huggingface.co/turboderp/command-r-plus-103B-exl2) |
|
47 |
| 4.25 | 4.8292 | [turboderp](https://huggingface.co/turboderp/command-r-plus-103B-exl2) |
|
|
|
50 |
| 3.5 | 4.9592 | [turboderp](https://huggingface.co/turboderp/command-r-plus-103B-exl2) |
|
51 |
| 3.25 | 5.0631 | [turboderp](https://huggingface.co/turboderp/command-r-plus-103B-exl2) |
|
52 |
| 3.0 | 5.2050 | [turboderp](https://huggingface.co/turboderp/command-r-plus-103B-exl2) |
|
53 |
+
| 2.75 | 5.3820 | Dracones |
|
54 |
| 2.5 | 5.6681 | [turboderp](https://huggingface.co/turboderp/command-r-plus-103B-exl2) |
|
55 |
+
| 2.25 | 5.9769 | Dracones |
|
56 |
|
|
|
57 |
|
58 |
## EQ Bench
|
59 |
|
|
|
62 |
| Quant Size | Alpaca | ChatML | Command-R | Command-R-Plus |
|
63 |
|------------|--------|--------|--------|--------|
|
64 |
| 6.0 | 70.77 | 62.58 | 75.81 | 74.95 |
|
65 |
+
| 5.5 | 71.93 | 67.7 | 74.9 | 75.48 |
|
66 |
| 5.0 | 69.51 | 63.94 | 74.92 | 75.28 |
|
67 |
|
68 |
|
69 |
+
_Note:_ EQ Bench scripting not working well, other quants may not be tested.
|
70 |
|
71 |
### Command-R-Plus Template
|
72 |
|