Commit
•
abf42d5
1
Parent(s):
136d8ad
Update README.md
Browse files
README.md
CHANGED
@@ -18,4 +18,14 @@ This is a 32k version of Sao10K/WinterGoddess-1.4x-70B-L2, extended using method
|
|
18 |
| D | 3 | 0 |
|
19 |
| S | 2.75 | 1.5 |
|
20 |
| P | 5.5 | 2.25 |
|
21 |
-
| Total | 14.75 | 8.25 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
18 |
| D | 3 | 0 |
|
19 |
| S | 2.75 | 1.5 |
|
20 |
| P | 5.5 | 2.25 |
|
21 |
+
| Total | 14.75 | 8.25 |
|
22 |
+
|
23 |
+
### Open LLM leaderboard
|
24 |
+
[Leaderboard on Huggingface](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
|
25 |
+
|Model |Average|ARC |HellaSwag|MMLU |TruthfulQA|Winogrande|GSM8K|
|
26 |
+
|---------------------------------------|-------|-----|---------|-----|----------|----------|-----|
|
27 |
+
|Sao10K/WinterGoddess-1.4x-70B-L2 |73.23 |72.78|90.11 |71.12|65.76 |85 |54.59|
|
28 |
+
|ChuckMcSneed/WinterGoddess-1.4x-70b-32k|69.4 |71.16|89.12 |66.42|63.87 |82.56 |43.29|
|
29 |
+
|Difference |3.83 |1.62 |0.99 |4.7 |1.89 |2.44 |11.3 |
|
30 |
+
|
31 |
+
Here the losses seem far less brutal than on my bench. It seems that extending with LongLORA kills MMLU and GSM8K performance.
|