Commit
•
09f1ec7
1
Parent(s):
ff12284
fix eval score table
Browse files
README.md
CHANGED
@@ -20,13 +20,13 @@ Instruction fine-tuned [cerebras-GPT-111M](https://huggingface.co/cerebras/Cereb
|
|
20 |
## Evaluation
|
21 |
|
22 |
The model has been evaluated with Huggingface's Open LLM leaderboard. Have a look at the leaderboard for more details: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard
|
23 |
-
The performance of the instruction fine-tuned model does improve compared to the cerebras base model:
|
24 |
|
25 |
-
|
26 |
-
|
27 |
-
|
28 |
-
|
29 |
-
|
30 |
|
31 |
## Training data
|
32 |
|
|
|
20 |
## Evaluation
|
21 |
|
22 |
The model has been evaluated with Huggingface's Open LLM leaderboard. Have a look at the leaderboard for more details: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard
|
23 |
+
The performance of the instruction fine-tuned model does improve compared to the cerebras base model by about 5.7% (average score):
|
24 |
|
25 |
+
Model | Average | ARC (25-shot) | HellaSwag (10-shot) | MMLU (5-shot) | TruthfulQA (0-shot)
|
26 |
+
--- | --- | --- | --- | --- | ---
|
27 |
+
SebastianSchramm/Cerebras-GPT-111M-instruction | 31.6 | 24.3 | 26.2 | 26.5 | 49.5
|
28 |
+
cerebras/Cerebras-GPT-111M | 29.9 | 20 | 26.7 | 26.7 | 46.3
|
29 |
+
||||||
|
30 |
|
31 |
## Training data
|
32 |
|