Commit
•
09f1ec7
1
Parent(s):
ff12284
fix eval score table
Browse files
README.md
CHANGED
|
@@ -20,13 +20,13 @@ Instruction fine-tuned [cerebras-GPT-111M](https://huggingface.co/cerebras/Cereb
|
|
| 20 |
## Evaluation
|
| 21 |
|
| 22 |
The model has been evaluated with Huggingface's Open LLM leaderboard. Have a look at the leaderboard for more details: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard
|
| 23 |
-
The performance of the instruction fine-tuned model does improve compared to the cerebras base model:
|
| 24 |
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
| 30 |
|
| 31 |
## Training data
|
| 32 |
|
|
|
|
| 20 |
## Evaluation
|
| 21 |
|
| 22 |
The model has been evaluated with Huggingface's Open LLM leaderboard. Have a look at the leaderboard for more details: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard
|
| 23 |
+
The performance of the instruction fine-tuned model does improve compared to the cerebras base model by about 5.7% (average score):
|
| 24 |
|
| 25 |
+
Model | Average | ARC (25-shot) | HellaSwag (10-shot) | MMLU (5-shot) | TruthfulQA (0-shot)
|
| 26 |
+
--- | --- | --- | --- | --- | ---
|
| 27 |
+
SebastianSchramm/Cerebras-GPT-111M-instruction | 31.6 | 24.3 | 26.2 | 26.5 | 49.5
|
| 28 |
+
cerebras/Cerebras-GPT-111M | 29.9 | 20 | 26.7 | 26.7 | 46.3
|
| 29 |
+
||||||
|
| 30 |
|
| 31 |
## Training data
|
| 32 |
|