SebastianSchramm
commited on
Commit
•
ff12284
1
Parent(s):
dd7f1ec
add evaluation scores to readme
Browse files
README.md
CHANGED
@@ -17,6 +17,17 @@ The smallest of [cerebras GPT models](https://huggingface.co/cerebras) with only
|
|
17 |
|
18 |
Instruction fine-tuned [cerebras-GPT-111M](https://huggingface.co/cerebras/Cerebras-GPT-111M)
|
19 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
20 |
## Training data
|
21 |
|
22 |
The model was fine-tuned with the following data: [alpaca_gpt4_data](https://github.com/Instruction-Tuning-with-GPT-4/GPT-4-LLM/blob/main/data/alpaca_gpt4_data.json) (data generated by GPT-4 using Alpaca prompts for fine-tuning LLMs) and [alpaca_data_cleaned](https://github.com/tloen/alpaca-lora/blob/a3027fea37c2087b8b0131b21a4cd948bbdcd9e0/alpaca_data_cleaned.json).
|
|
|
17 |
|
18 |
Instruction fine-tuned [cerebras-GPT-111M](https://huggingface.co/cerebras/Cerebras-GPT-111M)
|
19 |
|
20 |
+
## Evaluation
|
21 |
+
|
22 |
+
The model has been evaluated with Huggingface's Open LLM leaderboard. Have a look at the leaderboard for more details: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard
|
23 |
+
The performance of the instruction fine-tuned model does improve compared to the cerebras base model:
|
24 |
+
|
25 |
+
| Model | Average ⬆️ | ARC (25-shot) ⬆️ | HellaSwag (10-shot) ⬆️ | MMLU (5-shot) ⬆️ | TruthfulQA (0-shot) ⬆️ |
|
26 |
+
|------------------------------------------------ |----------- |----------------- |----------------------- |----------------- |----------------------- |
|
27 |
+
| SebastianSchramm/Cerebras-GPT-111M-instruction | 31.6 | 24.3 | 26.2 | 26.5 | 49.5 |
|
28 |
+
| cerebras/Cerebras-GPT-111M | 29.9 | 20 | 26.7 | 26.7 | 46.3 |
|
29 |
+
| | | | | | |
|
30 |
+
|
31 |
## Training data
|
32 |
|
33 |
The model was fine-tuned with the following data: [alpaca_gpt4_data](https://github.com/Instruction-Tuning-with-GPT-4/GPT-4-LLM/blob/main/data/alpaca_gpt4_data.json) (data generated by GPT-4 using Alpaca prompts for fine-tuning LLMs) and [alpaca_data_cleaned](https://github.com/tloen/alpaca-lora/blob/a3027fea37c2087b8b0131b21a4cd948bbdcd9e0/alpaca_data_cleaned.json).
|