SebastianSchramm commited on
Commit
ff12284
1 Parent(s): dd7f1ec

add evaluation scores to readme

Browse files
Files changed (1) hide show
  1. README.md +11 -0
README.md CHANGED
@@ -17,6 +17,17 @@ The smallest of [cerebras GPT models](https://huggingface.co/cerebras) with only
17
 
18
  Instruction fine-tuned [cerebras-GPT-111M](https://huggingface.co/cerebras/Cerebras-GPT-111M)
19
 
 
 
 
 
 
 
 
 
 
 
 
20
  ## Training data
21
 
22
  The model was fine-tuned with the following data: [alpaca_gpt4_data](https://github.com/Instruction-Tuning-with-GPT-4/GPT-4-LLM/blob/main/data/alpaca_gpt4_data.json) (data generated by GPT-4 using Alpaca prompts for fine-tuning LLMs) and [alpaca_data_cleaned](https://github.com/tloen/alpaca-lora/blob/a3027fea37c2087b8b0131b21a4cd948bbdcd9e0/alpaca_data_cleaned.json).
 
17
 
18
  Instruction fine-tuned [cerebras-GPT-111M](https://huggingface.co/cerebras/Cerebras-GPT-111M)
19
 
20
+ ## Evaluation
21
+
22
+ The model has been evaluated with Huggingface's Open LLM leaderboard. Have a look at the leaderboard for more details: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard
23
+ The performance of the instruction fine-tuned model does improve compared to the cerebras base model:
24
+
25
+ | Model | Average ⬆️ | ARC (25-shot) ⬆️ | HellaSwag (10-shot) ⬆️ | MMLU (5-shot) ⬆️ | TruthfulQA (0-shot) ⬆️ |
26
+ |------------------------------------------------ |----------- |----------------- |----------------------- |----------------- |----------------------- |
27
+ | SebastianSchramm/Cerebras-GPT-111M-instruction | 31.6 | 24.3 | 26.2 | 26.5 | 49.5 |
28
+ | cerebras/Cerebras-GPT-111M | 29.9 | 20 | 26.7 | 26.7 | 46.3 |
29
+ | | | | | | |
30
+
31
  ## Training data
32
 
33
  The model was fine-tuned with the following data: [alpaca_gpt4_data](https://github.com/Instruction-Tuning-with-GPT-4/GPT-4-LLM/blob/main/data/alpaca_gpt4_data.json) (data generated by GPT-4 using Alpaca prompts for fine-tuning LLMs) and [alpaca_data_cleaned](https://github.com/tloen/alpaca-lora/blob/a3027fea37c2087b8b0131b21a4cd948bbdcd9e0/alpaca_data_cleaned.json).