Commit
761f364
1 Parent(s): ac74fdd

Adding Evaluation Results (#1)

Browse files

- Adding Evaluation Results (b4fc2cefb6d88d773d6cc749298bec6b993e6332)


Co-authored-by: Open LLM Leaderboard PR Bot <leaderboard-pr-bot@users.noreply.huggingface.co>

Files changed (1) hide show
  1. README.md +117 -1
README.md CHANGED
@@ -2,6 +2,109 @@
2
  license: cc-by-nc-4.0
3
  datasets:
4
  - ChristophSchuhmann/essays-with-instructions
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
5
  ---
6
 
7
  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/63732ebbbd81fae2b3aaf3fb/iMK7IcqeIP64FjOTbQirv.png)
@@ -11,4 +114,17 @@ Prompt Example:
11
  ### Instruction:
12
  Provide the inputted essay that when summarized resulted in the following summary : The organization's learning and development practices are poor as they involve manual orientation from experienced workers. New employees are not advised of their roles in the organization before assignment of tasks. The orientation process takes long due to lack of a well-coordinated orientation programs.
13
  ### Response:
14
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2
  license: cc-by-nc-4.0
3
  datasets:
4
  - ChristophSchuhmann/essays-with-instructions
5
+ model-index:
6
+ - name: Galactica-6.7B-EssayWriter
7
+ results:
8
+ - task:
9
+ type: text-generation
10
+ name: Text Generation
11
+ dataset:
12
+ name: AI2 Reasoning Challenge (25-Shot)
13
+ type: ai2_arc
14
+ config: ARC-Challenge
15
+ split: test
16
+ args:
17
+ num_few_shot: 25
18
+ metrics:
19
+ - type: acc_norm
20
+ value: 40.1
21
+ name: normalized accuracy
22
+ source:
23
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=KnutJaegersberg/Galactica-6.7B-EssayWriter
24
+ name: Open LLM Leaderboard
25
+ - task:
26
+ type: text-generation
27
+ name: Text Generation
28
+ dataset:
29
+ name: HellaSwag (10-Shot)
30
+ type: hellaswag
31
+ split: validation
32
+ args:
33
+ num_few_shot: 10
34
+ metrics:
35
+ - type: acc_norm
36
+ value: 50.29
37
+ name: normalized accuracy
38
+ source:
39
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=KnutJaegersberg/Galactica-6.7B-EssayWriter
40
+ name: Open LLM Leaderboard
41
+ - task:
42
+ type: text-generation
43
+ name: Text Generation
44
+ dataset:
45
+ name: MMLU (5-Shot)
46
+ type: cais/mmlu
47
+ config: all
48
+ split: test
49
+ args:
50
+ num_few_shot: 5
51
+ metrics:
52
+ - type: acc
53
+ value: 33.88
54
+ name: accuracy
55
+ source:
56
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=KnutJaegersberg/Galactica-6.7B-EssayWriter
57
+ name: Open LLM Leaderboard
58
+ - task:
59
+ type: text-generation
60
+ name: Text Generation
61
+ dataset:
62
+ name: TruthfulQA (0-shot)
63
+ type: truthful_qa
64
+ config: multiple_choice
65
+ split: validation
66
+ args:
67
+ num_few_shot: 0
68
+ metrics:
69
+ - type: mc2
70
+ value: 40.27
71
+ source:
72
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=KnutJaegersberg/Galactica-6.7B-EssayWriter
73
+ name: Open LLM Leaderboard
74
+ - task:
75
+ type: text-generation
76
+ name: Text Generation
77
+ dataset:
78
+ name: Winogrande (5-shot)
79
+ type: winogrande
80
+ config: winogrande_xl
81
+ split: validation
82
+ args:
83
+ num_few_shot: 5
84
+ metrics:
85
+ - type: acc
86
+ value: 58.48
87
+ name: accuracy
88
+ source:
89
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=KnutJaegersberg/Galactica-6.7B-EssayWriter
90
+ name: Open LLM Leaderboard
91
+ - task:
92
+ type: text-generation
93
+ name: Text Generation
94
+ dataset:
95
+ name: GSM8k (5-shot)
96
+ type: gsm8k
97
+ config: main
98
+ split: test
99
+ args:
100
+ num_few_shot: 5
101
+ metrics:
102
+ - type: acc
103
+ value: 3.49
104
+ name: accuracy
105
+ source:
106
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=KnutJaegersberg/Galactica-6.7B-EssayWriter
107
+ name: Open LLM Leaderboard
108
  ---
109
 
110
  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/63732ebbbd81fae2b3aaf3fb/iMK7IcqeIP64FjOTbQirv.png)
 
114
  ### Instruction:
115
  Provide the inputted essay that when summarized resulted in the following summary : The organization's learning and development practices are poor as they involve manual orientation from experienced workers. New employees are not advised of their roles in the organization before assignment of tasks. The orientation process takes long due to lack of a well-coordinated orientation programs.
116
  ### Response:
117
+ ```
118
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
119
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_KnutJaegersberg__Galactica-6.7B-EssayWriter)
120
+
121
+ | Metric |Value|
122
+ |---------------------------------|----:|
123
+ |Avg. |37.75|
124
+ |AI2 Reasoning Challenge (25-Shot)|40.10|
125
+ |HellaSwag (10-Shot) |50.29|
126
+ |MMLU (5-Shot) |33.88|
127
+ |TruthfulQA (0-shot) |40.27|
128
+ |Winogrande (5-shot) |58.48|
129
+ |GSM8k (5-shot) | 3.49|
130
+