leaderboard-pr-bot commited on
Commit
2cd9dbd
1 Parent(s): 14c93f3

Adding Evaluation Results

Browse files

This is an automated PR created with https://huggingface.co/spaces/Weyaxi/open-llm-leaderboard-results-pr

The purpose of this PR is to add evaluation results from the Open LLM Leaderboard to your model card.

If you encounter any issues, please report them to https://huggingface.co/spaces/Weyaxi/open-llm-leaderboard-results-pr/discussions

Files changed (1) hide show
  1. README.md +119 -4
README.md CHANGED
@@ -9,14 +9,116 @@ language:
9
  - it
10
  - ru
11
  - fi
12
-
 
13
  pipeline_tag: text-generation
14
  inference: false
15
- library_name: transformers
16
- license: other
17
  license_name: gemma
18
  license_link: https://ai.google.dev/gemma/terms
19
  base_model: google/gemma-7b
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
20
  ---
21
 
22
  # OpenBuddy - Open Multilingual Chatbot
@@ -50,4 +152,17 @@ By using OpenBuddy, you agree to these terms and conditions, and acknowledge tha
50
 
51
  OpenBuddy按“原样”提供,不附带任何种类的明示或暗示的保证,包括但不限于适销性、特定目的的适用性和非侵权的暗示保证。在任何情况下,作者、贡献者或版权所有者均不对因软件或使用或其他软件交易而产生的任何索赔、损害赔偿或其他责任(无论是合同、侵权还是其他原因)承担责任。
52
 
53
- 使用OpenBuddy即表示您同意这些条款和条件,并承认您了解其使用可能带来的潜在风险。您还同意赔偿并使作者、贡献者和版权所有者免受因您使用OpenBuddy而产生的任何索赔、损害赔偿或责任的影响。
 
 
 
 
 
 
 
 
 
 
 
 
 
 
9
  - it
10
  - ru
11
  - fi
12
+ license: other
13
+ library_name: transformers
14
  pipeline_tag: text-generation
15
  inference: false
 
 
16
  license_name: gemma
17
  license_link: https://ai.google.dev/gemma/terms
18
  base_model: google/gemma-7b
19
+ model-index:
20
+ - name: openbuddy-gemma-7b-v19.1-4k
21
+ results:
22
+ - task:
23
+ type: text-generation
24
+ name: Text Generation
25
+ dataset:
26
+ name: AI2 Reasoning Challenge (25-Shot)
27
+ type: ai2_arc
28
+ config: ARC-Challenge
29
+ split: test
30
+ args:
31
+ num_few_shot: 25
32
+ metrics:
33
+ - type: acc_norm
34
+ value: 55.29
35
+ name: normalized accuracy
36
+ source:
37
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=OpenBuddy/openbuddy-gemma-7b-v19.1-4k
38
+ name: Open LLM Leaderboard
39
+ - task:
40
+ type: text-generation
41
+ name: Text Generation
42
+ dataset:
43
+ name: HellaSwag (10-Shot)
44
+ type: hellaswag
45
+ split: validation
46
+ args:
47
+ num_few_shot: 10
48
+ metrics:
49
+ - type: acc_norm
50
+ value: 71.07
51
+ name: normalized accuracy
52
+ source:
53
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=OpenBuddy/openbuddy-gemma-7b-v19.1-4k
54
+ name: Open LLM Leaderboard
55
+ - task:
56
+ type: text-generation
57
+ name: Text Generation
58
+ dataset:
59
+ name: MMLU (5-Shot)
60
+ type: cais/mmlu
61
+ config: all
62
+ split: test
63
+ args:
64
+ num_few_shot: 5
65
+ metrics:
66
+ - type: acc
67
+ value: 53.32
68
+ name: accuracy
69
+ source:
70
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=OpenBuddy/openbuddy-gemma-7b-v19.1-4k
71
+ name: Open LLM Leaderboard
72
+ - task:
73
+ type: text-generation
74
+ name: Text Generation
75
+ dataset:
76
+ name: TruthfulQA (0-shot)
77
+ type: truthful_qa
78
+ config: multiple_choice
79
+ split: validation
80
+ args:
81
+ num_few_shot: 0
82
+ metrics:
83
+ - type: mc2
84
+ value: 49.21
85
+ source:
86
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=OpenBuddy/openbuddy-gemma-7b-v19.1-4k
87
+ name: Open LLM Leaderboard
88
+ - task:
89
+ type: text-generation
90
+ name: Text Generation
91
+ dataset:
92
+ name: Winogrande (5-shot)
93
+ type: winogrande
94
+ config: winogrande_xl
95
+ split: validation
96
+ args:
97
+ num_few_shot: 5
98
+ metrics:
99
+ - type: acc
100
+ value: 67.48
101
+ name: accuracy
102
+ source:
103
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=OpenBuddy/openbuddy-gemma-7b-v19.1-4k
104
+ name: Open LLM Leaderboard
105
+ - task:
106
+ type: text-generation
107
+ name: Text Generation
108
+ dataset:
109
+ name: GSM8k (5-shot)
110
+ type: gsm8k
111
+ config: main
112
+ split: test
113
+ args:
114
+ num_few_shot: 5
115
+ metrics:
116
+ - type: acc
117
+ value: 39.35
118
+ name: accuracy
119
+ source:
120
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=OpenBuddy/openbuddy-gemma-7b-v19.1-4k
121
+ name: Open LLM Leaderboard
122
  ---
123
 
124
  # OpenBuddy - Open Multilingual Chatbot
 
152
 
153
  OpenBuddy按“原样”提供,不附带任何种类的明示或暗示的保证,包括但不限于适销性、特定目的的适用性和非侵权的暗示保证。在任何情况下,作者、贡献者或版权所有者均不对因软件或使用或其他软件交易而产生的任何索赔、损害赔偿或其他责任(无论是合同、侵权还是其他原因)承担责任。
154
 
155
+ 使用OpenBuddy即表示您同意这些条款和条件,并承认您了解其使用可能带来的潜在风险。您还同意赔偿并使作者、贡献者和版权所有者免受因您使用OpenBuddy而产生的任何索赔、损害赔偿或责任的影响。
156
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
157
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_OpenBuddy__openbuddy-gemma-7b-v19.1-4k)
158
+
159
+ | Metric |Value|
160
+ |---------------------------------|----:|
161
+ |Avg. |55.95|
162
+ |AI2 Reasoning Challenge (25-Shot)|55.29|
163
+ |HellaSwag (10-Shot) |71.07|
164
+ |MMLU (5-Shot) |53.32|
165
+ |TruthfulQA (0-shot) |49.21|
166
+ |Winogrande (5-shot) |67.48|
167
+ |GSM8k (5-shot) |39.35|
168
+