leaderboard-pr-bot commited on
Commit
8138c65
1 Parent(s): 2f275b1

Adding Evaluation Results

Browse files

This is an automated PR created with https://huggingface.co/spaces/Weyaxi/open-llm-leaderboard-results-pr

The purpose of this PR is to add evaluation results from the Open LLM Leaderboard to your model card.

If you encounter any issues, please report them to https://huggingface.co/spaces/Weyaxi/open-llm-leaderboard-results-pr/discussions

Files changed (1) hide show
  1. README.md +114 -6
README.md CHANGED
@@ -1,4 +1,9 @@
1
  ---
 
 
 
 
 
2
  datasets:
3
  - HuggingFaceH4/ultrachat_200k
4
  - allenai/ultrafeedback_binarized_cleaned
@@ -8,17 +13,107 @@ datasets:
8
  - LDJnr/Capybara
9
  - Intel/orca_dpo_pairs
10
  - hkust-nlp/deita-10k-v0
11
- language:
12
- - en
13
- tags:
14
- - causal-lm
15
  extra_gated_fields:
16
  Name: text
17
  Email: text
18
  Country: text
19
  Organization or Affiliation: text
20
  I ALLOW Stability AI to email me about new model releases: checkbox
21
- license: other
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
22
  ---
23
  # `StableLM 2 Zephyr 1.6B`
24
 
@@ -155,4 +250,17 @@ Finally, do not use the models if they are unsuitable for your application, or f
155
  title={Stable LM 2 1.6B},
156
  author={Stability AI Language Team}
157
  }
158
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ language:
3
+ - en
4
+ license: other
5
+ tags:
6
+ - causal-lm
7
  datasets:
8
  - HuggingFaceH4/ultrachat_200k
9
  - allenai/ultrafeedback_binarized_cleaned
 
13
  - LDJnr/Capybara
14
  - Intel/orca_dpo_pairs
15
  - hkust-nlp/deita-10k-v0
 
 
 
 
16
  extra_gated_fields:
17
  Name: text
18
  Email: text
19
  Country: text
20
  Organization or Affiliation: text
21
  I ALLOW Stability AI to email me about new model releases: checkbox
22
+ model-index:
23
+ - name: stablelm-2-zephyr-1_6b
24
+ results:
25
+ - task:
26
+ type: text-generation
27
+ name: Text Generation
28
+ dataset:
29
+ name: IFEval (0-Shot)
30
+ type: HuggingFaceH4/ifeval
31
+ args:
32
+ num_few_shot: 0
33
+ metrics:
34
+ - type: inst_level_strict_acc and prompt_level_strict_acc
35
+ value: 32.79
36
+ name: strict accuracy
37
+ source:
38
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=stabilityai/stablelm-2-zephyr-1_6b
39
+ name: Open LLM Leaderboard
40
+ - task:
41
+ type: text-generation
42
+ name: Text Generation
43
+ dataset:
44
+ name: BBH (3-Shot)
45
+ type: BBH
46
+ args:
47
+ num_few_shot: 3
48
+ metrics:
49
+ - type: acc_norm
50
+ value: 6.71
51
+ name: normalized accuracy
52
+ source:
53
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=stabilityai/stablelm-2-zephyr-1_6b
54
+ name: Open LLM Leaderboard
55
+ - task:
56
+ type: text-generation
57
+ name: Text Generation
58
+ dataset:
59
+ name: MATH Lvl 5 (4-Shot)
60
+ type: hendrycks/competition_math
61
+ args:
62
+ num_few_shot: 4
63
+ metrics:
64
+ - type: exact_match
65
+ value: 2.11
66
+ name: exact match
67
+ source:
68
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=stabilityai/stablelm-2-zephyr-1_6b
69
+ name: Open LLM Leaderboard
70
+ - task:
71
+ type: text-generation
72
+ name: Text Generation
73
+ dataset:
74
+ name: GPQA (0-shot)
75
+ type: Idavidrein/gpqa
76
+ args:
77
+ num_few_shot: 0
78
+ metrics:
79
+ - type: acc_norm
80
+ value: 0.0
81
+ name: acc_norm
82
+ source:
83
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=stabilityai/stablelm-2-zephyr-1_6b
84
+ name: Open LLM Leaderboard
85
+ - task:
86
+ type: text-generation
87
+ name: Text Generation
88
+ dataset:
89
+ name: MuSR (0-shot)
90
+ type: TAUR-Lab/MuSR
91
+ args:
92
+ num_few_shot: 0
93
+ metrics:
94
+ - type: acc_norm
95
+ value: 5.99
96
+ name: acc_norm
97
+ source:
98
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=stabilityai/stablelm-2-zephyr-1_6b
99
+ name: Open LLM Leaderboard
100
+ - task:
101
+ type: text-generation
102
+ name: Text Generation
103
+ dataset:
104
+ name: MMLU-PRO (5-shot)
105
+ type: TIGER-Lab/MMLU-Pro
106
+ config: main
107
+ split: test
108
+ args:
109
+ num_few_shot: 5
110
+ metrics:
111
+ - type: acc
112
+ value: 7.93
113
+ name: accuracy
114
+ source:
115
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=stabilityai/stablelm-2-zephyr-1_6b
116
+ name: Open LLM Leaderboard
117
  ---
118
  # `StableLM 2 Zephyr 1.6B`
119
 
 
250
  title={Stable LM 2 1.6B},
251
  author={Stability AI Language Team}
252
  }
253
+ ```
254
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
255
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_stabilityai__stablelm-2-zephyr-1_6b)
256
+
257
+ | Metric |Value|
258
+ |-------------------|----:|
259
+ |Avg. | 9.50|
260
+ |IFEval (0-Shot) |32.79|
261
+ |BBH (3-Shot) | 6.71|
262
+ |MATH Lvl 5 (4-Shot)| 2.11|
263
+ |GPQA (0-shot) | 0.00|
264
+ |MuSR (0-shot) | 5.99|
265
+ |MMLU-PRO (5-shot) | 7.93|
266
+