leaderboard-pr-bot commited on
Commit
48d7a3f
1 Parent(s): f8db7bf

Adding Evaluation Results

Browse files

This is an automated PR created with https://huggingface.co/spaces/Weyaxi/open-llm-leaderboard-results-pr

The purpose of this PR is to add evaluation results from the Open LLM Leaderboard to your model card.

If you encounter any issues, please report them to https://huggingface.co/spaces/Weyaxi/open-llm-leaderboard-results-pr/discussions

Files changed (1) hide show
  1. README.md +122 -14
README.md CHANGED
@@ -1,16 +1,111 @@
1
- ---
2
- base_model:
3
- - Qwen/Qwen2-72B-Instruct
4
- - alpindale/magnum-72b-v1
5
- library_name: transformers
6
- tags:
7
- - mergekit
8
- - merge
9
- - OG_finetune_merge
10
- license: other
11
- language:
12
- - en
13
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
14
 
15
  WIP of retouched [alpindale/magnum-72b-v1](https://huggingface.co/alpindale/magnum-72b-v1) but I will not use "Magnum" in the name. Call it FinalMix!
16
 
@@ -30,4 +125,17 @@ Credits to [Alpin](https://huggingface.co/alpindale) and the gang for [magnum-72
30
  {prompt}<|im_end|>
31
  <|im_start|>assistant
32
  {output}<|im_end|>
33
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ license: other
5
+ library_name: transformers
6
+ tags:
7
+ - mergekit
8
+ - merge
9
+ - OG_finetune_merge
10
+ base_model:
11
+ - Qwen/Qwen2-72B-Instruct
12
+ - alpindale/magnum-72b-v1
13
+ model-index:
14
+ - name: MG-FinalMix-72B
15
+ results:
16
+ - task:
17
+ type: text-generation
18
+ name: Text Generation
19
+ dataset:
20
+ name: IFEval (0-Shot)
21
+ type: HuggingFaceH4/ifeval
22
+ args:
23
+ num_few_shot: 0
24
+ metrics:
25
+ - type: inst_level_strict_acc and prompt_level_strict_acc
26
+ value: 80.14
27
+ name: strict accuracy
28
+ source:
29
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Undi95/MG-FinalMix-72B
30
+ name: Open LLM Leaderboard
31
+ - task:
32
+ type: text-generation
33
+ name: Text Generation
34
+ dataset:
35
+ name: BBH (3-Shot)
36
+ type: BBH
37
+ args:
38
+ num_few_shot: 3
39
+ metrics:
40
+ - type: acc_norm
41
+ value: 57.5
42
+ name: normalized accuracy
43
+ source:
44
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Undi95/MG-FinalMix-72B
45
+ name: Open LLM Leaderboard
46
+ - task:
47
+ type: text-generation
48
+ name: Text Generation
49
+ dataset:
50
+ name: MATH Lvl 5 (4-Shot)
51
+ type: hendrycks/competition_math
52
+ args:
53
+ num_few_shot: 4
54
+ metrics:
55
+ - type: exact_match
56
+ value: 33.61
57
+ name: exact match
58
+ source:
59
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Undi95/MG-FinalMix-72B
60
+ name: Open LLM Leaderboard
61
+ - task:
62
+ type: text-generation
63
+ name: Text Generation
64
+ dataset:
65
+ name: GPQA (0-shot)
66
+ type: Idavidrein/gpqa
67
+ args:
68
+ num_few_shot: 0
69
+ metrics:
70
+ - type: acc_norm
71
+ value: 18.01
72
+ name: acc_norm
73
+ source:
74
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Undi95/MG-FinalMix-72B
75
+ name: Open LLM Leaderboard
76
+ - task:
77
+ type: text-generation
78
+ name: Text Generation
79
+ dataset:
80
+ name: MuSR (0-shot)
81
+ type: TAUR-Lab/MuSR
82
+ args:
83
+ num_few_shot: 0
84
+ metrics:
85
+ - type: acc_norm
86
+ value: 21.22
87
+ name: acc_norm
88
+ source:
89
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Undi95/MG-FinalMix-72B
90
+ name: Open LLM Leaderboard
91
+ - task:
92
+ type: text-generation
93
+ name: Text Generation
94
+ dataset:
95
+ name: MMLU-PRO (5-shot)
96
+ type: TIGER-Lab/MMLU-Pro
97
+ config: main
98
+ split: test
99
+ args:
100
+ num_few_shot: 5
101
+ metrics:
102
+ - type: acc
103
+ value: 49.19
104
+ name: accuracy
105
+ source:
106
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Undi95/MG-FinalMix-72B
107
+ name: Open LLM Leaderboard
108
+ ---
109
 
110
  WIP of retouched [alpindale/magnum-72b-v1](https://huggingface.co/alpindale/magnum-72b-v1) but I will not use "Magnum" in the name. Call it FinalMix!
111
 
 
125
  {prompt}<|im_end|>
126
  <|im_start|>assistant
127
  {output}<|im_end|>
128
+ ```
129
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
130
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_Undi95__MG-FinalMix-72B)
131
+
132
+ | Metric |Value|
133
+ |-------------------|----:|
134
+ |Avg. |43.28|
135
+ |IFEval (0-Shot) |80.14|
136
+ |BBH (3-Shot) |57.50|
137
+ |MATH Lvl 5 (4-Shot)|33.61|
138
+ |GPQA (0-shot) |18.01|
139
+ |MuSR (0-shot) |21.22|
140
+ |MMLU-PRO (5-shot) |49.19|
141
+