Commit
e40e9c0
1 Parent(s): 2cd393e

Adding Evaluation Results (#8)

Browse files

- Adding Evaluation Results (0f42ffd322f057c9a39a05a76e60ba0e8dfe1cd9)


Co-authored-by: Open LLM Leaderboard PR Bot <leaderboard-pr-bot@users.noreply.huggingface.co>

Files changed (1) hide show
  1. README.md +116 -8
README.md CHANGED
@@ -1,23 +1,117 @@
1
  ---
2
- license: other
3
- license_name: tongyi-qianwen
4
- license_link: https://huggingface.co/Qwen/Qwen2-72B-Instruct/blob/main/LICENSE
5
  language:
6
  - en
7
- pipeline_tag: text-generation
 
8
  tags:
9
  - chat
10
  - qwen
11
  - qwen2
12
  - finetune
13
  - chatml
14
- library_name: transformers
 
 
 
 
15
  inference: false
16
  model_creator: MaziyarPanahi
17
  quantized_by: MaziyarPanahi
18
- base_model: Qwen/Qwen2-72B-Instruct
19
- model_name: MaziyarPanahi/Qwen2-72B-Instruct-v0.1
20
-
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
21
  ---
22
 
23
  <img src="./qwen2-fine-tunes-maziyar-panahi.webp" alt="Qwen2 fine-tune" width="500" style="margin-left:'auto' margin-right:'auto' display:'block'"/>
@@ -91,3 +185,17 @@ from transformers import AutoTokenizer, AutoModelForCausalLM
91
  tokenizer = AutoTokenizer.from_pretrained("MaziyarPanahi/Qwen2-72B-Instruct-v0.1")
92
  model = AutoModelForCausalLM.from_pretrained("MaziyarPanahi/Qwen2-72B-Instruct-v0.1")
93
  ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
 
 
 
2
  language:
3
  - en
4
+ license: other
5
+ library_name: transformers
6
  tags:
7
  - chat
8
  - qwen
9
  - qwen2
10
  - finetune
11
  - chatml
12
+ base_model: Qwen/Qwen2-72B-Instruct
13
+ model_name: MaziyarPanahi/Qwen2-72B-Instruct-v0.1
14
+ license_name: tongyi-qianwen
15
+ license_link: https://huggingface.co/Qwen/Qwen2-72B-Instruct/blob/main/LICENSE
16
+ pipeline_tag: text-generation
17
  inference: false
18
  model_creator: MaziyarPanahi
19
  quantized_by: MaziyarPanahi
20
+ model-index:
21
+ - name: Qwen2-72B-Instruct-v0.1
22
+ results:
23
+ - task:
24
+ type: text-generation
25
+ name: Text Generation
26
+ dataset:
27
+ name: IFEval (0-Shot)
28
+ type: HuggingFaceH4/ifeval
29
+ args:
30
+ num_few_shot: 0
31
+ metrics:
32
+ - type: inst_level_strict_acc and prompt_level_strict_acc
33
+ value: 81.63
34
+ name: strict accuracy
35
+ source:
36
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/Qwen2-72B-Instruct-v0.1
37
+ name: Open LLM Leaderboard
38
+ - task:
39
+ type: text-generation
40
+ name: Text Generation
41
+ dataset:
42
+ name: BBH (3-Shot)
43
+ type: BBH
44
+ args:
45
+ num_few_shot: 3
46
+ metrics:
47
+ - type: acc_norm
48
+ value: 57.33
49
+ name: normalized accuracy
50
+ source:
51
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/Qwen2-72B-Instruct-v0.1
52
+ name: Open LLM Leaderboard
53
+ - task:
54
+ type: text-generation
55
+ name: Text Generation
56
+ dataset:
57
+ name: MATH Lvl 5 (4-Shot)
58
+ type: hendrycks/competition_math
59
+ args:
60
+ num_few_shot: 4
61
+ metrics:
62
+ - type: exact_match
63
+ value: 36.03
64
+ name: exact match
65
+ source:
66
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/Qwen2-72B-Instruct-v0.1
67
+ name: Open LLM Leaderboard
68
+ - task:
69
+ type: text-generation
70
+ name: Text Generation
71
+ dataset:
72
+ name: GPQA (0-shot)
73
+ type: Idavidrein/gpqa
74
+ args:
75
+ num_few_shot: 0
76
+ metrics:
77
+ - type: acc_norm
78
+ value: 17.45
79
+ name: acc_norm
80
+ source:
81
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/Qwen2-72B-Instruct-v0.1
82
+ name: Open LLM Leaderboard
83
+ - task:
84
+ type: text-generation
85
+ name: Text Generation
86
+ dataset:
87
+ name: MuSR (0-shot)
88
+ type: TAUR-Lab/MuSR
89
+ args:
90
+ num_few_shot: 0
91
+ metrics:
92
+ - type: acc_norm
93
+ value: 20.15
94
+ name: acc_norm
95
+ source:
96
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/Qwen2-72B-Instruct-v0.1
97
+ name: Open LLM Leaderboard
98
+ - task:
99
+ type: text-generation
100
+ name: Text Generation
101
+ dataset:
102
+ name: MMLU-PRO (5-shot)
103
+ type: TIGER-Lab/MMLU-Pro
104
+ config: main
105
+ split: test
106
+ args:
107
+ num_few_shot: 5
108
+ metrics:
109
+ - type: acc
110
+ value: 49.05
111
+ name: accuracy
112
+ source:
113
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/Qwen2-72B-Instruct-v0.1
114
+ name: Open LLM Leaderboard
115
  ---
116
 
117
  <img src="./qwen2-fine-tunes-maziyar-panahi.webp" alt="Qwen2 fine-tune" width="500" style="margin-left:'auto' margin-right:'auto' display:'block'"/>
 
185
  tokenizer = AutoTokenizer.from_pretrained("MaziyarPanahi/Qwen2-72B-Instruct-v0.1")
186
  model = AutoModelForCausalLM.from_pretrained("MaziyarPanahi/Qwen2-72B-Instruct-v0.1")
187
  ```
188
+
189
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
190
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_MaziyarPanahi__Qwen2-72B-Instruct-v0.1)
191
+
192
+ | Metric |Value|
193
+ |-------------------|----:|
194
+ |Avg. |43.61|
195
+ |IFEval (0-Shot) |81.63|
196
+ |BBH (3-Shot) |57.33|
197
+ |MATH Lvl 5 (4-Shot)|36.03|
198
+ |GPQA (0-shot) |17.45|
199
+ |MuSR (0-shot) |20.15|
200
+ |MMLU-PRO (5-shot) |49.05|
201
+