MaziyarPanahi commited on
Commit
d1268a0
1 Parent(s): 40687fd

Adding Evaluation Results (#8)

Browse files

- Adding Evaluation Results (3beb2815d222e29678f23cd747e2ed7682a86ebb)

Files changed (1) hide show
  1. README.md +122 -5
README.md CHANGED
@@ -1,5 +1,7 @@
1
  ---
2
- base_model: MaziyarPanahi/Llama-3-8B-Instruct-v0.4
 
 
3
  library_name: transformers
4
  tags:
5
  - axolotl
@@ -9,16 +11,117 @@ tags:
9
  - pytorch
10
  - llama
11
  - llama-3
12
- language:
13
- - en
14
  pipeline_tag: text-generation
15
- license: other
16
  license_name: llama3
17
  license_link: LICENSE
18
  inference: false
19
  model_creator: MaziyarPanahi
20
- model_name: Llama-3-8B-Instruct-v0.8
21
  quantized_by: MaziyarPanahi
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
22
  ---
23
 
24
  <img src="./llama-3-merges.webp" alt="Llama-3 DPO Logo" width="500" style="margin-left:'auto' margin-right:'auto' display:'block'"/>
@@ -108,3 +211,17 @@ outputs = pipeline(
108
  )
109
  print(outputs[0]["generated_text"][len(prompt):])
110
  ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ language:
3
+ - en
4
+ license: other
5
  library_name: transformers
6
  tags:
7
  - axolotl
 
11
  - pytorch
12
  - llama
13
  - llama-3
14
+ base_model: MaziyarPanahi/Llama-3-8B-Instruct-v0.4
15
+ model_name: Llama-3-8B-Instruct-v0.8
16
  pipeline_tag: text-generation
 
17
  license_name: llama3
18
  license_link: LICENSE
19
  inference: false
20
  model_creator: MaziyarPanahi
 
21
  quantized_by: MaziyarPanahi
22
+ model-index:
23
+ - name: Llama-3-8B-Instruct-v0.8
24
+ results:
25
+ - task:
26
+ type: text-generation
27
+ name: Text Generation
28
+ dataset:
29
+ name: AI2 Reasoning Challenge (25-Shot)
30
+ type: ai2_arc
31
+ config: ARC-Challenge
32
+ split: test
33
+ args:
34
+ num_few_shot: 25
35
+ metrics:
36
+ - type: acc_norm
37
+ value: 71.67
38
+ name: normalized accuracy
39
+ source:
40
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MaziyarPanahi/Llama-3-8B-Instruct-v0.8
41
+ name: Open LLM Leaderboard
42
+ - task:
43
+ type: text-generation
44
+ name: Text Generation
45
+ dataset:
46
+ name: HellaSwag (10-Shot)
47
+ type: hellaswag
48
+ split: validation
49
+ args:
50
+ num_few_shot: 10
51
+ metrics:
52
+ - type: acc_norm
53
+ value: 87.77
54
+ name: normalized accuracy
55
+ source:
56
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MaziyarPanahi/Llama-3-8B-Instruct-v0.8
57
+ name: Open LLM Leaderboard
58
+ - task:
59
+ type: text-generation
60
+ name: Text Generation
61
+ dataset:
62
+ name: MMLU (5-Shot)
63
+ type: cais/mmlu
64
+ config: all
65
+ split: test
66
+ args:
67
+ num_few_shot: 5
68
+ metrics:
69
+ - type: acc
70
+ value: 68.3
71
+ name: accuracy
72
+ source:
73
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MaziyarPanahi/Llama-3-8B-Instruct-v0.8
74
+ name: Open LLM Leaderboard
75
+ - task:
76
+ type: text-generation
77
+ name: Text Generation
78
+ dataset:
79
+ name: TruthfulQA (0-shot)
80
+ type: truthful_qa
81
+ config: multiple_choice
82
+ split: validation
83
+ args:
84
+ num_few_shot: 0
85
+ metrics:
86
+ - type: mc2
87
+ value: 63.9
88
+ source:
89
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MaziyarPanahi/Llama-3-8B-Instruct-v0.8
90
+ name: Open LLM Leaderboard
91
+ - task:
92
+ type: text-generation
93
+ name: Text Generation
94
+ dataset:
95
+ name: Winogrande (5-shot)
96
+ type: winogrande
97
+ config: winogrande_xl
98
+ split: validation
99
+ args:
100
+ num_few_shot: 5
101
+ metrics:
102
+ - type: acc
103
+ value: 79.08
104
+ name: accuracy
105
+ source:
106
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MaziyarPanahi/Llama-3-8B-Instruct-v0.8
107
+ name: Open LLM Leaderboard
108
+ - task:
109
+ type: text-generation
110
+ name: Text Generation
111
+ dataset:
112
+ name: GSM8k (5-shot)
113
+ type: gsm8k
114
+ config: main
115
+ split: test
116
+ args:
117
+ num_few_shot: 5
118
+ metrics:
119
+ - type: acc
120
+ value: 68.46
121
+ name: accuracy
122
+ source:
123
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MaziyarPanahi/Llama-3-8B-Instruct-v0.8
124
+ name: Open LLM Leaderboard
125
  ---
126
 
127
  <img src="./llama-3-merges.webp" alt="Llama-3 DPO Logo" width="500" style="margin-left:'auto' margin-right:'auto' display:'block'"/>
 
211
  )
212
  print(outputs[0]["generated_text"][len(prompt):])
213
  ```
214
+
215
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
216
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_MaziyarPanahi__Llama-3-8B-Instruct-v0.8)
217
+
218
+ | Metric |Value|
219
+ |---------------------------------|----:|
220
+ |Avg. |73.20|
221
+ |AI2 Reasoning Challenge (25-Shot)|71.67|
222
+ |HellaSwag (10-Shot) |87.77|
223
+ |MMLU (5-Shot) |68.30|
224
+ |TruthfulQA (0-shot) |63.90|
225
+ |Winogrande (5-shot) |79.08|
226
+ |GSM8k (5-shot) |68.46|
227
+