Commit
984d76c
1 Parent(s): ecef4d7

Adding Evaluation Results (#1)

Browse files

- Adding Evaluation Results (ffd0c0d1320225594439caae3b5fc030eb5a935c)


Co-authored-by: Open LLM Leaderboard PR Bot <leaderboard-pr-bot@users.noreply.huggingface.co>

Files changed (1) hide show
  1. README.md +119 -3
README.md CHANGED
@@ -3,18 +3,121 @@ language:
3
  - en
4
  - es
5
  license: mit
 
6
  tags:
7
  - text-generation-inference
8
  - transformers
9
  - unsloth
10
  - mistral
11
  - trl
12
- base_model: BarraHome/zephyr-dpo-4bit
13
  datasets:
14
  - jondurbin/truthy-dpo-v0.1
15
  - BarraHome/ultrafeedback_binarized
16
- library_name: transformers
17
  pipeline_tag: text-classification
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
18
  ---
19
 
20
  # Uploaded model
@@ -25,4 +128,17 @@ pipeline_tag: text-classification
25
 
26
  This mistral model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
27
 
28
- [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  - en
4
  - es
5
  license: mit
6
+ library_name: transformers
7
  tags:
8
  - text-generation-inference
9
  - transformers
10
  - unsloth
11
  - mistral
12
  - trl
 
13
  datasets:
14
  - jondurbin/truthy-dpo-v0.1
15
  - BarraHome/ultrafeedback_binarized
16
+ base_model: BarraHome/zephyr-dpo-4bit
17
  pipeline_tag: text-classification
18
+ model-index:
19
+ - name: zephyr-dpo-v2
20
+ results:
21
+ - task:
22
+ type: text-generation
23
+ name: Text Generation
24
+ dataset:
25
+ name: AI2 Reasoning Challenge (25-Shot)
26
+ type: ai2_arc
27
+ config: ARC-Challenge
28
+ split: test
29
+ args:
30
+ num_few_shot: 25
31
+ metrics:
32
+ - type: acc_norm
33
+ value: 57.85
34
+ name: normalized accuracy
35
+ source:
36
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=BarraHome/zephyr-dpo-v2
37
+ name: Open LLM Leaderboard
38
+ - task:
39
+ type: text-generation
40
+ name: Text Generation
41
+ dataset:
42
+ name: HellaSwag (10-Shot)
43
+ type: hellaswag
44
+ split: validation
45
+ args:
46
+ num_few_shot: 10
47
+ metrics:
48
+ - type: acc_norm
49
+ value: 82.72
50
+ name: normalized accuracy
51
+ source:
52
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=BarraHome/zephyr-dpo-v2
53
+ name: Open LLM Leaderboard
54
+ - task:
55
+ type: text-generation
56
+ name: Text Generation
57
+ dataset:
58
+ name: MMLU (5-Shot)
59
+ type: cais/mmlu
60
+ config: all
61
+ split: test
62
+ args:
63
+ num_few_shot: 5
64
+ metrics:
65
+ - type: acc
66
+ value: 58.61
67
+ name: accuracy
68
+ source:
69
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=BarraHome/zephyr-dpo-v2
70
+ name: Open LLM Leaderboard
71
+ - task:
72
+ type: text-generation
73
+ name: Text Generation
74
+ dataset:
75
+ name: TruthfulQA (0-shot)
76
+ type: truthful_qa
77
+ config: multiple_choice
78
+ split: validation
79
+ args:
80
+ num_few_shot: 0
81
+ metrics:
82
+ - type: mc2
83
+ value: 56.16
84
+ source:
85
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=BarraHome/zephyr-dpo-v2
86
+ name: Open LLM Leaderboard
87
+ - task:
88
+ type: text-generation
89
+ name: Text Generation
90
+ dataset:
91
+ name: Winogrande (5-shot)
92
+ type: winogrande
93
+ config: winogrande_xl
94
+ split: validation
95
+ args:
96
+ num_few_shot: 5
97
+ metrics:
98
+ - type: acc
99
+ value: 74.35
100
+ name: accuracy
101
+ source:
102
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=BarraHome/zephyr-dpo-v2
103
+ name: Open LLM Leaderboard
104
+ - task:
105
+ type: text-generation
106
+ name: Text Generation
107
+ dataset:
108
+ name: GSM8k (5-shot)
109
+ type: gsm8k
110
+ config: main
111
+ split: test
112
+ args:
113
+ num_few_shot: 5
114
+ metrics:
115
+ - type: acc
116
+ value: 30.25
117
+ name: accuracy
118
+ source:
119
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=BarraHome/zephyr-dpo-v2
120
+ name: Open LLM Leaderboard
121
  ---
122
 
123
  # Uploaded model
 
128
 
129
  This mistral model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
130
 
131
+ [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
132
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
133
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_BarraHome__zephyr-dpo-v2)
134
+
135
+ | Metric |Value|
136
+ |---------------------------------|----:|
137
+ |Avg. |59.99|
138
+ |AI2 Reasoning Challenge (25-Shot)|57.85|
139
+ |HellaSwag (10-Shot) |82.72|
140
+ |MMLU (5-Shot) |58.61|
141
+ |TruthfulQA (0-shot) |56.16|
142
+ |Winogrande (5-shot) |74.35|
143
+ |GSM8k (5-shot) |30.25|
144
+