Files changed (1) hide show
  1. README.md +120 -4
README.md CHANGED
@@ -1,13 +1,116 @@
1
  ---
 
 
 
2
  library_name: transformers
3
  tags:
4
  - llm
5
  - 7b
6
- license: cc-by-4.0
7
  datasets:
8
  - jondurbin/truthy-dpo-v0.1
9
- language:
10
- - en
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
11
  ---
12
 
13
  # Jaskier-7b-dpo-v5.6
@@ -50,4 +153,17 @@ print(conversation.messages[-1]["content"])
50
 
51
  At bards.ai, we focus on providing machine learning expertise and skills to our partners, particularly in the areas of nlp, machine vision and time series analysis. Our team is located in Wroclaw, Poland. Please visit our website for more information: bards.ai
52
 
53
- Let us know if you use our model :). Also, if you need any help, feel free to contact us at info@bards.ai
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ language:
3
+ - en
4
+ license: cc-by-4.0
5
  library_name: transformers
6
  tags:
7
  - llm
8
  - 7b
 
9
  datasets:
10
  - jondurbin/truthy-dpo-v0.1
11
+ model-index:
12
+ - name: jaskier-7b-dpo-v6.1
13
+ results:
14
+ - task:
15
+ type: text-generation
16
+ name: Text Generation
17
+ dataset:
18
+ name: AI2 Reasoning Challenge (25-Shot)
19
+ type: ai2_arc
20
+ config: ARC-Challenge
21
+ split: test
22
+ args:
23
+ num_few_shot: 25
24
+ metrics:
25
+ - type: acc_norm
26
+ value: 73.29
27
+ name: normalized accuracy
28
+ source:
29
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=bardsai/jaskier-7b-dpo-v6.1
30
+ name: Open LLM Leaderboard
31
+ - task:
32
+ type: text-generation
33
+ name: Text Generation
34
+ dataset:
35
+ name: HellaSwag (10-Shot)
36
+ type: hellaswag
37
+ split: validation
38
+ args:
39
+ num_few_shot: 10
40
+ metrics:
41
+ - type: acc_norm
42
+ value: 88.89
43
+ name: normalized accuracy
44
+ source:
45
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=bardsai/jaskier-7b-dpo-v6.1
46
+ name: Open LLM Leaderboard
47
+ - task:
48
+ type: text-generation
49
+ name: Text Generation
50
+ dataset:
51
+ name: MMLU (5-Shot)
52
+ type: cais/mmlu
53
+ config: all
54
+ split: test
55
+ args:
56
+ num_few_shot: 5
57
+ metrics:
58
+ - type: acc
59
+ value: 64.39
60
+ name: accuracy
61
+ source:
62
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=bardsai/jaskier-7b-dpo-v6.1
63
+ name: Open LLM Leaderboard
64
+ - task:
65
+ type: text-generation
66
+ name: Text Generation
67
+ dataset:
68
+ name: TruthfulQA (0-shot)
69
+ type: truthful_qa
70
+ config: multiple_choice
71
+ split: validation
72
+ args:
73
+ num_few_shot: 0
74
+ metrics:
75
+ - type: mc2
76
+ value: 77.47
77
+ source:
78
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=bardsai/jaskier-7b-dpo-v6.1
79
+ name: Open LLM Leaderboard
80
+ - task:
81
+ type: text-generation
82
+ name: Text Generation
83
+ dataset:
84
+ name: Winogrande (5-shot)
85
+ type: winogrande
86
+ config: winogrande_xl
87
+ split: validation
88
+ args:
89
+ num_few_shot: 5
90
+ metrics:
91
+ - type: acc
92
+ value: 84.69
93
+ name: accuracy
94
+ source:
95
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=bardsai/jaskier-7b-dpo-v6.1
96
+ name: Open LLM Leaderboard
97
+ - task:
98
+ type: text-generation
99
+ name: Text Generation
100
+ dataset:
101
+ name: GSM8k (5-shot)
102
+ type: gsm8k
103
+ config: main
104
+ split: test
105
+ args:
106
+ num_few_shot: 5
107
+ metrics:
108
+ - type: acc
109
+ value: 69.45
110
+ name: accuracy
111
+ source:
112
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=bardsai/jaskier-7b-dpo-v6.1
113
+ name: Open LLM Leaderboard
114
  ---
115
 
116
  # Jaskier-7b-dpo-v5.6
 
153
 
154
  At bards.ai, we focus on providing machine learning expertise and skills to our partners, particularly in the areas of nlp, machine vision and time series analysis. Our team is located in Wroclaw, Poland. Please visit our website for more information: bards.ai
155
 
156
+ Let us know if you use our model :). Also, if you need any help, feel free to contact us at info@bards.ai
157
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
158
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_bardsai__jaskier-7b-dpo-v6.1)
159
+
160
+ | Metric |Value|
161
+ |---------------------------------|----:|
162
+ |Avg. |76.36|
163
+ |AI2 Reasoning Challenge (25-Shot)|73.29|
164
+ |HellaSwag (10-Shot) |88.89|
165
+ |MMLU (5-Shot) |64.39|
166
+ |TruthfulQA (0-shot) |77.47|
167
+ |Winogrande (5-shot) |84.69|
168
+ |GSM8k (5-shot) |69.45|
169
+