leaderboard-pr-bot commited on
Commit
81b5aed
1 Parent(s): 4bca8d7

Adding Evaluation Results

Browse files

This is an automated PR created with https://huggingface.co/spaces/Weyaxi/open-llm-leaderboard-results-pr

The purpose of this PR is to add evaluation results from the Open LLM Leaderboard to your model card.

If you encounter any issues, please report them to https://huggingface.co/spaces/Weyaxi/open-llm-leaderboard-results-pr/discussions

Files changed (1) hide show
  1. README.md +119 -0
README.md CHANGED
@@ -1,3 +1,108 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
 
2
  # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
3
  Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_PulsarAI__Chat-AYB-Nova-13B)
@@ -12,3 +117,17 @@ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-le
12
  | Winogrande (5-shot) | 77.58 |
13
  | GSM8K (5-shot) | 12.36 |
14
  | DROP (3-shot) | 8.03 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ model-index:
3
+ - name: Chat-AYB-Nova-13B
4
+ results:
5
+ - task:
6
+ type: text-generation
7
+ name: Text Generation
8
+ dataset:
9
+ name: AI2 Reasoning Challenge (25-Shot)
10
+ type: ai2_arc
11
+ config: ARC-Challenge
12
+ split: test
13
+ args:
14
+ num_few_shot: 25
15
+ metrics:
16
+ - type: acc_norm
17
+ value: 62.97
18
+ name: normalized accuracy
19
+ source:
20
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=PulsarAI/Chat-AYB-Nova-13B
21
+ name: Open LLM Leaderboard
22
+ - task:
23
+ type: text-generation
24
+ name: Text Generation
25
+ dataset:
26
+ name: HellaSwag (10-Shot)
27
+ type: hellaswag
28
+ split: validation
29
+ args:
30
+ num_few_shot: 10
31
+ metrics:
32
+ - type: acc_norm
33
+ value: 84.28
34
+ name: normalized accuracy
35
+ source:
36
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=PulsarAI/Chat-AYB-Nova-13B
37
+ name: Open LLM Leaderboard
38
+ - task:
39
+ type: text-generation
40
+ name: Text Generation
41
+ dataset:
42
+ name: MMLU (5-Shot)
43
+ type: cais/mmlu
44
+ config: all
45
+ split: test
46
+ args:
47
+ num_few_shot: 5
48
+ metrics:
49
+ - type: acc
50
+ value: 58.58
51
+ name: accuracy
52
+ source:
53
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=PulsarAI/Chat-AYB-Nova-13B
54
+ name: Open LLM Leaderboard
55
+ - task:
56
+ type: text-generation
57
+ name: Text Generation
58
+ dataset:
59
+ name: TruthfulQA (0-shot)
60
+ type: truthful_qa
61
+ config: multiple_choice
62
+ split: validation
63
+ args:
64
+ num_few_shot: 0
65
+ metrics:
66
+ - type: mc2
67
+ value: 51.28
68
+ source:
69
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=PulsarAI/Chat-AYB-Nova-13B
70
+ name: Open LLM Leaderboard
71
+ - task:
72
+ type: text-generation
73
+ name: Text Generation
74
+ dataset:
75
+ name: Winogrande (5-shot)
76
+ type: winogrande
77
+ config: winogrande_xl
78
+ split: validation
79
+ args:
80
+ num_few_shot: 5
81
+ metrics:
82
+ - type: acc
83
+ value: 77.58
84
+ name: accuracy
85
+ source:
86
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=PulsarAI/Chat-AYB-Nova-13B
87
+ name: Open LLM Leaderboard
88
+ - task:
89
+ type: text-generation
90
+ name: Text Generation
91
+ dataset:
92
+ name: GSM8k (5-shot)
93
+ type: gsm8k
94
+ config: main
95
+ split: test
96
+ args:
97
+ num_few_shot: 5
98
+ metrics:
99
+ - type: acc
100
+ value: 12.36
101
+ name: accuracy
102
+ source:
103
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=PulsarAI/Chat-AYB-Nova-13B
104
+ name: Open LLM Leaderboard
105
+ ---
106
 
107
  # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
108
  Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_PulsarAI__Chat-AYB-Nova-13B)
 
117
  | Winogrande (5-shot) | 77.58 |
118
  | GSM8K (5-shot) | 12.36 |
119
  | DROP (3-shot) | 8.03 |
120
+
121
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
122
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_PulsarAI__Chat-AYB-Nova-13B)
123
+
124
+ | Metric |Value|
125
+ |---------------------------------|----:|
126
+ |Avg. |57.84|
127
+ |AI2 Reasoning Challenge (25-Shot)|62.97|
128
+ |HellaSwag (10-Shot) |84.28|
129
+ |MMLU (5-Shot) |58.58|
130
+ |TruthfulQA (0-shot) |51.28|
131
+ |Winogrande (5-shot) |77.58|
132
+ |GSM8k (5-shot) |12.36|
133
+