Hugging Face's logo

Sharathhebbar24
/
math_gpt2

Text Generation
Transformers
PyTorch
English
gpt2
maths
arxiv-math
mathgpt2
text-generation-inference
Inference Endpoints
Model card Files Community
2
leaderboard-pr-bot commited on
Commit
14fb68e
•
1 Parent(s): 57b7106

Adding Evaluation Results

Browse files

This is an automated PR created with https://huggingface.co/spaces/Weyaxi/open-llm-leaderboard-results-pr

The purpose of this PR is to add evaluation results from the Open LLM Leaderboard to your model card.

If you encounter any issues, please report them to https://huggingface.co/spaces/Weyaxi/open-llm-leaderboard-results-pr/discussions

Files changed (1) hide show
  1. README.md +119 -4
README.md CHANGED
@@ -1,8 +1,7 @@
1
  ---
2
- license: apache-2.0
3
  language:
4
  - en
5
- pipeline_tag: text-generation
6
  tags:
7
  - maths
8
  - arxiv-math
@@ -10,12 +9,115 @@ tags:
10
  - mathgpt2
11
  datasets:
12
  - ArtifactAI/arxiv-math-instruct-50k
 
13
  widget:
14
  - text: Which motion is formed by an incident particle?
15
  example_title: Example 1
16
  - text: What type of diffusional modeling is used for diffusion?
17
  example_title: Example 2
18
-
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
19
  ---
20
 
21
  This model is a finetuned version of ```gpt2``` using ```ArtifactAI/arxiv-math-instruct-50k```
@@ -50,4 +152,17 @@ prompt.
50
  >>> prompt = "What structure is classified as a definite lie algebra?"
51
  >>> res = generate_text(prompt)
52
  >>> res
53
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
 
2
  language:
3
  - en
4
+ license: apache-2.0
5
  tags:
6
  - maths
7
  - arxiv-math
 
9
  - mathgpt2
10
  datasets:
11
  - ArtifactAI/arxiv-math-instruct-50k
12
+ pipeline_tag: text-generation
13
  widget:
14
  - text: Which motion is formed by an incident particle?
15
  example_title: Example 1
16
  - text: What type of diffusional modeling is used for diffusion?
17
  example_title: Example 2
18
+ model-index:
19
+ - name: math_gpt2
20
+ results:
21
+ - task:
22
+ type: text-generation
23
+ name: Text Generation
24
+ dataset:
25
+ name: AI2 Reasoning Challenge (25-Shot)
26
+ type: ai2_arc
27
+ config: ARC-Challenge
28
+ split: test
29
+ args:
30
+ num_few_shot: 25
31
+ metrics:
32
+ - type: acc_norm
33
+ value: 24.23
34
+ name: normalized accuracy
35
+ source:
36
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Sharathhebbar24/math_gpt2
37
+ name: Open LLM Leaderboard
38
+ - task:
39
+ type: text-generation
40
+ name: Text Generation
41
+ dataset:
42
+ name: HellaSwag (10-Shot)
43
+ type: hellaswag
44
+ split: validation
45
+ args:
46
+ num_few_shot: 10
47
+ metrics:
48
+ - type: acc_norm
49
+ value: 30.88
50
+ name: normalized accuracy
51
+ source:
52
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Sharathhebbar24/math_gpt2
53
+ name: Open LLM Leaderboard
54
+ - task:
55
+ type: text-generation
56
+ name: Text Generation
57
+ dataset:
58
+ name: MMLU (5-Shot)
59
+ type: cais/mmlu
60
+ config: all
61
+ split: test
62
+ args:
63
+ num_few_shot: 5
64
+ metrics:
65
+ - type: acc
66
+ value: 25.38
67
+ name: accuracy
68
+ source:
69
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Sharathhebbar24/math_gpt2
70
+ name: Open LLM Leaderboard
71
+ - task:
72
+ type: text-generation
73
+ name: Text Generation
74
+ dataset:
75
+ name: TruthfulQA (0-shot)
76
+ type: truthful_qa
77
+ config: multiple_choice
78
+ split: validation
79
+ args:
80
+ num_few_shot: 0
81
+ metrics:
82
+ - type: mc2
83
+ value: 39.23
84
+ source:
85
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Sharathhebbar24/math_gpt2
86
+ name: Open LLM Leaderboard
87
+ - task:
88
+ type: text-generation
89
+ name: Text Generation
90
+ dataset:
91
+ name: Winogrande (5-shot)
92
+ type: winogrande
93
+ config: winogrande_xl
94
+ split: validation
95
+ args:
96
+ num_few_shot: 5
97
+ metrics:
98
+ - type: acc
99
+ value: 51.07
100
+ name: accuracy
101
+ source:
102
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Sharathhebbar24/math_gpt2
103
+ name: Open LLM Leaderboard
104
+ - task:
105
+ type: text-generation
106
+ name: Text Generation
107
+ dataset:
108
+ name: GSM8k (5-shot)
109
+ type: gsm8k
110
+ config: main
111
+ split: test
112
+ args:
113
+ num_few_shot: 5
114
+ metrics:
115
+ - type: acc
116
+ value: 0.23
117
+ name: accuracy
118
+ source:
119
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Sharathhebbar24/math_gpt2
120
+ name: Open LLM Leaderboard
121
  ---
122
 
123
  This model is a finetuned version of ```gpt2``` using ```ArtifactAI/arxiv-math-instruct-50k```
 
152
  >>> prompt = "What structure is classified as a definite lie algebra?"
153
  >>> res = generate_text(prompt)
154
  >>> res
155
+ ```
156
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
157
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_Sharathhebbar24__math_gpt2)
158
+
159
+ | Metric |Value|
160
+ |---------------------------------|----:|
161
+ |Avg. |28.50|
162
+ |AI2 Reasoning Challenge (25-Shot)|24.23|
163
+ |HellaSwag (10-Shot) |30.88|
164
+ |MMLU (5-Shot) |25.38|
165
+ |TruthfulQA (0-shot) |39.23|
166
+ |Winogrande (5-shot) |51.07|
167
+ |GSM8k (5-shot) | 0.23|
168
+