Hugging Face's logo

RichardErkhov
/
EpistemeAI_-_Fireball-Alpaca-Llama3.1.07-8B-Philos-Math-KTO-beta-4bits

Safetensors
llama
4-bit precision
bitsandbytes
Model card Files Community
RichardErkhov commited on
Commit
eaba404
•
1 Parent(s): 2ab4d4a

uploaded readme

Browse files
Files changed (1) hide show
  1. README.md +161 -0
README.md ADDED
@@ -0,0 +1,161 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Quantization made by Richard Erkhov.
2
+
3
+ [Github](https://github.com/RichardErkhov)
4
+
5
+ [Discord](https://discord.gg/pvy7H8DZMG)
6
+
7
+ [Request more models](https://github.com/RichardErkhov/quant_request)
8
+
9
+
10
+ Fireball-Alpaca-Llama3.1.07-8B-Philos-Math-KTO-beta - bnb 4bits
11
+ - Model creator: https://huggingface.co/EpistemeAI/
12
+ - Original model: https://huggingface.co/EpistemeAI/Fireball-Alpaca-Llama3.1.07-8B-Philos-Math-KTO-beta/
13
+
14
+
15
+
16
+
17
+ Original model description:
18
+ ---
19
+ language:
20
+ - en
21
+ widget:
22
+ - text: "My name is Julien and I like to"
23
+ example_title: "Julien"
24
+ - text: "My name is Merve and my favorite"
25
+ example_title: "Merve"
26
+
27
+ license: apache-2.0
28
+ tags:
29
+ - text-generation-inference
30
+ - transformers
31
+ - unsloth
32
+ - llama
33
+ - trl
34
+ base_model: EpistemeAI2/Fireball-Alpaca-Llama3.1.07-8B-Philos-Math
35
+ model-index:
36
+ - name: Fireball-Alpaca-Llama3.1.07-8B-Philos-Math-KTO-beta
37
+ results:
38
+ - task:
39
+ type: text-generation
40
+ name: Text Generation
41
+ dataset:
42
+ name: IFEval (0-Shot)
43
+ type: HuggingFaceH4/ifeval
44
+ args:
45
+ num_few_shot: 0
46
+ metrics:
47
+ - type: inst_level_strict_acc and prompt_level_strict_acc
48
+ value: 72.74
49
+ name: strict accuracy
50
+ source:
51
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=EpistemeAI/Fireball-Alpaca-Llama3.1.07-8B-Philos-Math-KTO-beta
52
+ name: Open LLM Leaderboard
53
+ - task:
54
+ type: text-generation
55
+ name: Text Generation
56
+ dataset:
57
+ name: BBH (3-Shot)
58
+ type: BBH
59
+ args:
60
+ num_few_shot: 3
61
+ metrics:
62
+ - type: acc_norm
63
+ value: 26.9
64
+ name: normalized accuracy
65
+ source:
66
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=EpistemeAI/Fireball-Alpaca-Llama3.1.07-8B-Philos-Math-KTO-beta
67
+ name: Open LLM Leaderboard
68
+ - task:
69
+ type: text-generation
70
+ name: Text Generation
71
+ dataset:
72
+ name: MATH Lvl 5 (4-Shot)
73
+ type: hendrycks/competition_math
74
+ args:
75
+ num_few_shot: 4
76
+ metrics:
77
+ - type: exact_match
78
+ value: 13.22
79
+ name: exact match
80
+ source:
81
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=EpistemeAI/Fireball-Alpaca-Llama3.1.07-8B-Philos-Math-KTO-beta
82
+ name: Open LLM Leaderboard
83
+ - task:
84
+ type: text-generation
85
+ name: Text Generation
86
+ dataset:
87
+ name: GPQA (0-shot)
88
+ type: Idavidrein/gpqa
89
+ args:
90
+ num_few_shot: 0
91
+ metrics:
92
+ - type: acc_norm
93
+ value: 4.03
94
+ name: acc_norm
95
+ source:
96
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=EpistemeAI/Fireball-Alpaca-Llama3.1.07-8B-Philos-Math-KTO-beta
97
+ name: Open LLM Leaderboard
98
+ - task:
99
+ type: text-generation
100
+ name: Text Generation
101
+ dataset:
102
+ name: MuSR (0-shot)
103
+ type: TAUR-Lab/MuSR
104
+ args:
105
+ num_few_shot: 0
106
+ metrics:
107
+ - type: acc_norm
108
+ value: 4.28
109
+ name: acc_norm
110
+ source:
111
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=EpistemeAI/Fireball-Alpaca-Llama3.1.07-8B-Philos-Math-KTO-beta
112
+ name: Open LLM Leaderboard
113
+ - task:
114
+ type: text-generation
115
+ name: Text Generation
116
+ dataset:
117
+ name: MMLU-PRO (5-shot)
118
+ type: TIGER-Lab/MMLU-Pro
119
+ config: main
120
+ split: test
121
+ args:
122
+ num_few_shot: 5
123
+ metrics:
124
+ - type: acc
125
+ value: 28.26
126
+ name: accuracy
127
+ source:
128
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=EpistemeAI/Fireball-Alpaca-Llama3.1.07-8B-Philos-Math-KTO-beta
129
+ name: Open LLM Leaderboard
130
+ ---
131
+
132
+
133
+ # KTO Fine tuning!
134
+
135
+ A [**KTO**](https://arxiv.org/abs/2402.01306) version EpistemeAI2/Fireball-Alpaca-Llama3.1.07-8B-Philos-Math
136
+
137
+ # Uploaded model
138
+
139
+ - **Developed by:** EpistemeAI2
140
+ - **License:** apache-2.0
141
+ - **Finetuned from model :** EpistemeAI2/Fireball-Alpaca-Llama3.1.07-8B-Philos-Math
142
+
143
+ This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
144
+
145
+ [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
146
+
147
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
148
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_EpistemeAI__Fireball-Alpaca-Llama3.1.07-8B-Philos-Math-KTO-beta)
149
+
150
+ | Metric |Value|
151
+ |-------------------|----:|
152
+ |Avg. |24.90|
153
+ |IFEval (0-Shot) |72.74|
154
+ |BBH (3-Shot) |26.90|
155
+ |MATH Lvl 5 (4-Shot)|13.22|
156
+ |GPQA (0-shot) | 4.03|
157
+ |MuSR (0-shot) | 4.28|
158
+ |MMLU-PRO (5-shot) |28.26|
159
+
160
+
161
+