Update README.md
#12
by
MaziyarPanahi
- opened
README.md
CHANGED
@@ -20,7 +20,7 @@ inference: false
|
|
20 |
model_creator: MaziyarPanahi
|
21 |
quantized_by: MaziyarPanahi
|
22 |
model-index:
|
23 |
-
- name:
|
24 |
results:
|
25 |
- task:
|
26 |
type: text-generation
|
@@ -37,7 +37,7 @@ model-index:
|
|
37 |
value: 63.48
|
38 |
name: normalized accuracy
|
39 |
source:
|
40 |
-
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MaziyarPanahi/
|
41 |
name: Open LLM Leaderboard
|
42 |
- task:
|
43 |
type: text-generation
|
@@ -53,7 +53,7 @@ model-index:
|
|
53 |
value: 80.86
|
54 |
name: normalized accuracy
|
55 |
source:
|
56 |
-
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MaziyarPanahi/
|
57 |
name: Open LLM Leaderboard
|
58 |
- task:
|
59 |
type: text-generation
|
@@ -70,7 +70,7 @@ model-index:
|
|
70 |
value: 69.24
|
71 |
name: accuracy
|
72 |
source:
|
73 |
-
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MaziyarPanahi/
|
74 |
name: Open LLM Leaderboard
|
75 |
- task:
|
76 |
type: text-generation
|
@@ -86,7 +86,7 @@ model-index:
|
|
86 |
- type: mc2
|
87 |
value: 60.66
|
88 |
source:
|
89 |
-
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MaziyarPanahi/
|
90 |
name: Open LLM Leaderboard
|
91 |
- task:
|
92 |
type: text-generation
|
@@ -103,7 +103,7 @@ model-index:
|
|
103 |
value: 72.77
|
104 |
name: accuracy
|
105 |
source:
|
106 |
-
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MaziyarPanahi/
|
107 |
name: Open LLM Leaderboard
|
108 |
- task:
|
109 |
type: text-generation
|
@@ -120,7 +120,7 @@ model-index:
|
|
120 |
value: 74.53
|
121 |
name: accuracy
|
122 |
source:
|
123 |
-
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MaziyarPanahi/
|
124 |
name: Open LLM Leaderboard
|
125 |
- task:
|
126 |
type: text-generation
|
@@ -135,7 +135,7 @@ model-index:
|
|
135 |
value: 49.26
|
136 |
name: strict accuracy
|
137 |
source:
|
138 |
-
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/
|
139 |
name: Open LLM Leaderboard
|
140 |
- task:
|
141 |
type: text-generation
|
@@ -150,7 +150,7 @@ model-index:
|
|
150 |
value: 37.66
|
151 |
name: normalized accuracy
|
152 |
source:
|
153 |
-
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/
|
154 |
name: Open LLM Leaderboard
|
155 |
- task:
|
156 |
type: text-generation
|
@@ -165,7 +165,7 @@ model-index:
|
|
165 |
value: 2.95
|
166 |
name: exact match
|
167 |
source:
|
168 |
-
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/
|
169 |
name: Open LLM Leaderboard
|
170 |
- task:
|
171 |
type: text-generation
|
@@ -180,7 +180,7 @@ model-index:
|
|
180 |
value: 9.06
|
181 |
name: acc_norm
|
182 |
source:
|
183 |
-
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/
|
184 |
name: Open LLM Leaderboard
|
185 |
- task:
|
186 |
type: text-generation
|
@@ -195,7 +195,7 @@ model-index:
|
|
195 |
value: 7.75
|
196 |
name: acc_norm
|
197 |
source:
|
198 |
-
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/
|
199 |
name: Open LLM Leaderboard
|
200 |
- task:
|
201 |
type: text-generation
|
@@ -212,23 +212,23 @@ model-index:
|
|
212 |
value: 31.42
|
213 |
name: accuracy
|
214 |
source:
|
215 |
-
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/
|
216 |
name: Open LLM Leaderboard
|
217 |
---
|
218 |
|
219 |
<img src="./phi-3-instruct.webp" alt="Phi-3 Logo" width="500" style="margin-left:'auto' margin-right:'auto' display:'block'"/>
|
220 |
|
221 |
|
222 |
-
# MaziyarPanahi/
|
223 |
|
224 |
This model is a fine-tune (DPO) of `microsoft/Phi-3-mini-4k-instruct` model.
|
225 |
|
226 |
# ⚡ Quantized GGUF
|
227 |
|
228 |
-
All GGUF models are available here: [MaziyarPanahi/
|
229 |
|
230 |
# 🏆 [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
|
231 |
-
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/
|
232 |
|
233 |
|
234 |
|
@@ -259,7 +259,7 @@ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-le
|
|
259 |
|
260 |
|
261 |
|
262 |
-
`MaziyarPanahi/
|
263 |
|
264 |
![image/png](https://cdn-uploads.huggingface.co/production/uploads/5fd5e18a90b6dc4633f6d292/tKhQ55r7znR4X8GofwYj1.png)
|
265 |
|
@@ -280,7 +280,7 @@ This model uses `ChatML` prompt template:
|
|
280 |
|
281 |
# How to use
|
282 |
|
283 |
-
You can use this model by using `MaziyarPanahi/
|
284 |
transformers library.
|
285 |
|
286 |
```python
|
@@ -288,7 +288,7 @@ from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer
|
|
288 |
from transformers import pipeline
|
289 |
import torch
|
290 |
|
291 |
-
model_id = "MaziyarPanahi/
|
292 |
|
293 |
model = AutoModelForCausalLM.from_pretrained(
|
294 |
model_id,
|
|
|
20 |
model_creator: MaziyarPanahi
|
21 |
quantized_by: MaziyarPanahi
|
22 |
model-index:
|
23 |
+
- name: calme-2.3-phi3-4b
|
24 |
results:
|
25 |
- task:
|
26 |
type: text-generation
|
|
|
37 |
value: 63.48
|
38 |
name: normalized accuracy
|
39 |
source:
|
40 |
+
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MaziyarPanahi/calme-2.3-phi3-4b
|
41 |
name: Open LLM Leaderboard
|
42 |
- task:
|
43 |
type: text-generation
|
|
|
53 |
value: 80.86
|
54 |
name: normalized accuracy
|
55 |
source:
|
56 |
+
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MaziyarPanahi/calme-2.3-phi3-4b
|
57 |
name: Open LLM Leaderboard
|
58 |
- task:
|
59 |
type: text-generation
|
|
|
70 |
value: 69.24
|
71 |
name: accuracy
|
72 |
source:
|
73 |
+
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MaziyarPanahi/calme-2.3-phi3-4b
|
74 |
name: Open LLM Leaderboard
|
75 |
- task:
|
76 |
type: text-generation
|
|
|
86 |
- type: mc2
|
87 |
value: 60.66
|
88 |
source:
|
89 |
+
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MaziyarPanahi/calme-2.3-phi3-4b
|
90 |
name: Open LLM Leaderboard
|
91 |
- task:
|
92 |
type: text-generation
|
|
|
103 |
value: 72.77
|
104 |
name: accuracy
|
105 |
source:
|
106 |
+
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MaziyarPanahi/calme-2.3-phi3-4b
|
107 |
name: Open LLM Leaderboard
|
108 |
- task:
|
109 |
type: text-generation
|
|
|
120 |
value: 74.53
|
121 |
name: accuracy
|
122 |
source:
|
123 |
+
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MaziyarPanahi/calme-2.3-phi3-4b
|
124 |
name: Open LLM Leaderboard
|
125 |
- task:
|
126 |
type: text-generation
|
|
|
135 |
value: 49.26
|
136 |
name: strict accuracy
|
137 |
source:
|
138 |
+
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/calme-2.3-phi3-4b
|
139 |
name: Open LLM Leaderboard
|
140 |
- task:
|
141 |
type: text-generation
|
|
|
150 |
value: 37.66
|
151 |
name: normalized accuracy
|
152 |
source:
|
153 |
+
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/calme-2.3-phi3-4b
|
154 |
name: Open LLM Leaderboard
|
155 |
- task:
|
156 |
type: text-generation
|
|
|
165 |
value: 2.95
|
166 |
name: exact match
|
167 |
source:
|
168 |
+
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/calme-2.3-phi3-4b
|
169 |
name: Open LLM Leaderboard
|
170 |
- task:
|
171 |
type: text-generation
|
|
|
180 |
value: 9.06
|
181 |
name: acc_norm
|
182 |
source:
|
183 |
+
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/calme-2.3-phi3-4b
|
184 |
name: Open LLM Leaderboard
|
185 |
- task:
|
186 |
type: text-generation
|
|
|
195 |
value: 7.75
|
196 |
name: acc_norm
|
197 |
source:
|
198 |
+
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/calme-2.3-phi3-4b
|
199 |
name: Open LLM Leaderboard
|
200 |
- task:
|
201 |
type: text-generation
|
|
|
212 |
value: 31.42
|
213 |
name: accuracy
|
214 |
source:
|
215 |
+
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/calme-2.3-phi3-4b
|
216 |
name: Open LLM Leaderboard
|
217 |
---
|
218 |
|
219 |
<img src="./phi-3-instruct.webp" alt="Phi-3 Logo" width="500" style="margin-left:'auto' margin-right:'auto' display:'block'"/>
|
220 |
|
221 |
|
222 |
+
# MaziyarPanahi/calme-2.3-phi3-4b
|
223 |
|
224 |
This model is a fine-tune (DPO) of `microsoft/Phi-3-mini-4k-instruct` model.
|
225 |
|
226 |
# ⚡ Quantized GGUF
|
227 |
|
228 |
+
All GGUF models are available here: [MaziyarPanahi/calme-2.3-phi3-4b-GGUF](https://huggingface.co/MaziyarPanahi/calme-2.3-phi3-4b-GGUF)
|
229 |
|
230 |
# 🏆 [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
|
231 |
+
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_MaziyarPanahi__calme-2.3-phi3-4b)
|
232 |
|
233 |
|
234 |
|
|
|
259 |
|
260 |
|
261 |
|
262 |
+
`MaziyarPanahi/calme-2.3-phi3-4b` is the best-performing Phi-3-mini-4k model on the Open LLM Leaderboard. (03/06/2024).
|
263 |
|
264 |
![image/png](https://cdn-uploads.huggingface.co/production/uploads/5fd5e18a90b6dc4633f6d292/tKhQ55r7znR4X8GofwYj1.png)
|
265 |
|
|
|
280 |
|
281 |
# How to use
|
282 |
|
283 |
+
You can use this model by using `MaziyarPanahi/calme-2.3-phi3-4b` as the model name in Hugging Face's
|
284 |
transformers library.
|
285 |
|
286 |
```python
|
|
|
288 |
from transformers import pipeline
|
289 |
import torch
|
290 |
|
291 |
+
model_id = "MaziyarPanahi/calme-2.3-phi3-4b"
|
292 |
|
293 |
model = AutoModelForCausalLM.from_pretrained(
|
294 |
model_id,
|