Isaak Carter Augustus commited on
Commit
67299ca
1 Parent(s): dd33b23

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +187 -0
README.md CHANGED
@@ -1,6 +1,7 @@
1
  ---
2
  license: apache-2.0
3
  tags:
 
4
  - moe
5
  - frankenmoe
6
  - merge
@@ -103,8 +104,194 @@ base_model:
103
  - Felladrin/Smol-Llama-101M-Chat-v1
104
  - Felladrin/Smol-Llama-101M-Chat-v1
105
  - Felladrin/Smol-Llama-101M-Chat-v1
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
106
  ---
107
 
 
108
  # SmalJ.O.S.I.E.-48x101M-Chat
109
 
110
  SmalJ.O.S.I.E.-48x101M-Chat is a Mixure of Experts (MoE) made with the following models using [LazyMergekit](https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing):
 
1
  ---
2
  license: apache-2.0
3
  tags:
4
+ - text-generation
5
  - moe
6
  - frankenmoe
7
  - merge
 
104
  - Felladrin/Smol-Llama-101M-Chat-v1
105
  - Felladrin/Smol-Llama-101M-Chat-v1
106
  - Felladrin/Smol-Llama-101M-Chat-v1
107
+ datasets:
108
+ - Open-Orca/SlimOrca-Dedup
109
+ - VMware/open-instruct
110
+ - LDJnr/Capybara
111
+ - cognitivecomputations/ultrachat-uncensored
112
+ - starfishmedical/webGPT_x_dolly
113
+ - THUDM/webglm-qa
114
+ widget:
115
+ - messages:
116
+ - role: system
117
+ content: You are a helpful assistant who gives creative responses.
118
+ - role: user
119
+ content: Write the background story of a game about wizards and llamas in a sci-fi world.
120
+ - messages:
121
+ - role: system
122
+ content: A friendly chat between a user and an assistant.
123
+ - role: user
124
+ content: Got a question for you!
125
+ - role: assistant
126
+ content: "Sure! What's it?"
127
+ - role: user
128
+ content: I need to build a simple website. Where should I start learning about web development?
129
+ - messages:
130
+ - role: system
131
+ content: "You are a helpful assistant who provides concise answers to the user's questions."
132
+ - role: user
133
+ content: How to become more healthy?
134
+ - messages:
135
+ - role: system
136
+ content: You are a helpful assistant, who always answers with empathy.
137
+ - role: user
138
+ content: List the pros and cons of social media.
139
+ - messages:
140
+ - role: system
141
+ content: You are a helpful assistant, who always answers with empathy.
142
+ - role: user
143
+ content: Hello!
144
+ - role: assistant
145
+ content: Hi! How can I help you today?
146
+ - role: user
147
+ content: 'Take a look at the info below.
148
+
149
+ - The tape inside the VHS cassettes is very delicate and can be easily ruined,
150
+ making them unplayable and unrepairable. The reason the tape deteriorates is that
151
+ the magnetic charge needed for them to work is not permanent, and the magnetic
152
+ particles end up losing their charge in a process known as remanence decay. These
153
+ particles could also become demagnetised via being stored too close to a magnetic
154
+ source.
155
+
156
+ - One of the most significant issues with VHS tapes is that they have moving parts,
157
+ meaning that there are more occasions when something can go wrong, damaging your
158
+ footage or preventing it from playing back. The tape itself is a prominent cause
159
+ of this, and tape slippage can occur. Tapes slippage can be caused when the tape
160
+ loses its tension, or it has become warped. These problems can occur in storage
161
+ due to high temperatures or frequent changes in humidity.
162
+
163
+ - VHS tapes deteriorate over time from infrequent or overuse. Neglect means mold
164
+ and dirt, while overuse can lead to scratches and technical difficulties. This
165
+ is why old VHS tapes inevitably experience malfunctions after a long period of
166
+ time. Usually anywhere between 10 to 25+ years.
167
+
168
+ - Some VHS tapes like newer mini DVs and Digital 8 tapes can suffer from digital
169
+ corruption, meaning that the footage becomes lost and cannot be recovered. These
170
+ tapes were the steppingstone from VHS to the digital age when capturing footage
171
+ straight to digital became the norm. Unfortunately,they are susceptible to digital
172
+ corruption, which causes video pixilation and/or loss of audio.'
173
+ - role: assistant
174
+ content: Alright!
175
+ - role: user
176
+ content: 'Now I''m going to write my question, and if the info above is useful, you can use them in your response.
177
+
178
+ Ready?'
179
+ - role: assistant
180
+ content: Ready for your question!
181
+ - role: user
182
+ content: Why do VHS tapes deteriorate over time?
183
+ inference:
184
+ parameters:
185
+ max_new_tokens: 250
186
+ penalty_alpha: 0.5
187
+ top_k: 4
188
+ repetition_penalty: 1.105
189
+ model-index:
190
+ - name: Smol-Llama-101M-Chat-v1
191
+ results:
192
+ - task:
193
+ type: text-generation
194
+ name: Text Generation
195
+ dataset:
196
+ name: AI2 Reasoning Challenge (25-Shot)
197
+ type: ai2_arc
198
+ config: ARC-Challenge
199
+ split: test
200
+ args:
201
+ num_few_shot: 25
202
+ metrics:
203
+ - type: acc_norm
204
+ value: 22.87
205
+ name: normalized accuracy
206
+ source:
207
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Felladrin/Smol-Llama-101M-Chat-v1
208
+ name: Open LLM Leaderboard
209
+ - task:
210
+ type: text-generation
211
+ name: Text Generation
212
+ dataset:
213
+ name: HellaSwag (10-Shot)
214
+ type: hellaswag
215
+ split: validation
216
+ args:
217
+ num_few_shot: 10
218
+ metrics:
219
+ - type: acc_norm
220
+ value: 28.69
221
+ name: normalized accuracy
222
+ source:
223
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Felladrin/Smol-Llama-101M-Chat-v1
224
+ name: Open LLM Leaderboard
225
+ - task:
226
+ type: text-generation
227
+ name: Text Generation
228
+ dataset:
229
+ name: MMLU (5-Shot)
230
+ type: cais/mmlu
231
+ config: all
232
+ split: test
233
+ args:
234
+ num_few_shot: 5
235
+ metrics:
236
+ - type: acc
237
+ value: 24.93
238
+ name: accuracy
239
+ source:
240
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Felladrin/Smol-Llama-101M-Chat-v1
241
+ name: Open LLM Leaderboard
242
+ - task:
243
+ type: text-generation
244
+ name: Text Generation
245
+ dataset:
246
+ name: TruthfulQA (0-shot)
247
+ type: truthful_qa
248
+ config: multiple_choice
249
+ split: validation
250
+ args:
251
+ num_few_shot: 0
252
+ metrics:
253
+ - type: mc2
254
+ value: 45.76
255
+ source:
256
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Felladrin/Smol-Llama-101M-Chat-v1
257
+ name: Open LLM Leaderboard
258
+ - task:
259
+ type: text-generation
260
+ name: Text Generation
261
+ dataset:
262
+ name: Winogrande (5-shot)
263
+ type: winogrande
264
+ config: winogrande_xl
265
+ split: validation
266
+ args:
267
+ num_few_shot: 5
268
+ metrics:
269
+ - type: acc
270
+ value: 50.04
271
+ name: accuracy
272
+ source:
273
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Felladrin/Smol-Llama-101M-Chat-v1
274
+ name: Open LLM Leaderboard
275
+ - task:
276
+ type: text-generation
277
+ name: Text Generation
278
+ dataset:
279
+ name: GSM8k (5-shot)
280
+ type: gsm8k
281
+ config: main
282
+ split: test
283
+ args:
284
+ num_few_shot: 5
285
+ metrics:
286
+ - type: acc
287
+ value: 0.08
288
+ name: accuracy
289
+ source:
290
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Felladrin/Smol-Llama-101M-Chat-v1
291
+ name: Open LLM Leaderboard
292
  ---
293
 
294
+
295
  # SmalJ.O.S.I.E.-48x101M-Chat
296
 
297
  SmalJ.O.S.I.E.-48x101M-Chat is a Mixure of Experts (MoE) made with the following models using [LazyMergekit](https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing):