Update README.md
Browse files
README.md
CHANGED
@@ -6,8 +6,8 @@ A frankenMoE of [TinyLlama-1.1B-1T-OpenOrca](https://huggingface.co/jeff31415/Ti
|
|
6 |
[TinyLlama-1.1B-intermediate-step-1195k-token-2.5T](https://huggingface.co/TinyLlama/TinyLlama-1.1B-intermediate-step-1195k-token-2.5T),
|
7 |
and [tiny-llama-1.1b-chat-medical](https://huggingface.co/SumayyaAli/tiny-llama-1.1b-chat-medical).
|
8 |
|
9 |
-
# Most 1.1B models are decoherent and can't even answer simple questions. I
|
10 |
|
11 |
OpenOrca experts have been given the task of creating responses for simple questions about things like pop culture, history, and science...step-1195k experts have been chosen to provide warmth and a positive environment, while chat-medical experts have been chosen to provide further detail about human subjects, and to give small little bits of medical advice: I.E. "how do I get rid of this headache I gave myself from making you?"
|
12 |
|
13 |
-
###
|
|
|
6 |
[TinyLlama-1.1B-intermediate-step-1195k-token-2.5T](https://huggingface.co/TinyLlama/TinyLlama-1.1B-intermediate-step-1195k-token-2.5T),
|
7 |
and [tiny-llama-1.1b-chat-medical](https://huggingface.co/SumayyaAli/tiny-llama-1.1b-chat-medical).
|
8 |
|
9 |
+
# Most 1.1B models are decoherent and can't even answer simple questions. I picked out some models that aren't as bad, then mashed 32 copies of those 3 models together into a 32x MoE
|
10 |
|
11 |
OpenOrca experts have been given the task of creating responses for simple questions about things like pop culture, history, and science...step-1195k experts have been chosen to provide warmth and a positive environment, while chat-medical experts have been chosen to provide further detail about human subjects, and to give small little bits of medical advice: I.E. "how do I get rid of this headache I gave myself from making you?"
|
12 |
|
13 |
+
### p.s. ...this is using 32 different experts mashed together, it's more likely to be paranoid schizophrenic than anything else.
|