Commit
•
5e087a7
1
Parent(s):
f572ae4
Update README.md
Browse files
README.md
CHANGED
|
@@ -1,5 +1,11 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
# Demixtral
|
| 2 |
|
| 3 |
Contains eight Mistral models using the MLP layers from the corresponding Mixtral expert. Note that the ordering of experts between layers is arbitrary, so this is not the only (and almost certainly not the best) extraction possible.
|
| 4 |
|
| 5 |
-
None of these are usable on their own. No coherent text will come out of any of these. This is purely for research and experimentation.
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
tags:
|
| 4 |
+
- mistral
|
| 5 |
+
- mixtral
|
| 6 |
+
---
|
| 7 |
# Demixtral
|
| 8 |
|
| 9 |
Contains eight Mistral models using the MLP layers from the corresponding Mixtral expert. Note that the ordering of experts between layers is arbitrary, so this is not the only (and almost certainly not the best) extraction possible.
|
| 10 |
|
| 11 |
+
None of these are usable on their own. No coherent text will come out of any of these. This is purely for research and experimentation.
|