Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,27 @@
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
---
|
| 4 |
+
<!DOCTYPE html>
|
| 5 |
+
<img src="https://imgur.com/a/3HUIVxJ" width="300">
|
| 6 |
+
|
| 7 |
+
# Mistral 29b: A New Base Model
|
| 8 |
+
|
| 9 |
+
The objective of this model is to serve as a new base model for Mistral 28b.
|
| 10 |
+
|
| 11 |
+
This model spits out jargon, and needs to be fine tuned, either with qlora, with the adapter attached to every layer, or better yet a full fine tune.
|
| 12 |
+
|
| 13 |
+
## Model Creation
|
| 14 |
+
|
| 15 |
+
The model was created by merging stacking four models: Dolphin, Zephyr, Meta-math7b, and Speechless code, to form a single model.
|
| 16 |
+
|
| 17 |
+
## Useful Resources
|
| 18 |
+
|
| 19 |
+
- Model Merging (MergeKit) (https://github.com/cg123/mergekit)
|
| 20 |
+
|
| 21 |
+
## Source Models
|
| 22 |
+
|
| 23 |
+
- Dolphin2.1-mistral-7b by Eric Hartford (https://huggingface.co/ehartford/dolphin-2.1-mistral-7b)
|
| 24 |
+
- Zephyr-7b-beta by HuggingFace (https://huggingface.co/HuggingFaceH4/zephyr-7b-beta)
|
| 25 |
+
- MetaMath-Mistral-7B by meta-math (https://huggingface.co/meta-math/MetaMath-Mistral-7B)
|
| 26 |
+
- speechless-code-mistral-7b-v1.0 (https://huggingface.co/uukuguy/speechless-code-mistral-7b-v1.0)
|
| 27 |
+
|