File size: 987 Bytes
b5c6898 5626e8e |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
---
license: apache-2.0
---
<!DOCTYPE html>
<img src="https://imgur.com/a/3HUIVxJ" width="300">
# Mistral 29b: A New Base Model
The objective of this model is to serve as a new base model for Mistral 28b.
This model spits out jargon, and needs to be fine tuned, either with qlora, with the adapter attached to every layer, or better yet a full fine tune.
## Model Creation
The model was created by merging stacking four models: Dolphin, Zephyr, Meta-math7b, and Speechless code, to form a single model.
## Useful Resources
- Model Merging (MergeKit) (https://github.com/cg123/mergekit)
## Source Models
- Dolphin2.1-mistral-7b by Eric Hartford (https://huggingface.co/ehartford/dolphin-2.1-mistral-7b)
- Zephyr-7b-beta by HuggingFace (https://huggingface.co/HuggingFaceH4/zephyr-7b-beta)
- MetaMath-Mistral-7B by meta-math (https://huggingface.co/meta-math/MetaMath-Mistral-7B)
- speechless-code-mistral-7b-v1.0 (https://huggingface.co/uukuguy/speechless-code-mistral-7b-v1.0)
|