metadata
license: apache-2.0
Mistral 29b: A New Base Model
The objective of this model is to serve as a new base model for Mistral 28b.
This model spits out jargon, and needs to be fine tuned, either with qlora, with the adapter attached to every layer, or better yet a full fine tune.
Model Creation
The model was created by merging stacking four models: Dolphin, Zephyr, Meta-math7b, and Speechless code, to form a single model.
Useful Resources
- Model Merging (MergeKit) (https://github.com/cg123/mergekit)
Source Models
- Dolphin2.1-mistral-7b by Eric Hartford (https://huggingface.co/ehartford/dolphin-2.1-mistral-7b)
- Zephyr-7b-beta by HuggingFace (https://huggingface.co/HuggingFaceH4/zephyr-7b-beta)
- MetaMath-Mistral-7B by meta-math (https://huggingface.co/meta-math/MetaMath-Mistral-7B)
- speechless-code-mistral-7b-v1.0 (https://huggingface.co/uukuguy/speechless-code-mistral-7b-v1.0)