RichardErkhov/wandb_-_mistral-7b-zephyr-sft-4bits

Quantization made by Richard Erkhov.

Github

Discord

Request more models

mistral-7b-zephyr-sft - bnb 4bits

• Model creator: https://huggingface.co/wandb/
• Original model: https://huggingface.co/wandb/mistral-7b-zephyr-sft/

Original model description:

license: mit library_name: transformers datasets: - HuggingFaceH4/deita-10k-v0-sft base_model: mistralai/Mistral-7B-v0.1

Mistral 7B Zephyr SFT V2

The Zephyr SFT recipe applied on top of Mistral 7B (new recipe with chatML format)

Model description

• Model type: A 7.2B parameter GPT-like model fine-tuned on a mix of publicly available, synthetic datasets.
• Language(s) (NLP): Primarily English
• Finetuned from model: mistralai/Mistral-7B-v0.1

Recipe

We trained using the alignment handbook recipe and logging to W&B

Visit the W&B workspace here

Compute provided by Lambda Labs - 8xA100 80GB node