FuturisticVibes/dolphin-2.9.2-mixtral-8x22b-6.0bpw-h8-exl2

I have no idea what I’m doing… if this causes the apocalypse someone please let me know.

dolphin-2.9.2-mixtral-8x22b 6.0bpw h8 EXL2

Includes measurement.json file for further quantization

Original Model: https://huggingface.co/cognitivecomputations/dolphin-2.9.2-mixtral-8x22b

Original Model Card

Dolphin 2.9.2 Mixtral 8x22b 🐬

Curated and trained by Eric Hartford, Lucas Atkins, and Fernando Fernandes, and Cognitive Computations

Discord: https://discord.gg/cognitivecomputations

New in 2.9.2 is SystemChat 2.0 - a dataset designed to teach Dolphin to obey the system prompt, even over a long conversation.

My appreciation for the sponsors of Dolphin 2.9.2:

• Crusoe Cloud - provided excellent on-demand 8xH100 node
• OnDemand - provided inference sponsorship, enabling creation of SystemChat

This model is based on Dolphin-2.9-Mixtral-8x22b, and is Apache-2.0 licensed.

The base model has 64k context, and fine-tuning was with 16k sequence length.

It took 1 week on 8xH100 provided by Crusoe Cloud

This model was trained FFT on 50% parameters (targeted with Laser Scanner by Fernando Fernandes, David Golchinfar, Lucas Atkins, and Eric Hartford), using ChatML prompt template format.

example:

<|im_start|>system
You are Dolphin, a helpful AI assistant.<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant

Dolphin-2.9 has a variety of instruction, conversational, and coding skills. It also has initial agentic abilities and supports function calling.

Dolphin is uncensored. I have filtered the dataset to remove alignment and bias. This makes the model more compliant. You are advised to implement your own alignment layer before exposing the model as a service. It will be highly compliant with any requests, even unethical ones. Please read my blog post about uncensored models. https://erichartford.com/uncensored-models You are responsible for any content you create using this model. Enjoy responsibly.

Dolphin is licensed Apache 2.0. I grant permission for any use, including commercial, that falls within accordance with Apache-2.0 license. Dolphin was trained on data generated from GPT4, among other models.

Evals

Training