README.md

---
base_model:
- Doctor-Shotgun/Mixtral-8x7B-Instruct-v0.1-LimaRP-ZLoss
- jondurbin/bagel-dpo-8x7b-v0.2
- NeverSleep/Noromaid-v0.4-Mixtral-Instruct-8x7b-Zloss
license: cc-by-nc-4.0
tags:
- not-for-all-audiences
- nsfw
- mergekit
- merge
- exllamav2
---

# Undi95/BagelMix-8x7B 3.5bpw
Exllama quant of [Undi95/BagelMix-8x7B](https://huggingface.co/Undi95/BagelMix-8x7B)

You will need 24gb of vram to run this model at about half context (16k, you can probably go a bit higher too)


### Prompt format:
ChatML? maybe? Unclear

```
<|im_start|>system
{sysprompt}<|im_end|>
<|im_start|>user
{input}<|im_end|>
<|im_start|>assistant
{output}<|im_end|>
```

### Contact
Kooten on discord.