This is a 3-bit exl2 (Exllamav2) conversion of MistralAI's Mixtral-8x7B-Instruct-v0.1

It takes about 17GB on disk, so it should load on consumer cards with 24GB of VRAM.

More on ExLlamav2 -> https://github.com/turboderp/exllamav2 ( works with Oobabooga's textgen web ui )
Original Model -> https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1 ( Original files are ~87GB on disk )

Downloads last month: 2

Inference Providers NEW

This model is not currently available via any of the supported third-party Inference Providers, and the model is not deployed on the HF Inference API.