JayhC
/

Mixtral-8x7B-Instruct-v0.1-4.5bpw-h6-exl2

Text Generation

Inference Endpoints

text-generation-inference

Model card Files Files and versions Community

JayhC commited on Mar 26

Commit

e6db444

•

1 Parent(s): 4160409

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -16,7 +16,7 @@ widget:
 ---
-4.5bpw/h6 exl2 quantization of [mistralai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1) using default exllamav2 calibration dataset, to fully use my 31gb VRAM (-1 cuz windows..).
 ---

 ---
+4.5bpw/h6 exl2 quantization of [mistralai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1) using default exllamav2 calibration dataset, to fully use my 31gb VRAM (-1 cuz windows..) at 16k-32k context.
 ---