jarrelscy
/

Mixtral-8x22B-Instruct-v0.1-GPTQ-8bit

Text Generation

Inference Endpoints

text-generation-inference

8-bit precision

Model card Files Files and versions Community

Edit model card

Model Card for Model ID

Optimum GPTQ quantized 8-bit version of Mixtral-8x22B-Instruct-v0.1
WIP - do not use currently
See original model card for more information.

How to load

Downloads last month: 8

Safetensors

Model size

36.9B params

Tensor type

I32

·

FP16

·