comaniac
/

Mixtral-8x22B-Instruct-v0.1-FP8-v2

Text Generation

Inference Endpoints

text-generation-inference

Model card Files Files and versions Community

Edit model card

YAML Metadata Warning: empty or missing yaml metadata in repo card (https://huggingface.co/docs/hub/model-cards#model-card-metadata)

Mixtral-8x22B-Instruct-v0.1-FP8-v2

Weights and activations are per-tensor quantized to float8_e4m3.
Quantization with AutoFP8 with updated activation scaling factor names.
Calibration dataset: Ultrachat (mgoin/ultrachat_2k)
Samples: 2048
Sequence length: 8192

Evaluation

TBA

Downloads last month: 9

Safetensors

Model size

141B params

Tensor type

BF16

·

F8_E4M3

·