comaniac
/

Mixtral-8x22B-Instruct-v0.1-FP8-v2

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

Mixtral-8x22B-Instruct-v0.1-FP8-v2 / README.md

comaniac's picture

Upload folder using huggingface_hub

fee9f43 verified about 1 month ago

|

No virus

248 Bytes

Mixtral-8x22B-Instruct-v0.1-FP8-v1

Weights and activations are per-tensor quantized to float8_e4m3.
Quantization with AutoFP8.
Calibration dataset: Ultrachat (mgoin/ultrachat_2k)
Samples: 2048
Sequence length: 8192

Evaluation

TBA