comaniac
/

Mixtral-8x7B-Instruct-v0.1-FP8-v2

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

Mixtral-8x7B-Instruct-v0.1-FP8-v2 / README.md

comaniac's picture

Update README.md

200f867 verified 5 months ago

|

231 Bytes

Mixtral-8x7B-Instruct-v0.1-FP8-v2

Weights and activations are per-tensor quantized to float8_e4m3.
Quantization with AutoFP8.
Calibration dataset: Ultrachat-200k
Samples: 4096
Sequence length: 8192

Evaluation

TBA