mph/d3-turbo-MXFP8

d3-turbo-MXFP8

MXFP8 quantization of vllm-sr/d3-turbo, produced with NVIDIA TensorRT Model Optimizer (ModelOpt) v0.47.0.

d3-turbo reads one input (text or JSON, optionally with images and videos) and answers every question on it — choice, yes/no, score, box, or track — with a probability for every option. Load this checkpoint with trust_remote_code=True and call system_one.

What is quantized

MXFP8 stores weights and activations as FP8 (E4M3) values with a shared power-of-two (E8M0) scale for every block of 32 elements. This gives roughly half the weight memory of BF16 and keeps accuracy close to it, because the scaling is fine-grained. Quantized Linear weights are packed into MXFP8QTensor and restored from model.safetensors together with modelopt_state.pth.

The following modules are left in BF16:

• Language-model embeddings (language_model.embed_tokens)
• Gated DeltaNet convolution and in_proj_a / in_proj_b
• All vision modules (visual.*)

The 255-way answer-code readout (readout.safetensors) stays FP32. The backbone has no lm_head and no MTP modules. The KV cache stays BF16.

Quantized layers are the language-model linears: full-attention q/k/v/o, MLP gate/up/down, and Gated DeltaNet in_proj_qkv, in_proj_z, and out_proj.

Hardware requirements

• Native MXFP8 compute needs NVIDIA Blackwell (SM100/SM120) GPUs.
• Weights take about 27 GB. The runtime loads onto one device, so one 32 GB+ GPU can hold the weights. Add headroom for vision activations and the per-request state cache.