intel-ai/MiniMax-H3-MXFP8-CT-AutoRound-tmp

MiniMax-H3 — MXFP8 (weight-quantized, vLLM-Omni native layout)

An MXFP8 (OCP microscaling FP8) weight-quantized build of MiniMaxAI/MiniMax-H3 — the omni-modal (video + native stereo audio) generative DiT — produced with Intel auto-round (zero-shot RTN) and converted to the native fused layout that vLLM-Omni serves. Both diffusion transformers are quantized; the large AdaLN branch is deliberately kept in BF16.

TL;DR

• Each DiT ~62 GB (bf16) → ~43.65 GB (≈0.71×, −29 %); AdaLN (≈13 B params/DiT) stays BF16 because inference caches it and it never hits a matmul.
• Quality is effectively unchanged: VBench imaging_quality 0.6306 (bf16) → 0.6313 (mxfp8), Δ +0.12 % over 93 prompts.
• Serves end-to-end on 2×H200 (TP2) and produces video with synchronized audio. H200 runs the FP8 weights through a Marlin dequant kernel (correct, memory-saving, not faster).

⚠️ Research / evaluation artifact. H3 has native/diffusers dual packaging; this repo is the native fused checkpoint (serve-ready). A diffusers-layout quant output also exists at MiniMax-H3-mxfp8/ (not directly servable — see below).

---

Showcase — official T2VA example, bf16 vs MXFP8

Reproduction of the official MiniMax-H3 Text-to-Audio-Video (T2VA) showcase (assets/t2va.mp4), generated on our own vLLM-Omni stack with identical settings (1344×768, 24 fps, 32 steps, 10 s, seed 0, flow-shift 12 / audio-flow-shift 3.0, task t2va) so the only variable is the quantization. Prompt = the official H3-Context-IR "starship captain" T2VA prompt.

bf16 (reference)	MXFP8 (this model)
assets/t2va_official_bf16.mp4	assets/t2va_official_mxfp8.mp4

Both clips: 1344×768, 24 fps, 243 frames (10.12 s), video + stereo audio, matching the official t2va.mp4 geometry (1344×768 / 24 fps / 10.12 s).

Metric (this single example)	bf16	MXFP8
VBench imaging_quality	0.6671	0.6614
PSNR (mxfp8 vs bf16)	—	14.69 dB
SSIM (mxfp8 vs bf16)	—	0.5589
LPIPS (mxfp8 vs bf16)	—	0.3804

How to read the PSNR/SSIM/LPIPS. They are moderate, not near-identical, because the two videos are not pixel-for-pixel copies. A ~2–3 % MXFP8 weight perturbation, compounded over the denoising steps, nudges the sampler onto a different but equally valid trajectory — same scene, motion and soundscape, slightly different pixels. The matching VBench imaging_quality (and the 93-prompt result below) confirm there is no systematic quality loss; the reference-metric distance is sampling chaos, not degradation. (The distance is a little larger than Wan2.2's here because this clip is longer/higher-fps, giving more room for trajectory divergence.)

---

What was quantized

H3 has two DiTs sharing one architecture:

Component	Role	Quantized?
transformer	FL2VA / T2VA DiT (33.12 B)	✅ MXFP8 (from BF16)
transformer_ref (diffusers layout) / Ref2VA	Ref2VA DiT (33.12 B)	✅ MXFP8 (from BF16)
text_encoder (Qwen3-VL)	text/vision encoding	❌ kept bf16
vae / audio_vae	video / audio decode	❌ unchanged
adaln_proj.linear (×52/DiT)	AdaLN modulation (~13 B)	❌ kept BF16 on purpose

Quantized Linears — 6 per block × 52 = 312 per DiT: attn.{to_q,to_k,to_v,to_out.0} + ff.net.0.proj (SwiGLU gate+up) + ff.net.2 (down). Per DiT: 312 × F8_E4M3 weights + 312 × U8 E8M0 block scales + 314 × BF16 (AdaLN, norms, embedders, biases).

Why AdaLN stays BF16. The official card notes ~13 B params live in AdaLN, and because AdaLN modulation can be precomputed and cached, pure inference never loads them into a matmul (vLLM-Omni ships adaln_cache.py and caches by default). Quantizing them would cost accuracy for zero runtime benefit, so they are pinned to BF16 via extra_config: {".*adaln_proj\\.linear.*": {bits: 16, data_type: float, act_bits: 16}}.

Quant format (per transformer/config.json): quant_method=auto-round, data_type=mx_fp, bits=8, group_size=32, sym=True; activations act_data_type=mx_fp, act_bits=8, act_dynamic=True, act_group_size=32; block_name_to_quantize="token_refiner.blocks,blocks"; autoround_version=0.17.0.

Reproduce the quantization

Unlike Wan2.2, H3 is a Modular Diffusers pipeline, so the one-line CLI does not work — you must build the pipeline programmatically and hand it to auto-round. Outline (scripts/quant_h3_mxfp8_v2.py):

pipe = MiniMaxH3ModularPipeline.from_pretrained(LOCAL)            # offline
for name in ("transformer", "transformer_ref"):                   # both DiTs
    t = MiniMaxH3Transformer3DModel.from_pretrained(f"{LOCAL}/{name}", torch_dtype=torch.bfloat16)
    pipe.update_components(**{name: t})
ar = AutoRound(pipe, scheme="MXFP8", iters=0, ignore_layers="adaln_proj.linear")
ar.quantize_and_save("MiniMax-H3-mxfp8", format="auto_round")     # RTN, both DiTs

• iters=0 ⇒ zero-shot RTN; ignore_layers="adaln_proj.linear" pins all 52 AdaLN layers to BF16.
• Needs the dual-transformer zero-shot fix (auto-round PR #2481); transformer_ref is then auto-detected and quantized too.
• The diffusers-layout output (MiniMax-H3-mxfp8/) is converted to this native fused layout with scripts/convert_h3_mxfp8_to_native.py (head-grouped qkv fusion + fc1 gate/up + synchronized U8 scale reordering). See "Serving" for the three fixes that make it serve correctly.

---

Size / compression (safetensors header scan, per DiT)

dtype	#tensors	Size	Meaning
F8_E4M3	312	18.66 GiB	quantized weights (attn + ff)
U8 (E8M0)	312	0.58 GiB	per-32 block scales
BF16	314	24.34 GiB	AdaLN (52) + norms + embedders + biases
F32	12	0.06 GiB	proj_in / audio_proj_in / time embedder
Total	950	43.65 GiB	vs ~62 GiB bf16 ⇒ 0.71× (−29 %)

Because the source is already BF16 (not FP32 like Wan2.2) and AdaLN (~55 % of bytes) stays BF16, the per-DiT ratio is a modest ~0.71× rather than ~4×.

---

Directory layout — how this differs from the base MiniMax-H3

This repo is a serve-ready, self-contained slice of the base model: 233 GB vs ~465 GB. The reduction is not mostly from quantization — it is from dropping redundant copies that the base ships for the modular/diffusers distribution but vLLM-Omni never loads.

The base MiniMax-H3 is double-packaged: it stores both (a) a top-level modular component library (transformer/, transformer_ref/, text_encoder/, vae/, audio_vae/, tokenizer/, processor/, scheduler/, …) and (b) two already-self-contained pipelines (FL2VA/, Ref2VA/), each bundling its own transformer + text_encoder + VAEs + tokenizer. The two sets are the same weights duplicated — e.g. the 63 GiB text encoder exists 3× and the DiT ~4×.

vLLM-Omni serves only from the two pipeline folders (combined mode loads FL2VA + Ref2VA), so this repo keeps just that one copy and removes the rest.

Top-level entry	base MiniMax-H3 (~465 G)	this repo (233 G)	why
FL2VA/, Ref2VA/	135 G each	117 G each	kept; transformer swapped to MXFP8 (62→44 G)
transformer/	62 G (bf16, 14 shards)	config.json only (~1 KB)	shim: carries quantization_config so the quant is auto-detected; weights live under the pipelines
transformer_ref/, text_encoder/, vae/, audio_vae/, tokenizer/, processor/, scheduler/, audio_scheduler/	present	removed	redundant duplicates of what's already inside FL2VA/ + Ref2VA/
docs/, scripts/, modular_model_index.json	present	removed	not used for inference
assets/, README.md, model_index.json	official	our showcase + this card	model card + bf16-vs-MXFP8 samples

Inside each pipeline, only transformer/ changes vs the base: 13 bf16 shards → 11 MXFP8 shards, tensors are fused into the vLLM-Omni native naming (blocks.N.attn.qkv_proj.weight, fc1 gate/up) each with a paired .weight_scale, plus a quantization_config.json. The non-quantized components (text_encoder/, video_vae/, audio_vae/, tokenizer/, processor/) are byte-identical copies of the base — materialized as real files (no symlinks) so the directory is upload-ready.

One intrinsic redundancy remains: FL2VA/ and Ref2VA/ each carry their own 63 GiB text encoder (identical content). De-duplicating would need a symlink/shared path, which conflicts with being a self-contained, uploadable checkpoint, so both copies are kept on purpose.

---

Quality — VBench imaging_quality (dense93, n = 93)

Both models generated the same 93 prompts on the same code path (the non-modular combined runner, differing only in transformer weights) with the same recipe (512×288, 16:9, 24 fps, 16 steps, flow-shift 12, duration 4.0 s, audio-flow-shift 3.0, task t2va, seed=1000+row), then scored with the same VBench imaging_quality.

Model	imaging_quality	/100	vs bf16
bf16 (native)	0.6306	63.06	—
MXFP8 (this model)	0.6313	63.13	+0.12 %

Mean difference is +0.0007 (mxfp8 marginally higher). Per-video deltas are near-symmetric (mxfp8 higher on 49/93, lower on 44/93; mean |Δ| ≈ 3.41 on the 0–100 scale) while per-video scores span 26.8–80.0 — the jitter is diffusion sampling non-determinism (amplified by the low-res, short, few-step recipe), not a quantization floor.

---

Serving (vLLM-Omni, 2×H200 / TP2)

CUDA_VISIBLE_DEVICES=6,7 \
VLLM_WORKER_MULTIPROC_METHOD=spawn VLLM_OMNI_VIDEO_SYNC_TIMEOUT=14400 \
vllm serve /path/to/MiniMax-H3-mxfp8-native --omni \
  --host 0.0.0.0 --port 8092 --trust-remote-code \
  --num-gpus 2 --tensor-parallel-size 2 --text-encoder-tp-size 2 \
  --enforce-eager --diffusion-attention-backend CUDNN_ATTN
# no --task-type ⇒ combined (loads FL2VA + Ref2VA; required for T2VA)

• TP2 is mandatory: create_weights allocates the BF16 skeleton first (two DiTs ≈ 176 GB), so a single 141 GB H200 OOMs; --enable-cpu-offload is incompatible (Marlin repack is CUDA-only).
• Ready in ~5–6 min; GET /health → 200; ~81 GB/GPU; 0 weight_scale skips (quantization active).
• duration must be in [4, 15] s. Submit to /v1/videos/sync.

Three fixes required for correct MXFP8 serving (carried as local, un-upstreamed patches):

1. a top-level transformer/config.json carrying quantization_config so vLLM-Omni auto-detects the quant (native weights live under FL2VA/Ref2VA);
2. load_weights must apply the same grouped-qkv / fc1 gate-up reordering to .weight_scale as to .weight (otherwise scales misalign and output is noise);
3. TP2 + no cpu-offload (above).

---

Hardware note (important)

H200 (Hopper) has no native MXFP8 tensor core; vLLM-Omni runs these weights through a Marlin weight-only dequant GEMM — functionally correct and memory-saving, but 2.5× slower than bf16 (13 s vs ~5 s per 4 s/512×288 clip). Use on H200 to validate accuracy; native MXFP8 speedups require Blackwell/B300. Dequant does not change stored values, so accuracy matches a native kernel.

Runtime is W8A16, not W8A8. The Marlin kernel is documented as MXFP8 W8A16 GEMM via Marlin: it dequantizes the MXFP8 weight (E4M3 + E8M0 block scale) inside the GEMM and multiplies it against unquantized bf16 activations (a_scales=None; a 1-byte/fp8 activation path explicitly raises "Marlin W8A8 is not supported."). The checkpoint's act_data_type=mx_fp / act_bits=8 / act_dynamic=True is quantization-recipe metadata that is not exercised on H200 — activations stay bf16. True W8A8 (FP8 activations on a native microscaling tensor core) requires Blackwell/B300.

---

Limitations

• Not faster than bf16 on Hopper; needs 2 GPUs to serve.
• Only the two DiTs are quantized (text encoder, VAEs, AdaLN kept BF16).
• Evaluated with no-reference imaging_quality on short 512×288 clips; other dimensions not swept.
• Native/diffusers dual packaging means serving needs the conversion + three fixes above.

License & attribution

Inherits the base model's license: MiniMax-H3 Community License Agreement (license: other, see LICENSE). This is a derivative (quantized) checkpoint of MiniMaxAI/MiniMax-H3; all model credit belongs to MiniMax. Quantization by Intel auto-round (zero-shot RTN, MXFP8, AdaLN kept BF16) with PR #2481. Please also comply with the upstream MiniMax-H3 usage terms.