feizhai123/MiniMax-H3-ModelOpt-Mixed9-Dynamic-FP8

MiniMax-H3 ModelOpt Mixed Dynamic FP8

This repository contains mixed BF16/FP8 derivatives of both MiniMax-H3 video generation partitions:

• FL2VA, at the repository root, supports T2VA/FL2VA and targets an effective precision of 9.0 bits.
• Ref2VA, under Ref2VA/, supports reference-conditioned video and audio generation and targets an effective transformer precision of 9.25 bits.

The checkpoints target single-GPU vLLM-Omni serving. CPU component offload is recommended on GPUs that cannot hold all resident components.

License: these derivatives remain subject to the MiniMax H3 Community License Agreement in LICENSE, including its territorial, redistribution, notice, and acceptable-use requirements. Review that license before using or redistributing this model.

Runtime format

Setting	Value
Weight format	FP8 E4M3 for selected Linear weights; BF16 otherwise
Weight scaling	Per-output-channel
Activation scaling	Dynamic per-token
ModelOpt algorithm	FP8_PER_CHANNEL_PER_TOKEN
Kernel	CUTLASS FP8 in vLLM-Omni

“Mixed9” and “Mixed9.25” are parameter-weighted effective precisions. They do not denote scalar 9-bit dtypes.

FL2VA Mixed9

The root checkpoint was selected using BF16-vs-dynamic-FP8 relative MSE.

Component	Achieved precision	FP8 groups	FP8 parameters	BF16 groups	BF16 parameters
H3 transformer	8.997527 bits	203	28,988,080,128	63	4,129,456,128
Qwen3-VL language encoder	8.994409 bits	166	21,349,007,360	34	3,030,384,640

FL2VA transformer scope

Kept in BF16:

• proj_in, audio_proj_in, context_embedder, both timestep embedding linears, proj_out, and audio_proj_out.
• Attention output projections in transformer blocks 29-48.
• FFN input/gate-up projections in blocks 28, 31, 32, 33, 41, 45, and 46.
• FFN output projections in blocks 5, 6, 8-10, 12-14, 19, 20, and 30-48.
• Biases, normalization parameters, RoPE buffers, and other non-linear state.

Quantized to dynamic FP8:

• All Q/K/V projections and AdaLN linears in transformer blocks 0-49.
• All Linear projections in both token-refiner blocks and norm_out.linear.
• Attention and FFN projections not listed in the BF16 sets above.

Shared Qwen3-VL text encoder scope

Kept in BF16:

• Token embeddings, RMSNorms, rotary state, all non-linear parameters, and the complete Qwen3-VL vision encoder.
• Attention o_proj in language layers 24, 25, 30, 31, 35, 37, 38, 40, and 42-49.
• MLP down_proj in language layers 17-19, 21-30, 34, 36, 46, 48, and 49.

Quantized to dynamic FP8:

• All Q/K/V and MLP gate/up projections in language layers 0-49.
• Every attention o_proj and MLP down_proj not listed above.

The video VAE, audio VAE, tokenizer, processor, embeddings, normalization layers, and vision encoder remain unquantized.

Exact selections and sensitivity scores are stored in:

• transformer/transformer_mixed_precision_config.json
• transformer/transformer_sensitivity_ranking.tsv
• text_encoder/text_encoder_mixed_precision_config.json
• text_encoder/text_encoder_sensitivity_ranking.tsv

Ref2VA GlobalGrad Mixed9.25

The Ref2VA transformer was independently selected from BF16 using an output-probed global-gradient sensitivity score. This is a custom search inspired by mixed-precision AutoQuant; it is not an NVIDIA ModelOpt AutoQuant export. ModelOpt performs the dynamic FP8 conversion and export.

Setting	Value
Target effective precision	9.25 bits
Achieved effective precision	9.248862 bits
FP8 groups / Linear modules	212 / 296
FP8 candidate parameters	27,947,630,592
BF16 groups / Linear modules	54 / 74
BF16 candidate parameters	5,169,905,664

Ref2VA transformer scope

Projection family	Dynamic FP8 blocks	BF16 blocks
Q/K/V	0, 2-40	1, 41-49
Attention output	0-4, 6-38	5, 39-49
FFN input/gate-up	1-38, 40	0, 39, 41-49
FFN output	0, 2-40	1, 41-49
AdaLN Linear	2-47	0-1, 48-49

Also quantized to dynamic FP8:

• All Q/K/V, attention-output, and FFN Linear projections in both token-refiner blocks.
• norm_out.linear.

Always retained in BF16:

• context_embedder, proj_in, audio_proj_in, both timestep embedding linears, proj_out, and audio_proj_out.
• Biases, normalization parameters, RoPE buffers, and all other non-linear state.

Ref2VA reuses the Mixed9 Qwen3-VL text encoder described above. Its VAEs, tokenizer, processor, embeddings, normalization layers, and vision encoder are not quantized. The exact transformer decisions are stored in:

• Ref2VA/transformer/transformer_mixed_precision_config.json
• Ref2VA/transformer/transformer_sensitivity_ranking.tsv

Similarity samples

These are deterministic short regression samples, not comprehensive perceptual-quality benchmarks.

Partition	Task	Resolution	Steps	Seed	Video SSIM vs BF16	Video PSNR	Audio spectral cosine
FL2VA Mixed9	T2VA	672 x 384	10	1101	0.858496	24.7968 dB	not measured
Ref2VA Mixed9.25	Ref2VA	672 x 384	10	3101	0.758005	22.234802 dB	0.996300151

For the Ref2VA sample, FP8 peak GPU memory was 84,842 MiB versus 131,308 MiB for BF16, saving 46,466 MiB (35.39%). The request used the same extracted reference frame/audio and saved BF16 baseline, produced 107 frames, and used a 4-second requested duration. Full machine-readable results are included in:

• evaluation/t2va_bf16_similarity.json
• evaluation/ref2va_globalgrad9p25_bf16_similarity.json

vLLM-Omni serving

This checkpoint requires vLLM-Omni with MiniMax-H3 ModelOpt mixed-FP8 loading support.

FL2VA/T2VA:

CUDA_VISIBLE_DEVICES=0 \
vllm-omni serve feizhai123/MiniMax-H3-ModelOpt-Mixed9-Dynamic-FP8 \
  --omni \
  --host 0.0.0.0 \
  --port 8000 \
  --trust-remote-code \
  --enforce-eager \
  --force-cutlass-fp8 \
  --enable-cpu-offload \
  --stage-init-timeout 1800 \
  --init-timeout 2400

Ref2VA:

hf download feizhai123/MiniMax-H3-ModelOpt-Mixed9-Dynamic-FP8 \
  --local-dir ./MiniMax-H3-ModelOpt-Mixed9-Dynamic-FP8

CUDA_VISIBLE_DEVICES=0 \
vllm-omni serve ./MiniMax-H3-ModelOpt-Mixed9-Dynamic-FP8/Ref2VA \
  --omni \
  --host 0.0.0.0 \
  --port 8000 \
  --trust-remote-code \
  --enforce-eager \
  --force-cutlass-fp8 \
  --enable-cpu-offload \
  --stage-init-timeout 1800 \
  --init-timeout 2400

Modification notice

Selected H3 transformer and Qwen3-VL language-model Linear weights were modified from the original MiniMax-H3 checkpoints by mixed BF16/FP8 quantization. The VAEs, Qwen3-VL vision encoder, tokenizer, processor, and other explicitly retained parameters remain in their original precision.