yitongl/minimax-h3-deltaquant-nvfp4

DeltaQuant on MiniMax-H3 — end-to-end quality comparison

DeltaQuant ("4-bit Video Diffusion Models with Spatiotemporal Delta Smoothing", CVPR 2026) implemented as an NVFP4 Triton kernel for sm_120 and run end to end on MiniMax-H3, a 33 B dense omni video model the paper did not target. Compared against BF16 and against nunchaku's SVDQuant NVFP4 W4A4 — the paper's own headline competitor.

All 18 clips (3 scenes x 3 arms x {6, 50} steps) on one RTX 5090, 768x1344, 124 frames, seed 1101, on an otherwise idle GPU. metrics.json records the GPU's other tenants for every run; they are empty.

Headline: on H3 the two 4-bit methods tie on video, and DeltaQuant wins on audio

scene	steps	video latent PSNR nunchaku / DeltaQuant	audio latent PSNR nunchaku / DeltaQuant
cats	6	25.82 / 25.74	21.24 / 24.75
cats	50	21.67 / 21.12	19.36 / 21.67
weld	6	21.17 / 21.37	25.64 / 25.84
weld	50	18.68 / 18.62	22.71 / 24.45
reentry	6	24.55 / 24.21	23.69 / 23.40
reentry	50	21.52 / 21.76	25.04 / 25.46

Video: nunchaku takes 4 of 6, every margin under 0.55 dB — a tie. Audio: DeltaQuant takes 5 of 6.

Watch the videos. sbs-<scene>-s<steps>.mp4 puts BF16 | nunchaku | DeltaQuant side by side.

The metric that actually ranks them

End-to-end PSNR is confounded by the sampler: under a shared seed any perturbation moves the sample to a different basin. Teacher-forced fidelity removes that — BF16's per-step transformer inputs are dumped and replayed through each quantizer, so only the quantization error remains.

scene	video rel-L2 nunchaku / DeltaQuant	audio rel-L2 nunchaku / DeltaQuant
cats	0.1384 / 0.1160	0.1590 / 0.1385
weld	0.2340 / 0.2711	0.1569 / 0.1543
reentry	0.1921 / 0.2011	0.1349 / 0.1232

Video: nunchaku ahead by 4% on the mean, sign flips by scene. Audio: DeltaQuant 3/3, 7.7% better. The arms are not rank-matched (DeltaQuant 128, nunchaku 32), so a video tie is a mild disappointment rather than a neutral result.

Visual verdict

At 50 steps all three arms are indistinguishable blind — same subject, style and detail, different camera framing. Pixel PSNR of 13–15 dB is measuring framing, not damage: the weld 6-step grid holds three excellent photoreal welding shots that score 11 dB against each other. DeltaQuant's weld frames carry more legible fine structure than either other arm; its reentry frames are the one place it is visibly softer than BF16, where nunchaku is soft too. No banding, colour drift, flicker or mush anywhere in 18 clips.

Read the 6-step rows, and even those only weakly. The 50-step rows show overall character only.

Why DeltaQuant does not beat SVDQuant here

The method assumes tokens inside a spatiotemporal cube are similar enough that one core plus small deltas represents them. Measured across depth and timestep on H3, delta/|x| is 0.639 — the core absorbs only ~36% of the magnitude. A tie is what that predicts.

(An earlier single-layer probe reported 0.811; that point turned out to be the worst layer in the model. 0.639 is the figure measured across blocks 0/12/25/37/49 and 18 timesteps.)

This is a finding about transferring the method to a packed-sequence omni model, not a refutation of the paper. H3 differs from Wan2.2/LTX-Video in ways that plausibly matter: full self-attention over [text | refs | audio | video] with no cross-attention at all, and a video grid whose T is prime.

The audio win has a mechanism, and it is not in the paper

Non-video rows decompose better than video rows: delta/|x| 0.476 vs 0.639. H3 packs audio channel-major, so cover_remaining's 1-D cubes group consecutive latent frames of a single channel — a better neighbourhood than the paper's 3-D video cube gets on this grid. That is the mechanism behind the audio result, and it is a property of this port rather than of the method.

Speed

Whole denoise, 50 steps, all three arms re-measured solo on one RTX 5090, zero neighbours (1344x768, 124 frames, seed 1101, cats prompt):

arm	denoise (s)	hot step (s)	vs BF16
BF16 reference (pinned + prefetched offload)	888.6	18.14	1.00x
DeltaQuant (this repo, rank 128, paper cube schedule)	797.2	14.89	1.115x
nunchaku SVDQuant W4A4 (rank 32)	631.3	12.88	1.407x

Correction — an earlier version of this page reported DeltaQuant at 2.42x over BF16. Withdrawn. That figure was inflated by a defect in the reference, not by anything about the quantizer. Our BF16 path used diffusers' enable_group_offload(use_stream=False), which never pins host memory and does a synchronous device→host copy into freshly allocated pageable memory for every block on every step — 38.5 GB per step at a measured 2.27 GB/s, against 24.2 GB/s for a straight copy on the same card. Pinning the host copies and prefetching one block ahead (which is what SGLang's --dit-offload-prefetch-size 1 does) takes the same BF16 run from 1901.9 s to 888.6 s with bit-identical latents, and the honest ratio is 1.115x. The kernel-level results below are unaffected: they never involved offload.

Per hot step DeltaQuant is 1.219x; the whole-denoise figure is lower because the Triton autotuner costs ~61 s once per process (first evaluation 45.2 s, plus 46.5 s when the cube size changes) and a 49-evaluation run cannot amortize it. Against a hypothetical resident BF16 the earlier profile predicted 1.15x by subtracting the streaming term — the measured 1.115x agrees within 3%.

In-situ category split (sums to the whole within 0.005%; its three quantization-independent categories match an independent nunchaku profile to 0.3%):

	ms	share
attention	10,201	69.5%
quantized linears	3,778	25.8%
glue	493	3.4%
SwiGLU	174	1.2%

Attention dominates and is unquantizable, so further kernel work has little left to buy: DeltaQuant's linears are 2.12x slower than nunchaku's, and that buys nunchaku only 1.19x end to end. The step cache — which skips whole blocks including attention — is the remaining lever.

Isolated-Linear speed at the paper configuration: 1.89–2.14x over eager bf16, 1.80–2.15x over torch.compiled bf16.

A short run pays a 60.7 s one-off Triton autotune (once per cube size). That fixed cost, not any per-step defect, is the entire gap between the isolated 2.1x and a 5-step run's 1.34x.

Standing configuration for future runs

• 480x864, embeddings regenerated; the video grid is now derived per request from position_ids (measured (37,15,27)), never hardcoded.
• First-Block Cache ported to H3 on this box. Threshold 0.06 → 32.7% skip, 1.48x/step, 1.36x whole denoise. 0.07 reaches 1.60x but doubles trajectory perturbation past the knee.
• Cache and quantization do not interfere — tested, not argued: cube sequence and n_cubes identical across all 49 steps, saturation identical, x0 check bit-exact. Geometry is driven by a transformer-level hook that fires once per step regardless of block skipping.
• A trap worth knowing: at the same threshold 0.05, BF16 skips 42.9% and DeltaQuant 24.5%. A shared threshold perturbs the reference more than the arm under test. Match skip rates, or leave the cache off on the reference.
• x0 stage capture: the model's clean-latent estimate is saved at ~5 points and VAE-decoded, to see where distortion enters. Verified bit-exact (rel_l2 = 0.0) using x0 = x_t + (1−t)·v — the + matters, H3's transformer predicts x0 − eps, the opposite of the usual convention, and a minus sign yields something plausible-looking and wrong. See x0-stages-*.png: layout is fixed by step 10, only detail arrives after.

The activation scale bound held

act_global was derived from a bound (|x − core| ≤ 2·max|x|) rather than measured on real deltas. The live runs validate it: 0 saturated NVFP4 groups out of 9.88 x 10^11, every timestep, three prompts, both step counts. Counted at runtime, not assumed.

Deviations from the paper, and one replication

• Cross-attention K/V at 6 bits: non-applicable. H3 has no cross-attention (transformer_minimax_h3.py:159); text rows use the same projections as video.
• The paper's timestep cube schedule replicates — 3% better than uniform cube-64 on both scenes tested. This contradicted an earlier reading of our own per-step errors.
• Cube shapes follow the paper by default. Divisor-aligned variants cost 1.1% accuracy for 12.3% fewer GEMM rows and are opt-in.
• Core branch runs in bf16 — tl.dot_scaled cannot carry an fp32-per-64 scale. Cost 4–9%.

Two things that were wrong on the way, worth recording

• H3 hands block linears a 3-D [1, 37736, 5376], not the 2-D packed sequence this repo and nunchaku's wrapper both assumed. The 3-D branch re-entered forward and double-counted itself, inflating the first profile 2x. Caught because the instrumentation logged 100 calls/step against 50 blocks.
• The torch.compile comparison is on an isolated nn.Linear, which gives inductor nothing to fuse. A whole-block comparison has not been made, and on another model in this project a hand-written Triton stack was 18.5% slower than deleting it and letting inductor fuse. Not a closed question.

REPORT.md (988 lines) has the full method, deviations, correctness and benchmark detail; its §11 indexes every evidence file. All 44 correctness checks pass.