QrusherZA/MiniMaxH3_Omni

FastH3 Omni Ref2VA — Extracted r64 LoRA + Exact Support

A community extraction of the FastH3 Omni Ref2VA 8-step student into a rank-64 SVD adapter and an exact support checkpoint.

This repository distributes two complementary Safetensors files derived from the official FastH3 Omni ComfyUI BF16 checkpoint. It is intended for developers who want to study, transfer, or reconstruct FastH3 Omni's weight changes relative to the original MiniMax H3 Ref2VA transformer without distributing another full transformer checkpoint.

Important: This is not a standalone diffusion model and not a drop-in ComfyUI LoRA. Both files are required for a faithful structural reconstruction. A PDD-aware runtime is required for inference after reconstruction. The rank-64 approximation is experimental and has not been demonstrated to reproduce the official model's output quality.

Files

File	Contents	Size
FastH3_Omni_r64_fp16.safetensors	211 low-rank matrix deltas (422 LoRA factors), plus 93 full-precision exact-difference tensors	0.582 GiB
FastH3_Omni_support_bf16.safetensors	157 unaltered target tensors: PDD output heads, VSA compression gates, and the complete AdaLN/time-curve subsystem	3.825 GiB

Combined payload: approximately 4.41 GiB, in addition to a compatible Ref2VA base model and the remaining components required for inference.

The support_bf16 filename denotes the source family, not an all-BF16 dtype guarantee: exact tensors retain their original upstream dtypes, including FP32 where present.

What is inside the adapter?

• 211 SVD LoRA candidates: 200 main-transformer attention/MLP matrices, 8 token-refiner matrices, and 3 projection matrices.
• 422 factors: one lora_up.weight and one lora_down.weight per candidate, stored as FP16. The maximum rank is 64; layers with smaller dimensions may use a lower effective rank.
• 93 exact deltas: non-LoRA-compatible changed tensors, stored as FP32 additive differences.

For each eligible weight matrix, extraction computes the target-minus-base update and approximates it as:

ΔW = W_Omni − W_Ref2VA
ΔW ≈ LoRA_up @ LoRA_down
W_reconstructed = W_Ref2VA + (LoRA_up @ LoRA_down)

Factor normalization: the saved factor product is already the additive delta. Do not apply an additional alpha / rank multiplier unless your loader explicitly compensates for this representation. Adapter keys use diffusion_model.<module>.lora_up.weight and diffusion_model.<module>.lora_down.weight. Exact differences use diffusion_model.<module>.diff (or .diff_b for biases). These extra difference tensors require a compatible application tool; conventional LoRA loaders do not automatically consume them.

What is inside the exact support checkpoint?

Group	Tensor count	Treatment
VSA attention compression gates, blocks.0–blocks.49	50	Copy exactly from Omni
Time curve adaln_t_table	1	Copy exactly from Omni
AdaLN projection weights and biases, 50 blocks + final layer	102	Copy exactly from Omni
Expanded video/audio output weights and biases	4	Copy exactly from Omni
Total	157	No SVD or precision conversion

The PDD student has 32 output heads, which explains the output-projection shape changes relative to ordinary Ref2VA:

Tensor	Original Ref2VA	FastH3 Omni
final_layer.video_out.weight	[96, 5376]	[3072, 5376]
final_layer.video_out.bias	[96]	[3072]
final_layer.audio_out.weight	[32, 5376]	[1024, 5376]
final_layer.audio_out.bias	[32]	[1024]

These four tensors cannot be expressed as a same-shape additive LoRA delta. They must be installed as replacements, and the runtime must understand PDD's multi-head decoding semantics.

Extraction summary

The comparison used a 532-tensor MiniMax H3 Ref2VA checkpoint (minimax_h3_ref2va_pruned_bf16.safetensors) and the official 582-tensor ComfyUI Omni checkpoint (fasth3_omniref_8step_pruned_bf16.safetensors).

Comparison result	Count
Shared tensor names	532
Matching tensor shapes	528
Changed, shape-compatible tensors	407
Identical, shape-compatible tensors	121
Shape mismatches (PDD output heads)	4
Target-only tensors (VSA gates)	50
Base-only tensors	0

The 407 changed, shape-compatible tensors are partitioned into 211 SVD LoRA matrices, 93 exact additive differences, and 103 exact AdaLN/time-curve replacements. The remaining 121 shape-compatible tensors are left unchanged during reconstruction.

Measured reconstruction fidelity

These measurements come from the 211 SVD-approximated matrices only, comparing the Ref2VA-to-Omni weight deltas with their extracted rank-64 reconstructions. They are weight-space measurements, not perceptual video/audio metrics.

Metric	Measured result
Overall Base → Omni difference relative to Omni weights	0.28278%
Overall r64 reconstruction error relative to Omni weights	0.21944%
r64 residual relative to original Base → Omni delta	77.59953%
Squared delta magnitude captured by r64	39.78%

How to interpret these numbers: The extracted factors leave most of the small, distributed training delta outside the rank-64 subspace, even though the reconstructed full matrices remain numerically close to the corresponding Omni weights. Small weight differences can still matter significantly in few-step diffusion, and errors can propagate through many transformer layers.

Matrix family	r64 residual relative to its delta
Backbone attention QKV	73.22%
Backbone attention output	77.88%
Backbone MLP FC1	77.70%
Backbone MLP FC2	85.06%

A limited four-matrix rank probe found that r128 and r256 improve reconstruction, but some middle-layer deltas remain high-rank even at r256. Rank 64 is provided as a compact experimental baseline, not as a claim of output parity with the official Omni checkpoint.

The exact support tensors are copied byte for byte, preserving their source values and dtypes. Exact additive differences are stored in FP32. This does not make the complete extraction lossless, because the 211 SVD approximations and FP16 factors are lossy.

Requirements and integration notes

To work with this extraction you need:

1. A compatible, matching MiniMax H3 Ref2VA base in ComfyUI-style tensor layout. A different model revision may change the meaning of the extracted deltas.
2. Both Safetensors files from this repository.
3. A custom application/reconstruction implementation that understands the LoRA factors and the 93 .diff / .diff_b tensors.
4. A runtime that supports FastH3 Omni's 32-head PDD output projections, VSA gates, and the complete AdaLN time-curve subsystem.
5. The appropriate text encoder, video VAE and audio VAE for reference-conditioned generation with synchronized audio.

Reconstruction order (developer reference)

The two files are meant to be combined with a base checkpoint, not loaded separately as two diffusion models:

1. Start with the matching MiniMax H3 Ref2VA weights.
2. Add the 211 reconstructed LoRA updates: up @ down.
3. Add the 93 exact FP32 differences.
4. Replace all 157 exact-support tensors with the target Omni values.
5. Save in the target runtime's expected naming/layout and verify shapes.
6. Run with an Omni/PDD-aware inference implementation.

For a shape-compatible matrix, the intended operation is additive. For support entries, the intended operation is replacement—not addition and not LoRA blending.

Ordinary Load LoRA nodes are insufficient. A successful file load is not proof that .diff/.diff_b tensors, AdaLN replacements, VSA gates, or expanded PDD heads were actually applied.

Upstream inference contract

These are the official FastH3 Omni settings, included as a reference for testing a properly reconstructed checkpoint—not a claim that the extracted adapter has been quality-validated:

Parameter	Official Omni contract
Model family	MiniMax H3 Ref2VA
Distillation	Parallel Decoding Distillation (PDD)
PDD heads	32
Transformer evaluations	8
PDD head intervals	[0, 4, 8, 12, 16, 20, 24, 28, 32]
Guidance	1.0
Video scheduler shift	12.0
Audio scheduler shift	3.0
Attention	VSA-H3; reference-aware sparse attention

See the upstream FastH3 distilled inference contract and FastH3 Omni ComfyUI model card for the scheduler, reference input conventions, and runtime requirements. This extraction does not package the scheduler or inference implementation.

Limitations

• No output-parity claim. No controlled video/audio quality benchmark has been established for this r64 extraction against the official Omni student.
• Lossy low-rank approximation. A global 77.60% residual relative to the distillation delta remains in the SVD-approximated matrices.
• Not standalone. Neither file includes the Ref2VA base transformer, text encoder, VAEs, or inference runtime.
• Not plug-and-play. The exact-diff tensors and 32-head PDD output projections require custom handling.
• Base checkpoint matters. The measured deltas are relative to the local Ref2VA source checkpoint used during extraction; another upstream revision may not reconstruct equivalently.
• Hardware and acceleration not benchmarked. This package should not be interpreted as evidence of a particular speedup or memory footprint during inference.

Original models and acknowledgements

This is an independent redistribution of extracted model components, not an official FastVideo release. The source student and its training are the work of the original teams; this repository does not claim to have trained or authored FastH3 Omni.

• MiniMax — MiniMax H3, the base model and reference-conditioned architecture.
• FastVideo / Hao AI Lab — FastH3 Omni 8-Step V1, the accelerated Ref2VA PDD student and sparse-attention work.
• FastVideo — FastH3 Omni ComfyUI, the BF16 checkpoint used for this extraction.
• Nuva Lab — FastH3 Omni collaboration, including the reference-data and production-workflow contributions acknowledged upstream.
• NVIDIA FastGen team — acknowledged by FastVideo for relevant distillation work.

Please credit and review the original releases when using or redistributing this extraction.

License

The extracted tensors are derived from MiniMax H3 and FastH3 Omni weights, whose use is governed by the MiniMax H3 Community License Agreement. This is not an MIT, Apache-2.0, or unrestricted commercial-use model license. Review the original license—including territorial and use restrictions—before downloading, using, modifying, deploying, or redistributing these weights. This model card does not grant any additional rights beyond the governing upstream license.