abhishekchohan/minimax-h3-int8

MiniMax-H3 — INT8 Weight-Only Quantized (torchao)

An INT8 weight-only quantized version of MiniMaxAI/MiniMax-H3 — the 33B video+audio omni diffusion model — repackaged so it runs on consumer GPUs (24–32 GB) instead of needing 80 GB+ (or multi-GPU) cards. Quantized with torchao (Int8WeightOnlyConfig, version=2) at load time and validated end-to-end (quantize → save → reload → generate) with the streamed-offload recipe shown below.

Why this repo: full-precision MiniMax-H3 is ~130 GB of BF16 weights (33B DiT + Qwen3-VL-32B encoder + video/audio VAEs). Here the three heavy components are stored in INT8, cutting their footprint ~2× so the model fits a single consumer card when combined with CPU offload.

Sample output (INT8, t2va, on a 32 GB consumer card)

A validated 124-frame (~5 s), 544×960 t2va clip with its generated stereo soundtrack — prompt: "a golden retriever running through tall grass at sunset, cinematic". This run peaked at 16.36 GB VRAM with the group-offload recipe below — well inside a 24 GB budget.

What was quantized (and what was protected)

Weight-only INT8 keeps the sensitive I/O boundary in BF16 to preserve quality:

Component	Class	Quantized	Protected (kept BF16)
transformer (t2va + fl2va)	MiniMaxH3Transformer3DModel	block attention/FFN linears	proj_in, audio_proj_in, context_embedder, time_embedder, time_proj, token_refiner, norm_out, proj_out, audio_proj_out
transformer_ref (ref2va)	MiniMaxH3Transformer3DModel	block attention/FFN linears	same as above
text_encoder	Qwen3VLForConditionalGeneration	attention/MLP linears	model.visual, model.language_model.embed_tokens, model.language_model.norm, lm_head
vae / audio_vae	AutoencoderKLMiniMaxH3(Audio)	—	full precision
tokenizer / processor / schedulers	—	—	full precision

All three workflows are supported: t2va and fl2va (shared transformer/) and ref2va (transformer_ref/).

Footprint

Metric	BF16 (base)	INT8 (this repo, measured)
Transformer (t2va/fl2va)	~66 GB	31.7 GB
Transformer_ref (ref2va)	~66 GB	31.7 GB
Text encoder (Qwen3-VL-32B)	~65 GB	33.1 GB
Total heavy weights	~197 GB	96.5 GB (~2×)
Full repo (incl. VAEs)	~215 GB	106.8 GB
GPU needed (with offload)	80 GB+ / multi-GPU	16–18 GB measured (fits 24 GB)

Validated on a consumer GPU (RTX 5090, 32 GB)

Loaded drop-in from this repo (ModularPipeline.from_pretrained(...) + load_components), then ran the streamed group-offload recipe below and generated real clips. Peak VRAM was measured with torch.cuda.max_memory_allocated():

Workflow	Call	Peak VRAM	Result
t2va	124 frames, 544×960, 20 steps	16.36 GB	✅ coherent video + stereo audio
ref2va	+ 1 image reference, 124 frames, 544×960, 20 steps	18.21 GB	✅ coherent video + stereo audio

Both are well under 24 GB, so the INT8 repo runs on an RTX 4090 / 5090 (and fl2va uses the same transformer/ partition as t2va). Peak is dominated by the VAEs + the few resident transformer blocks + activations; the bulk of the weights live in host RAM (the t2va path keeps ~66 GB resident, so plan for ≈75 GB of system RAM). Output frames/audio from these runs were spot-checked for coherence (prompt-aligned, no quantization artefacts).

Requirements

• diffusers from main (the MiniMax-H3 pipeline is main-only): pip install git+https://github.com/huggingface/diffusers.git
• pip install torchao transformers accelerate
• Enough host RAM to stream the weights: at INT8 the resident weights are ~65–99 GB depending on workflow (the t2va path loads one ~33 GB transformer + the ~33 GB text encoder), so plan for ≈75 GB+ of system RAM.

Usage (consumer GPU, 24–32 GB)

Loads drop-in from this repo (every component — including the quantized transformer and text encoder — resolves from this repo), then streams the big components from host RAM with group offload:

import torch
from diffusers import ModularPipeline
from diffusers.hooks import apply_group_offloading

pipe = ModularPipeline.from_pretrained("abhishekchohan/minimax-h3-int8")
pipe.load_components(workflow="t2va", dtype=torch.bfloat16)

# version=2 int8 tensors are pinnable (streamed offload needs this); freezing
# removes the one autograd path the quantized tensors cannot serve.
pipe.transformer.requires_grad_(False)
pipe.text_encoder.requires_grad_(False)

offload = dict(onload_device=torch.device("cuda"), offload_device=torch.device("cpu"), use_stream=True)
pipe.transformer.enable_group_offload(offload_type="block_level", num_blocks_per_group=1, **offload)
apply_group_offloading(pipe.text_encoder.model, offload_type="leaf_level", **offload)
pipe.vae.to("cuda")
pipe.audio_vae.to("cuda")

out = pipe(
    prompt="a golden retriever running through tall grass at sunset, cinematic",
    num_frames=124,
    height=544,   # validated canvas (16.4 GB peak on a 32 GB card); 768x1344 also
    width=960,    # works on 32 GB. height/width must be multiples of 32.
    generator=torch.Generator().manual_seed(42),
    output=["videos", "audio", "sampling_rate"],
)
video, audio, sr = out["videos"], out["audio"], out["sampling_rate"]

• Lower VRAM (12–16 GB): also group-offload the video VAE (offload_type="leaf_level", no stream) and use a smaller canvas such as 960×544.
• ref2va: pipe.load_components(workflow="ref2va", ...) (loads transformer_ref), then pass references=[...].
• fl2va (keyframe): workflow="fl2va", pass image= (and optional last_image=).

⚠️ Loading notes

• The transformer/ and transformer_ref/ weights are stored as pickle .bin (torchao INT8 tensor subclasses can't be written to safetensors), so loading them uses torch.load deserialization. Your HF client may prompt you to trust pickle files; load with the standard from_pretrained above and diffusers re-materializes the INT8 weights automatically (requires torchao).
• The text_encoder/ is stored as safetensors (transformers ≥ its torchao-serialization support) and reloads quantized automatically.
• This is an experimental diffusers main modular pipeline; the API may shift.

Quality

INT8 weight-only is near-lossless for diffusion transformers. We validated the full quantize → save → reload → generate round-trip on a consumer GPU for both t2va and ref2va (coherent, prompt-aligned video + stereo audio; see the measured table above). A full VBench-style fidelity benchmark is out of scope for this release; treat quality as indicative and spot-check your own prompts.

License

This repo redistributes quantized weights derived from MiniMaxAI/MiniMax-H3 and is governed by the MiniMax H3 Community License Agreement (license:other). Open-weight use is region-limited to the US / EU / UK / South Korea; other regions apply via MiniMax's platform. See the base model's license terms — they apply in full to these quantized weights.

Limitations

• Weight-only INT8 reduces memory; speed depends on kernels and the offload bandwidth of your host (PCIe / RAM speed).
• Requires ~75 GB+ host RAM for the streamed-offload consumer path.
• Consumer VRAM figure is validated on a 24–32 GB card; if your workflow OOMs, drop resolution / num_frames or offload the video VAE as noted above.
• Base-model license and region restrictions (above) still apply.