enoky/StereoCrafter2-FP8

StereoCrafter2 — FP8 weight-only quantization

FP8 (e4m3) weight-only quantization of TencentARC/StereoCrafter2 (the Wan2.1-VACE-14B-based stereo video inpainting transformer), produced with torchao Float8WeightOnlyConfig.

• Size: 16.2 GiB (vs 32.3 GiB bf16) — linear weights stored as FP8 with per-row scales, norms/embeddings kept in bf16. Compute remains bf16 (weights dequantize on the fly), so quality impact is minimal.
• Why: lets the 14B transformer run fully GPU-resident on ~24 GB cards (RTX 4090 / 5090) with no CPU offload streaming, and loads with only ~17 GB of system RAM.
• Verified: save/load roundtrip is bit-identical to the freshly quantized model; output statistics match the bf16 original on identical inputs.

Requirements

• torch >= 2.10, diffusers 0.36.x, accelerate
• torchao >= 0.13, < 0.16 — the upper bound matters: torchao ≥ 0.16 removes module paths that diffusers 0.36 imports, which breaks diffusers model loading entirely while torchao is installed.

Loading

Note: the checkpoint is a pickled torch state dict containing torchao tensor subclasses (safetensors cannot represent them), so it must be loaded with weights_only=False. Only load checkpoints from sources you trust.

import torch
from accelerate import init_empty_weights
from diffusers import WanVACETransformer3DModel
from huggingface_hub import hf_hub_download, snapshot_download

repo = snapshot_download("enoky/StereoCrafter2-FP8")
cfg = WanVACETransformer3DModel.load_config(repo)
with init_empty_weights():
    transformer = WanVACETransformer3DModel.from_config(cfg)

sd = torch.load(f"{repo}/diffusion_pytorch_model_fp8.pt", map_location="cpu", weights_only=False)
transformer.load_state_dict(sd, assign=True)   # peak RAM ~= checkpoint size
transformer.eval().requires_grad_(False).to("cuda")

The meta-device init + assign=True load keeps peak system RAM at roughly the checkpoint size (~17 GB) — the bf16 model is never materialized.

Ready-made integration: the enoky/StereoCrafter GUI suite loads this checkpoint via the "FP8 resident" offload mode in its V2 inpainting GUI (place this folder at weights/StereoCrafter2-FP8). The checkpoint was produced by that repo's export_fp8_transformer.py.

License and usage restrictions

This is a derivative of TencentARC/StereoCrafter2 and is distributed under the same license (see LICENSE): academic, research and education purposes only — no commercial or production use. The VACE/Wan2.1 base components remain under Apache 2.0 as described in the upstream license.

All credit for the model goes to the StereoCrafter2 authors at ARC Lab, Tencent PCG. This repository only re-packages their released weights in a lower-precision format.