intel-ai/Wan2.2-Animate-14B-Diffusers-MXFP8-AutoRound

Wan2.2-Animate-14B · MXFP8 (AutoRound RTN)

This repository contains an MXFP8 W8A8 quantized version of Wan-AI/Wan2.2-Animate-14B-Diffusers, produced with AutoRound and intended for Diffusers inference on NVIDIA Blackwell (sm_120).

Model Summary

Item	Value
Base model	Wan-AI/Wan2.2-Animate-14B-Diffusers
Task	Character animation / replacement (WanAnimatePipeline)
Quantization	AutoRound 0.17.0
Scheme	MXFP8, W8A8, group size 32
Calibration	None (pure RTN, --iters 0 --disable_opt_rtn)
Model size	29 GB (transformer 17 GB)
Inference	Diffusers 0.40 + AutoRound quantizer
Tested on	NVIDIA RTX PRO 5000 72GB (sm_120)

The text encoder, image encoder, VAE, tokenizer, and scheduler are unchanged.

Quantization Details

transformer/quantization_config.json:

{
  "bits": 8,
  "act_bits": 8,
  "data_type": "mx_fp",
  "act_data_type": "mx_fp",
  "group_size": 32,
  "act_group_size": 32,
  "sym": true,
  "act_sym": true,
  "act_dynamic": true,
  "block_name_to_quantize": "motion_encoder.res_blocks,motion_encoder.motion_network,blocks,face_adapter",
  "quant_method": "auto-round",
  "packing_format": "auto_round:llm_compressor"
}

Coverage:

• Quantized: 512 linear layers — 40 transformer blocks × 12 linears (attn1.to_q/k/v/out, attn2.to_q/k/v/out/add_k_proj/add_v_proj, ffn.net.0.proj, ffn.net.2) plus 8 face_adapter blocks × 4 linears. Weights are stored as FP8 E4M3 with per-32 E8M0 scales; activations are dynamically quantized to MXFP8 at runtime.
• Kept at higher precision: motion_encoder convolutions, time_embedder, scale_shift_table, norm1/2/3, rope, patch_embedding, condition_embedder, and the embedding/norm/conv layers of the face encoder.
• vae, text_encoder, image_encoder, tokenizer, scheduler are not quantized.

Quantization Command

export MODEL=Wan-AI/Wan2.2-Animate-14B-Diffusers
export OUTPUT_DIR=./Wan2.2-Animate-14B-Diffusers-MXFP8

CUDA_VISIBLE_DEVICES=0,1 auto_round \
  --model "$MODEL" \
  --scheme MXFP8 \
  --iters 0 --disable_opt_rtn --disable_model_free \
  --device_map 0,1 --low_gpu_mem_usage --batch_size 1 \
  --output_dir "$OUTPUT_DIR"

# auto-round 0.17.0 writes model-*.safetensors for this pipeline; Diffusers
# expects diffusion_pytorch_model*.safetensors.
python - <<'PY'
from auto_round.utils.model import rename_weights_files
rename_weights_files("./Wan2.2-Animate-14B-Diffusers-MXFP8/transformer")
PY

--disable_opt_rtn forces pure RTN (need_calib=False); without it, --iters 0 would default to OptRTN and require calibration data. No calibration is used here because WanAnimate needs image + pose_video + face_video, while the available opens2v set only provides image+prompt. Quantization used 2× RTX PRO 5000 72GB and takes about 1-2 minutes for this model. MODEL can be a local Diffusers directory instead of the HF ID.

Inference

Requirements:

pip install "diffusers==0.40.0" "auto-round>=0.17.0" transformers accelerate safetensors "imageio[ffmpeg]"

Minimal example (run from this repository root):

import torch
from diffusers import AutoencoderKLWan, WanAnimatePipeline
from diffusers.utils import export_to_video, load_image, load_video

MODEL_ID = "."
vae = AutoencoderKLWan.from_pretrained(MODEL_ID, subfolder="vae", torch_dtype=torch.float32)
pipe = WanAnimatePipeline.from_pretrained(
    MODEL_ID, torch_dtype=torch.bfloat16, vae=vae, device_map="balanced"
)
pipe.reset_device_map()
pipe.enable_model_cpu_offload(device="cuda")

image = load_image("examples/animate/src_ref.png")
pose_video = load_video("examples/animate/src_pose.mp4")
face_video = load_video("examples/animate/src_face.mp4")

frames = pipe(
    image=image,
    pose_video=pose_video,
    face_video=face_video,
    prompt="People in the video are doing actions.",
    height=720,
    width=1280,
    segment_frame_length=77,
    prev_segment_conditioning_frames=1,
    num_inference_steps=20,
    mode="animate",
    guidance_scale=1.0,
    generator=torch.Generator(device="cuda").manual_seed(42),
    output_type="np",
).frames[0]
export_to_video(frames, "animate.mp4", fps=30)

device_map="balanced" is required so the AutoRound quantizer selects the CUDA kernel. enable_model_cpu_offload(device="cuda") keeps peak VRAM around 68 GB at 720p; omit it if all components fit. Set PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True if you hit fragmentation OOMs.

Running the Official Example

The repository ships the preprocessed official example inputs under examples/animate/ and a ready-to-run script. The inputs were produced with the official wan/modules/animate/preprocess/preprocess_data.py (--resolution_area 1280 720).

python examples/run_example.py --backend native
python examples/run_example.py --backend sage_v1
python examples/run_example.py --backend sage_v2
python examples/run_example.py --backend sage_v3

--limit-frames, --steps, and --segment-frame-length are available for shorter smoke runs. For a true short smoke, set both --limit-frames 5 and --segment-frame-length 5; otherwise the 5-frame input is padded to a 77-frame segment.

Results

Official example: 1280×720, 106 frames, 20 steps, guidance_scale=1.0, seed 42, segment_frame_length=77. One RTX PRO 5000 72GB, CPU offload, wall time includes pipeline load and VAE encode/decode.

Attention backend	Wall time	MAE vs BF16	SSIM vs BF16
native (SDPA)	59.7 min	3.85	0.9254
SageAttention v1	57.3 min	4.01	0.9201
SageAttention v2	52.4 min	2.77	0.9445
SageAttention v3	51.9 min	18.04	0.7649

Validation: all four commands above were re-run end-to-end on an RTX PRO 5000 72GB (sm_120). Each output is bit-identical to the original campaign run (MAE 0, SSIM 1.0).

SageAttention v3 uses Blackwell microscaling FP4 attention. It is the fastest backend but has visibly lower fidelity, including a brightness/lighting shift in the later frames. Use v1/v2 or native when quality matters.

Frames 0 / 40 / 80 / 105, MAE is against BF16 native. The v3 lighting shift becomes obvious in the last two rows.

Sample Outputs

Backend	Video
native	examples/mxfp8_native.mp4
SageAttention v1	examples/mxfp8_sage_v1.mp4
SageAttention v2	examples/mxfp8_sage_v2.mp4
SageAttention v3	examples/mxfp8_sage_v3.mp4

Limitations

• Pure RTN, no calibration: WanAnimate needs image + pose_video + face_video, and the available I2V calibration set (opens2v) only provides image+prompt. An OptRTN version would need a pose/face calibration set.
• All numbers were measured on one RTX PRO 5000 72GB (sm_120) with Diffusers; other hardware/runtimes may differ.
• SageAttention v3 quality is lower; see the table above.
• For research and local evaluation only. Follow the licenses of the base model and the example assets.