ModelsLab/Qwen-Image-2.1-W4A4-int4

Qwen-Image-2.1 W4A4 (int4)

Built with Qwen.

Non-commercial use only. An educational artifact under the Qwen Research License, which the base model carries. Commercial use needs a separate licence from the rights holder; publishing this here does not grant one.

4-bit weights, 4-bit activations, for the Qwen-Image-2.1 transformer. Calibrated with SVDQuant, error-compensated with GPTQ, packed for Nunchaku's fused kernel. 4.34 GB against about 10 GB for the same layers in bf16.

The low-rank branch is fused into the 4-bit GEMM, which is where the speed comes from — an unfused branch gives back 24–34% of it.

Hardware

sm_75 through sm_120 — Ada (4090, L40S) and Blackwell alike.

Measured

card	cap	seconds	vs bf16	resident GB	LPIPS vs bf16
RTX 4090 48GB	sm_89	11.576	1.84x	21.35	0.231
RTX PRO 6000 Max-Q	sm_120	37.616	0.35x	21.29	0.1836

40 steps, 1024×1024, seed 42, six prompts. LPIPS is AlexNet against the bf16 image of the same prompt on the same card.

GPTQ is worth 6% here: 0.1946 with round-to-nearest against 0.1836 with GPTQ, at the same speed, the same file size and the same resident memory — it only changes which grid point each weight lands on.

Read LPIPS as distance from the bf16 trajectory, not as damage. On a dense-text prompt every build renders the text correctly while scoring 0.24–0.35, and on one prompt the bf16 reference garbles a sign that this checkpoint renders correctly — and is scored worse for it.

Only the transformer is quantized. The Qwen3-VL text encoder stays bf16 and is about 16.3 GB of a 21.5 GB resident total, so it, not this file, decides how many reference images fit on a card.

Text accuracy against bf16

The nvfp4 build of this same recipe scored 15/16 against bf16's 14/16 on a reference-free seed sweep (8 seeds, two text prompts, text read against the prompt) — indistinguishable. This int4 build was not put through that sweep; its LPIPS is 0.1836 against the nvfp4 build's 0.1425, so treat it as the same recipe at slightly lower fidelity, on the far wider set of cards it runs on.

Requirements

Packed for a specific kernel. The layout is an MMA fragment swizzle, so a different Nunchaku build may read it incorrectly — and would produce a wrong image rather than an error. The loader checks and warns.

torch==2.12.1
nunchaku==1.3.0.dev20260306+cu13.0torch2.12

Loading

import torch
from diffusers import QwenImage21Pipeline
from quantize import nunchaku_io

transformer = nunchaku_io.load_checkpoint(precision="int4", device="cuda")
pipe = QwenImage21Pipeline.from_pretrained(
    "Qwen/Qwen-Image-2.1", torch_dtype=torch.bfloat16, transformer=transformer)

nunchaku_io and the build pipeline: https://github.com/ModelsLab/qwen-image-2-1-server

How it was built

Smoothing factors from real activation statistics (SmoothQuant, α=0.5), a rank-128 SVD branch kept in bf16, and the residual rounded onto Nunchaku's grid by GPTQ against a calibration Hessian. One Hessian pass, 128 distinct inputs across 224 layers, 24 GB resident, ~5 minutes.

w4a4_build.json records the whole build, including per-layer GPTQ statistics.

License

Derivative of Qwen/Qwen-Image-2.1 under the Qwen Research License Agreement — non-commercial only. See NOTICE.