Qwen-Image-2.1-NVFP4 (SVDQuant, DiT + text encoder, native in Diffusers)

Built with Qwen. Non-commercial research and evaluation use only (Qwen Research License, see LICENSE and NOTICE).

An NVFP4 quantization of Qwen/Qwen-Image-2.1, made from the official BF16 weights with SVDQuant + GPTQ and calibrated on both text-to-image prompts and image edits. Both large components run on native Blackwell FP4 tensor cores, and the pipeline loads with a single DiffusionPipeline.from_pretrained call.

At 512x512 on an RTX PRO 6000 Blackwell it is about 1.7x faster than BF16 and needs about 40% less VRAM, and it matches BF16 closely on text-to-image, typography and edits, including a hard recolor edit that earlier 4-bit text encoders failed.

Usage

import torch
from diffusers import DiffusionPipeline

pipe = DiffusionPipeline.from_pretrained(
    "joseplcam/Qwen-Image-2.1-NVFP4", dtype=torch.bfloat16, trust_remote_code=True
).to("cuda")

image = pipe("A capybara reading a book by candlelight", height=512, width=512, num_inference_steps=40).images[0]
edited = pipe("Make it night time with moonlight", image=image, output_resolution=512, num_inference_steps=40).images[0]

trust_remote_code=True loads two small files from this repository:

• text_encoder/modeling_nunchaku_qwen3vl.py: a Qwen3VLForConditionalGeneration subclass that swaps the quantized linears for Diffusers' own SVDQW4A4Linear before loading the weights.
• transformer/modeling_nunchaku_qwenimage21.py: the stock transformer class, unchanged.

Both files start with the same kernel setup, so it runs whichever component loads first. Diffusers loads the NVFP4 kernels from rootonchair/nunchaku-lite-kernels, which is no longer downloadable. The setup points that name at joseplcam/nunchaku-lite-kernels, an unmodified build of the same open-source kernels, and sets DIFFUSERS_TRUST_REMOTE_KERNELS=true unless you already set it. Set LOCAL_KERNELS yourself to use a different build.

Use a different seed for an edit than the one that generated its input image. Qwen-Image-2.1 returns an over-sharpened copy that ignores the prompt when the edit starts from the same noise (diffusers #14824); this is a base-model behaviour.

Requirements

• NVIDIA Blackwell GPU with compute capability 12.0 (RTX 50 series, RTX PRO 6000). The published kernel build targets sm_120a only.
• Linux x86_64, Python 3.12, PyTorch 2.13 with CUDA 13.0, which the kernel build targets.
• Diffusers from main with Qwen-Image-2.1 and Nunchaku Lite support (tested at commit 0377f0c), transformers>=5.12, accelerate, kernels>=0.14.

What is inside

Component	Precision	Details
transformer/ (7B DiT)	NVFP4 W4A4, group 16, FP8 block scales + BF16 rank-32 low-rank branch	All attention and MLP projections of blocks 2-29 (196 layers). Blocks 0, 1, 30, 31 and the global modulation stay BF16.
text_encoder/ (Qwen3-VL 8B)	NVFP4 W4A4 + BF16 rank-128 low-rank branch	MLP projections (gate/up/down) of decoder layers 4-31 (84 layers). All attention projections, layers 0-3 and 32-35, the vision tower, embeddings and lm_head stay BF16.
vae/	BF16	Official weights cast from FP32.
processor/, scheduler/	as released	Unmodified.

Total download: about 17 GB, against about 32 GB for the BF16 release.

How it was made

• Method: SVDQuant with GPTQ residual rounding, via diffuse-compressor (commit 0965874). A low-rank BF16 branch absorbs the outliers of each weight and SmoothQuant-style scaling migrates activation outliers; GPTQ then rounds the 4-bit residual using calibration statistics. Activation scales are dynamic, so nothing is fixed to the calibration inputs.
• Calibration data (128 samples per component): 64 prompts from the qdiff prompt set and 64 image edits from the train split of VyoJ/NHR-Edit-Change_Only. Three of every four samples at 512x512, the rest at 1024x1024. The DiT saw 20 denoising steps per sample, with the prefix KV cache disabled so the prompt and reference-image tokens pass through every step.
• Sensitive layers: the first and last two DiT blocks and the modulation stay BF16. For the text encoder, quantizing attention hurt edits most. With every encoder linear quantized, the "turn the parrots blue" recolor below worked in 1 of 8 seeds (rank 32) or 4-5 of 8 (rank 128); quantizing only the MLPs with rank 128 brought it to 7 of 8, the same as BF16.
• Runtime: Diffusers' built-in Nunchaku Lite quantizer for the DiT and the same SVDQW4A4Linear layers for the text encoder, running the nunchaku-lite CUDA kernels.

The scripts that produced this repository are in tools/: quantize_dit.py, quantize_text_encoder.py --rank 128 --edge-layers 4 --skip-attention, package.py, evaluate.py and parrot_test.py. They need a checkout of diffuse-compressor (its examples/ package) at $DIFFUSE_COMPRESSOR.

Results

RTX PRO 6000 Blackwell (96 GB), Diffusers, 40 steps, guidance 1, no torch.compile, after a warmup. Peak VRAM is PyTorch's peak allocation.

	BF16 (official)	This repo
Text-to-image, 512x512	3.32 s	1.91 s (1.74x)
Edit, 512x512	3.73 s	2.18 s (1.71x)
Text-to-image, 1024x1024	14.06 s	8.12 s (1.73x)
Peak VRAM, 512x512	31.9-32.5 GiB	18.6-19.3 GiB
Throughput, 512x512 batch 1-8	0.28-0.30 img/s	0.49-0.55 img/s

Batching several prompts gives no extra throughput in Diffusers on this GPU; it is saturated at batch 1.

Runtime options (text-to-image, 512x512, 40 steps, one fixed prompt, mean of 3 runs after warmup):

Setup	BF16 (official)	This repo
Eager, PyTorch SDPA	3.20 s	1.79 s
Eager, cuDNN SDPA	3.26 s	1.88 s
compile_repeated_blocks(), PyTorch SDPA	2.99 s	1.50 s

• Use eager (the default) for mixed workloads. torch.compile only wins while every call has the same shapes. Prompt length and the reference image's aspect ratio change the prefix KV cache, so varied text-to-image prompts and edits keep recompiling until TorchDynamo's recompile limit, after which new shapes run uncompiled anyway. In a 12-call mix (6 prompts of different lengths, 6 edits with different reference shapes) eager took 37.2 s in total, compile_repeated_blocks() 44.3 s and dynamic=True 56.5 s; steady-state edits were 2.0 s, 2.0 s and 1.7 s. Compile is only worth it for fixed-shape batch jobs.
• CUDA graphs (mode="reduce-overhead") do not work: the prefix KV cache keeps tensors that graph replays overwrite, and the Nunchaku kernels cannot be captured (cudaErrorStreamCaptureInvalidated). With the KV cache off, BF16 gained only 6% from them.
• SageAttention and PyTorch's flash SDPA reject the attention mask Qwen-Image-2.1 passes, so they cannot be used with this model in Diffusers. The pipeline shares one reference-image list across a batch, so edits with different images run one at a time.

Closeness to BF16 (LPIPS with AlexNet, same seeds, 512x512; lower is closer, around 0.1 is hard to tell apart):

Set	LPIPS vs BF16
8 text-to-image prompts (portraits, typography, counting, scenes)	0.148
12 held-out NHR-Edit test edits + 1 parrot recolor	0.046

Instruction following on a hard recolor ("Turn the parrots blue", 8 seeds, tiles with at least one blue parrot), with this repo's DiT:

Text encoder	Followed
BF16	7 / 8
NVFP4, every linear, rank 32	1 / 8
NVFP4, every linear, rank 128, 4 BF16 edge layers	4-5 / 8
NVFP4, MLPs only, rank 128, 4 BF16 edge layers (this repo)	7 / 8

With the BF16 text encoder, this repo's DiT followed the edit in 8 of 8 seeds, the same as the BF16 DiT.

Limitations

• Measured on one GPU model and one software stack; quantization shifts details of individual images.
• Not bit-reproducible: the same seed gives slightly different images from run to run (LPIPS 0.04-0.17 between repeats, against about 0.001 for BF16), most likely from non-deterministic accumulation in the W4A4 kernels. Part of the LPIPS against BF16 above is this variation.
• The evaluation is small (8 prompts, 13 edits, one 8-seed recolor test). It is not a benchmark.
• The kernel build covers sm_120a, PyTorch 2.13, CUDA 13 and CPython 3.12 only.
• Non-commercial research and evaluation use only.

Previous version

Until 2026-09-25 this repository held a different combination for SGLang Diffusion: the ModelOpt NVFP4 DiT from HangGlidersRule/Darkstar-Qwen-Image-2.1-Base-ModelOpt-W4A4-NVFP4 and the NVFP4 Qwen3-VL encoder from BennyDaBall/Qwen-Image-2.1-NVFP4. It is still available in this repository's commit history. It was replaced because its 4-bit text encoder lost hard edits and it did not load in Diffusers.

Credits and license

• Qwen team: Qwen-Image-2.1.
• SVDQuant (MIT Han Lab), nunchaku-lite and diffuse-compressor (rootonchair).
• Calibration edits: VyoJ/NHR-Edit-Change_Only.

Qwen is licensed under the Qwen RESEARCH LICENSE AGREEMENT, Copyright (c) 2026 Hangzhou Tongyi Laboratory Technology Co., Ltd. All Rights Reserved. See LICENSE and NOTICE, which lists every modified file.