Qwen-Image-2.1: Quantized Inference Measurements on a Single RTX 5090
Memory and latency for fp8 and NF4 loading in diffusers on Windows 11
Scope. This note records measured VRAM and latency for Qwen/Qwen-Image-2.1 at revision 790c926, loaded with two quantization schemes. It covers one GPU and one seed per showcase prompt, and it reports no bf16 baseline.
Components and loading
The pipeline has three components:
transformer(DiT): 7.1B parameterstext_encoder: Qwen3-VL 8Bvae: 0.34B parameters
The bf16 checkpoint totals 30.9 GB. Two load-time configurations were used, each applied to the DiT and the text encoder with the VAE left in its original precision:
- 8-bit: fp8 weights with fp8 activations through
torchao0.18.0, per-tensor scaling - 4-bit: NF4 through
bitsandbytes0.50.2
All weights remained on the GPU with no CPU offload. A guard held PyTorch allocations below free VRAM and flagged any spill into system RAM; every reported run completed without one.
Measurements
Sampling used seed 11 and 40 steps.
| Quantity | 8-bit | 4-bit |
|---|---|---|
| Weights resident | 16.6 GB | 10.6 GB |
| Peak, 1024x1024 | 21.3 GB | 15.2 GB |
| Peak, 2048x2048 | 22.6 GB | 15.4 GB |
| Latency, 1024x1024 | 19.3 s | 19.2 s |
| Latency, 1920x1088 | 42.5 s | 45.2 s |
| Latency, 2048x2048 | 115.7 s | 118.8 s |
Latency grows faster than pixel count: a 4x increase in pixels produced a 6x increase in time, consistent with attention scaling quadratically in image tokens. Two reference images at 8-bit reached a 25.6 GB peak; at 4-bit the same case reached 19.1 GB.
Cost relative to hosted APIs
The checkpoint is openly downloadable, so local inference carries no per-image fee. For comparison, hosted list prices per 1024x1024 image are $0.053 (GPT Image 2, medium), $0.067 (Nano Banana 2), $0.134 (Nano Banana Pro) and $0.211 (GPT Image 2, high).
1,000 images x $0.134 (Nano Banana Pro) = $134
1,000 images x $0.211 (GPT Image 2 high) = $211
1,000 images, local open weights = $0 in fees
The local figure assumes the GPU is already owned. Quality was not compared side by side against the hosted models.
Environment: torch 2.11.0+cu128, diffusers 0.41.0.dev0 (git main), transformers 5.17.0, driver 610.88. torch.compile and NVFP4 were not used because Triton is unavailable on Windows.
Full article: https://kgptalkie.com/tutorials/llm-benchmarking/qwen-image-2-1-rtx-5090-local-benchmark
Video: https://youtu.be/B131CUZIjYE