FP4DiT: Towards Effective Floating Point Quantization for Diffusion Transformers
Authors: Ruichen Chen
Keith G. Mills
Di Niu
Venue: Accepted by TMLR ()

Abstract: Diffusion Models (DM) have revolutionized the text-to-image visual generation process. However, the large computational cost and model footprint of DMs hinders practical deployment, especially on edge devices. Post-training quantization (PTQ) is a lightweight method to alleviate these burdens without the need for training or fine-tuning. While recent DM PTQ methods achieve W4A8 on integer-based PTQ, two key limitations remain: First, while most existing DM PTQ methods evaluate on classical DMs like Stable Diffusion XL, 1.5 or earlier, which use convolutional U-Nets, newer Diffusion Transformer (DiT) models like the PixArt series, Hunyuan and others adopt fundamentally different transformer backbones to achieve superior image synthesis. Second, integer (INT) quantization is prevailing in DM PTQ but does not align well with the network weight and activation distribution, while Floating-Point Quantization (FPQ) is still under-investigated, yet it holds the potential to better align the weight and activation distributions in low-bit settings for DiT. In this paper, we introduce FP4DiT, a PTQ method that leverages FPQ to achieve W4A6 quantization. Specifically, we extend and generalize the Adaptive Rounding PTQ technique to adequately calibrate weight quantization for FPQ and demonstrate that DiT activations depend on input patch data, necessitating robust online activation quantization techniques. Experimental results demonstrate that FP4DiT outperforms integer-based PTQ at W4A6 and W4A8 precision and generates convincing visual content on PixArt-$\alpha$, PixArt-$\Sigma$ and Hunyuan in terms of several T2I metrics such as HPSv2 and CLIP.

Published in Transactions on Machine Learning Research (11/2025)
FP4DiT: Towards Effective Floating Point Quantization for Diffusion Transformers
Ruichen Chen"
ruichen @ualberta.co
Deportment of Electrical and Computer Engineering, Faculty of Engineering University of Alberta
Keith G. Mills
keith.mills&ssn. edu
Division of Computer Science and Engineering, College of Engineering Louisiana State University
Di Niu
dniu@ualberto.co
Department of Electrical and Computer Engineering, Faculty of Engineering University of Alberta
Reviewed on OpenReview: /ttps://openppen opw.reieee.........om forum? id- ConH4mSQUP
Abstract
Diffusion Models (DM) have revolutionized the text-to- text-to-image visual generation process. However, the large computational cost and model footprint of DMs hinders practical do- ployment, especially on edge devices. Post-training quantization (PTQ) method to alleviate these burdens without the need for training or fine-tuning. While recent DM PTQ methods achieve W4A8 (i.c., 4-bit weights and 8-bit activations) on integer-based PTQ, two key limitations remain: First, while most existing DM PTQ methods evaluate on classical DMs like Stable Diffusion XI., 1.5 or earlier, which use convolutional U-Nets, newer Diffusion Transformer (DiT) models like the PixArt series, Hunyuan and others adopt fundamentally different transformer backbones to achieve superior image synthesis. Second, integer (INT) quantization prevailing in DM PTQ but does not. align well with the net- work weight, and activation distribution, while Troatin-PPiit Quantization (FPQ) is still under-investigated., yet it holds the potential to better aligu the weight and activation distri- butions in low-bit bit settings for DiT. In this paper, we introduce FP4DIT, PTQ method that leverages FPQ to achieve W4A6 quantization. Specifically, we extend and generalize the Adaptive Rounding PTQ technique to adequately calibrate weight quantization for FPQ and demonstrate that DiT activations depend on input patch data, necessitating robust. online activation quantization techniques. Experimental results demonstrate that FP4DIT achieves higher CLIP, ImageReward and HPSv2 performance compared integer- based PTQ the W4A6 and W4A8 precision levels while generating convicning visual content on PixArt-a, PixArt-E and Hunyuan. Code is available at
lightweight
1 Introduction
Dilfusion Transformers (DiT) (Peebles & Xie, 2023 are on the forefront. of open-source generative visual
synthesis. In contrast to carlier text-to-image (T2)) Diffusion Models (DMs) like Stable Diffusion v1.5 (Rom- bach et al. 2022) and Stable Diffusion XL (Podell et al. 2024h that utilize a classical U-Net structure, DiTs
such as PixArt-a (Chen et al.] (024b), PixArt- (Chen et al. 2024a) and Stable Diffusion 3 (SD3) Esser
et al. 2024 leverage streamlined, patch-based Transformer architectures to generate high-resolution images.
Nevertheless, similar to U-Nets, DiTs utilize lengthy denoising process that ineurs high computational inference cost. Approaches to reducing this cost generally aim to reduce the inference cost of the denoising
"Equal contribution.

---

Published in Transactions on Machine Learning Research (11/2025)
network Fang et al.
2023 Castells et al. 2024] reduce the number of denoising steps Yin et al 2024
Salimans & Ho 2022) One specific method to reduce the denoiser cost. is quant (Dettmers et al. 2022
Yao et al.
2022), which reduces the bit- feteprecision of neural network weights and activations. As the first Post- Training Quantization (PTQ) schemes for DMs, PTQ4DM (Shang et al.) 2023b) and Q-Diffusion (Li et al.
2023 demonstrate that the range and distribution of U-Net activations crucially depend on the diffussoo timestep. More recent state-of-the-art works like TFMQ-DM (Huang al. 2024 specialize quantization for C-Net timestep conditioning which may not generalize to newer DīTs. Purther, methods like ViDiT-Q (Zhao) et al. 2024) adapt outlier suppression technique (Xiao al.,
2023 to DiTs, but overlook broader advantages 2020 Li et al.
of prior DM PTQ like weight reconstruction NNgel et al.
2021
Moreover, the prevailing datatypes in existing DM PTQ literature (Shang et al. 2023b Li et al. 2023 Huang
et al. 2024 Zhao et al. 2024 So et al., 2024 Feng et al.) 2025 Mills et al.) 2025) are integer- based (INT).
which provide uniformly distributed values [Nahshan et al., 2021) unlike the non-uniform distribution of
weights and activations in modern neural networks (Shen et al. 2024). Thus, PTQ for text-to-image (T2I)
DiTs below W4A8 precision (4-bit, weights and 8-bit activations) without severely compromising generation quality remains open challenge.
In this paper, we present FP4DiT, which achieves W4A6 PTQ on Diffusion Transformers with non- con-uriform Floating-Point Quantization (FPQ) (uuzmin et al. (022). thus achieving high quantitative and qualitative T21 performance. Besides, by introducing FPQ, FP4D not only aligns the quantization levels better with the weight and activation distribution with negligible computational overhead, it also massively reduces the cost of weight calibration by over 8x. Our detailed contributions are summarized as follows:
1. We apply FPQ to DiT to address the misaligument between the existing DM PTQ literature and the non-uniform distribution of network weights and activations.
2. We reveal the critical role of preserving the sensitive interval of DiT's GELU activation function and propose a mixed-format FPQ method tailored for DIT.
3. We examine the adaptive rounding (AdaRound) Nagel el al. 2020 mechanism, originally designed for integer PTQ, and reveal a performance perfrmance-hampprinng hampering design limitation when applied FPQ. In this paper, we introduce a novel ma mathematical scaling mechanism that greatly improves the performance of AdaRound when utilized in the FPQ scenario.
4. We analyze DiT activation distributions and visualize how they contrast to those of convolutional U-Nets, especially with respect to diffusion timesteps. Specifically, while U-Net activation ranges shrink with timestep progression, DiT activations ranges instead shift over time. To address this,
we implement an effective online activation quantization (Wu et al. 2023b Yao et al. 2022 scheme to accommodate DiT activations.
We apply FP4DÍT as a PTQ method on T21 DřT models, namely PixArt-a, PixArt-E, and Hunyuan. To verify the effectiveness of FP4DIT, we conduct experiments on the MS-COCO dataset (Lin et al. 2014b) and the Human Preference Score v2 (HPSv2) T21 benchmark, to outperform existing methods like Q- Diffusion (Li ct al., (023). TFMQ-DM (Huang et al. 2024). ViDiT-Q (Zhao et al. (024) and Q-DiT (Chen) let al.) 2025a) at the W4A8 and W4A6 precision levels. Additionally, we perform humnan preference study which demonstrates the superiority of FP4DiT-genee generated images.
2 Related Work
Diffusion Transformers (DiT) Peebles & Xie 2023) replace the classical convolutional U-Net (Rombach)
ct al.
) backbone with modified Vision Transformer (ViT) (Dosovitskiy et al. 2020 to increase bility. Although the rntruccti of DiT architectures in newer DMs (Chen et al. 2024b a Esser el al.
sca la
2024
Li et al. 2024 Labs] Xie et al., 2024) enables the generation of high-quality visual content (Brooks 2024h, DiTs still suffer from a computationally expensive diffusion process, rendering deployment on
ct al.
odge devices impractical and cumbersome. Methods to reduce this cost might aim to compress the denoiser
neural network by pruning (Fang et al. 2023 Castells et al. 2024), sparsifying key operations (Chen et. al.
2

---

Published in Transactions on Machine Learning Research (11/2025)
2025by. by targeting reduction in the number of denoising steps needed to generate an image (in et al, 2024 Salimans & Ho. 2022). This work focuses on the former objective while also aiming to discover some of the challenges associated with achieving DiT' improvement compared to prior U-Nets.
Quantization is a neural network compression technique that involves reducing the bit- bit-precision of weights and activations to lower hardware metrics like model size, inference latency and memory consumption Nagel ett al. 2021p. The objective of quantization research is to reduce bit-precision as much as possible while preserving overall model performance (Ma et al.) 2 There are two main classes of quantization:
He et al. 2023 Feng et al 2025) and Post- Tost-TTraning Frantar & Alistarh 2022). Specifically, PTQ is more lightweight and neither requires re- re-training nor substantial amounts of data. Rather, PTQ requires a small amount of data to calibrate quantization scales (Nagel et al., 2020), typically in a block-wise manner (Li et al. 2.21h. p. However, while most PTQ methods rely on miiformly-diitributte integer (INT) quantization
Guantization-Aware Training (QAT) SSui et al.
2025
Quantization (PTQ) (Li et al. 2021 Mills et al.
2025
tochniques (Krishnamoorthi 2018 Jacob et al., 2018 Zhang et al.) 2025), recent literature highlights the ad- vantages of low-bit floating point quantization (FPQ) (Wu et al. 2023b Liu et al., 2023) for Large Language
Models (LLM). Therefore, in this paper we investigate the challenges in applying FPQ to DM PTQ.
The denoising process of DMs brings new challenges for PTQ compared with traditional computer vision neural networks. The earliest DM PTQ research (Shang et al. 2023b) reveals the significant activation range changes across different denoising timesteps. Q-Diffusion (Li et al. 2023 samples calibration data across different denoising timesteps to address this challenge. TDQ (So et al. 2024 calibrates an individual set of the quantization parameters across different time steps, offering a more fine- tne-grained approach to managing temporal dependencies. TFMQ-DM (Huang et al.) 2024) highlights the sensitivity of temporal features in U-Nets and introduce calibration objective aimed at better preserving temporal characteristies. How- ever, the above works are specific to U-Net architectures while DiT architectures feature distinct activation characteristics.
In contrast, LLM quantization (Dettmers et al. 2022 Frantar et al. 2022 Chee et al. 2023 Liu et al. 2024 Zhang et. al.
2024 operates on generalive AT transformers. The key challenge is that multi-billion parameter transformers tend to generate outlier hidden states that are difficult to effectively quantize while preserving end-to-end performance (Lee et al. 2024a Shang et al.) 2023a). This can be addressed by leveraging the learnable alfine shift of layernorm operations to adjust transformer attention and feedforward weights (Xiao) et al. 2023 Lin et al.] 2024b a). However, DiIs use Adaptive Layernorm (AdaLN) (Perez et al. 2018) which ties the affine shift to the timestep embedding, so these methods are less applicable. Additionally, LLM quantization typically features more lightweight calibration (Ashkboos et al. 2024 as parameter-heay models make advanced PTQ (Nagel et al., 2020 Li et al. 22021) costly. However, DiTs typically have fewer parameters and benefit from diverse calibration sets cover multiple timesteps. Thus, FP4DiT leverages prior work on PTQ calibration to quantize DiTs.
Finally, some early rescarch exists on DiT PTQ (Chen et al. 2025a). HQ-DiT (Liu & Zhang 2024 apply FPQ to class-conditional ImageNet (Deng et al. 2009) DiTs, but do not consider text-to-image (T2I) models like the PixArt (Chen et al. 2024b ab series. ViDIT-Q (Zhao et al. 2024 utilizes fine-grained techniques including channel balancing, mixed- mixed-precinee precision and LLM outlier suppression, it does not incorporate weight roconstruction (Nagel et al. 20200 Li ct al. 2021) from prior DM PTQ works. In contrast, this work aggregates knowledge from existing DM PTQ methods and refines them for application on T2I DiT models.
3 Methodology
In this section, we present our PTQ solution for the T21 DIT model. First, we analyze the sensitivity of the DiT block in the PixArt and Hunyuan model and propose a mixed FP format for the FP4 weight quantization. Second, we propose a scale-aware AdaRound tailored for FP weight quantization. Lastly, we investigate and contrast U-Net and DiT activation distribution information.
3

---

Published in Transactions on Machine Learning Research (11/2025)
3.1 Uniform vs. Non-Uniform Quantization
Quanlization compresses neural network size by reducing the bit-precision of weights and activations, e.g. rounding from 32/ 32/16-bit datatypes into an n-bit quantized datatype, where I 8 typically. For instance, we can porform uniform intoger (INT) quantization on a tensor X to round it into a lower-bit representation
( as follows:
( = cclip (1
X
+ 2, Fmin Tmax
(1)
where is scale, is the zero point, and 1-1 is operation nounding-to-ncares INT quantization rounds the values to range with 2" points. Specifically, the range is always uniform grid, whose size decreases by half each time decreases by 1.
In contrast, Floating-Poin Point Quantization (FPQ) uses standard floating- praaing--ointt point numbers as follows:
INT4
f (-11)]0220- 1
di + 2
d2
dm
(2)
E1M2
2m
E2M1
where dg E {0,1} the sign bit and b the bias. p + + de 2e-1 represents the e-bit exponent part di + d2 2
E3MD
while (1 4 $ + ....... ..... + dm represents the m-bit mantissa part. Note that de 10 for bits in both the mantissa and the exponent part. The FP format can be seen as multiple con- secutive m-bit uniform grids with different exponential scales. Therefore, the FPQ is operated similarly to Equation [ with distinct scaling factors applied to values across varying magni- tudes.
Figure 1: Value distribut for INT4 and three variants of FP4: E1M2, E2M1 and E3M0. Note that EOM3 is INT4. Observe how INT4 values are evenly distributed, while FP4 values cluster at the origin the number of exponent (E) bits increases.
The key advantage of FPQ, especially at low-bit precision for quantization, is that they enjoy richer granularity of value distributions owing to the numerous ways we can vary the allocation of exponent and mantissa bits. This is analogous to the int roduction of the 'BFloat (Kalamkar et al. 2019) format, which achieves superiority over the older IEEE stan- dard 754 'Float16' (Kahan 1996) in certain deep machine al- gorithms (Lee et al. 2024b by allocating 8-bits towards the exponent, as the larger 'Float32' format does. Broadly, an n- possible distributions
Output
Linear Layer
Scale
GELU
Pointwise Feedffgrward
Text Embedding
Scale, Shift
Linear Layer
Mult-Head Crops:Altention
adaL M-iingle
Scale
MultiFead Salf-Attntion.
Time
bit floating point datatype posses as m E [0, 1], and even adopts the uniform distribution of the corresponding n-bit integer format when m = 1.
Not-quantized
Scale, Shift
Quentizable Weight
Input
Features
Figure visualizes this advantage by showing the discrete value distribution of INT4 and FP4 under different. FP formats. The bits allocation between the mantissa and exponent significantly inffuences the performance of quantization as depicted. Whilc format benefits the quantiza- tion, improper FP format can result in sub-optimal perfor- mance (Shen et al.) 20224). Hence, in the following section, we present our analysis of the DiT blocks and introduce our method, which adjusts the FP format when quantizing differ- ent DiT weights.
Figure 2: T2I DiT block diagram. In PixArt-a/E, all DiT blocks share the same Asigle MLP for time conditions.
the flexibility of floating-poin
The scale and shift for layer normalization in DiT blocks depend on the embedding from AdaLN-single and the layer-specific training embedding. Colored blocks demar- cate quantizable weight layers from activa- tions and normalization
4

---

Published in Transactions on Machine Learning Research (11/2025)
3.1.1 Optimized FP Formats in DiT Blocks.
Figure illustrates the structure of T2I DiT Block. a DiT block, the Pointwise Feedforward is unique in that it consists of non-linear GELU activation flanked by linear layer before and after. GELU, plotted in Figure contains sensitive region
Activation Function (
3.5
3.0
Uniform Quantization
2.5
Non-Uniform Quantization
where the function returns negative output. Interestingly,
1.5
Reggiani et al., 2023 (Reggiani et al. 2023 show that focusing on this sensitive interval helps reduce the mean-squared error when approximating GELU using Look-Up Tables (LUTs) or breakpoints. Building on this insight, we apply denser floating point formats, c.g., E3M0, the first pointwise linear layer. This allocates more values closer zero, i.e, where the GELU sensitive interval lies, thereby enhancing the precision of the approximation. We further elaborate this point in the ap- pendix.
1.0
Sensitive Interval
0.0
-3
-2
0
2
3
Figure 3: The GELU activation and its sen- sitive interval. With the same amount of discrete values, non-uniform quantization can better capture the scnsitive interval.
3.2 AdaRound for FP
By default, quantization is seaiding-tt-neareee e.g., Eq. I AdaRound Nagel al. 2020) show that roundingtto-ncarest is not always optimal and instead apply second-order Taylor Expansion the loss degradation from weight perturbation Aw caused by quantization:
E [[A[(w)] = gwf + A Aw.
(3)
The gradient term g (()) is close 0 as neural networks are trained be converged. Hence, the loss degradation is determined by the Hessian matrix H(w), which defines the interactions between different perturbed weights in terms of their joint impact on the task loss. The rounding-to-nestt because it only considers the on-diagonal elements of H(w), However, optimizing via full Hessian matrix is infeasible because of its computational and memory complexity issues. To tackle these issues, the authors make assumptions such as each non-zero block in (() corresponds to one layer, and then propose an objective function:
is sub-optimal
arg Wx Wx| + Afreg(V); (V),
(4)
The optimization objective to minimize the Frobenius norm of the difference between the full fall-precision output Wx and the quantized output Wx for each layer and freg is a differentiable regularizer to en- courage the variable V converge. BRECQ (Li et al.] 2021]] proposed similar block-wise optimization objective that further advances the performance of weight reconstruction in PTQ. In detail, W is defined as follows:
(1 W
W = s clip
+ +h(V), min, mar
(5)
where min and mar denotes the quantization threshold. h(V) is the rectified sigmoid function proposed by Louizos el al. 2017):
h(V) clip (o(V) 7) 0,
(6)
where a(-) is the sigmoid function and and y are fixed to 1.1 and -0.1. During optimization the value of A(V) is contimuuss, while during inference its value will be set to 0 or 1 indicating rounding up or down.
3.2.1 Scale-aware AdaRound.
AdaRound has been widely adopted to improve the performance of quantized neural networks (Li et al. 2021 2023) in low-bit sottings like 4-bit weights. However, AdaRound assumes weight quantization low- bit integer formats, like INT4 rather than low-bit FP formats (van Baalen et al., 2023), where non-uniform value distribution (Fig. may introduce unique challenges.
5

---

Published in Transactions on Machine Learning Research (11/2025)
1.00
0.75
3
0.50
N
N 2
0.25
1
10
-5
0
10
-10
-5
5
10
-10
-5
0
5
10
V
V
(a) AdaRound for Integers
(b) AdaRound Applied to Floats
(c) Scale-Aware AdaRound
Figure 4: (a) The binary gate function of INT AdaRound. All gates are identical because there is only one scale in INT quantization. (b) The binary gate functions of origin AdaRound on FP quantization. (c) The binary gate functions of scale-aware AdaRound. The red dashed line indicates the demarcation of rounding up (right) down (left). Our scale-aware AdaRound normalizes the slope near the turning point, which stabilizes the optimization and helps improve the quantization performance.
02
Activation Values
Input Value
-02
5
123
5t0
66955 O. 222
-15
Timestop
200
-20
1
3
5
9
11
13
15
17
19
3
5
9
11
13
15
17
19
Timesteps
Timesteps
(b) Output Scale 7th DiT Block
(a) DiT Input Value Box Plot
(c) 7th DiT Block Output Box Plot
Figure 5: (a) Different timestep input values for PixArt-a on 128 images sampled from MS-COCO. The input does not shrink progressively across timesteps like U-Net DM. (b) The time-embedded scale for the output of the 7th DiT block's FoodForward. is almost constant across timesteps. (c) The output of the 7th DiT block. Its range tends to remain constant but shifts as a function of time.
Specifically, we identify that the original INT-based AdaRound assumes the scale & consistent across different quantized values. However, this does not hold for FPQ, where there are 2 scales. Therefore, we propose scale-aware AdaRound which improves the performance and leads to faster convergence.
Our scale-aware AdaRound inherits Equation as the learning objective because reducing the layer-wise and block-wise quantization error is the common goal of FPQ and INT quantization. Differently, We modified the W as:
(1 W
W clip
+ +VV('), min, max
(7)
a V ())
R' R'(V') clip
(8)
where R+(+) the scale-aware rectified sigmoid function and V is a new continuous variable we optimized over.
The rectified sigmoid function functions as binary gates that control the rounding of weights. Specifically, the gate function INT AdaRound. The gates are equivalent across all the weights, which matches the even distribution of INT quantization. In Figure 4b we show the origin AdaRound's binary gates under different scales. The gales' slope depends on their scale, which causes imbalanced update during the gradient descent. In Figure Ac we show the binary gates of our scale-aware AdaRound. In contrast, the gates' slope normalized to the same level, which makes the weight reconstruction much more stable and thus aids the quantization. For the mathematical proof of our scale-aware AdaRound, please refer to the appendix.
f(V) is optimized according to Equation 4 Figure 4a shows those binary gates in

---

Published in Transactions on Machine Learning Research (11/2025)
3.3
Token-wist wise Activation Quantization
Prior DM PTQ approaches (He al. 2024 Huang al, calibration dataset to learn temporally-aware (He et al., 2023 activation quantization scales. This approach is predicated on knowledge of how U-Net activation distribu- tions change as a function of denoising timesteps, i.e., acti- vation ranges taper-off towards the end of the denoising pro- ces8 (Shag et al., 2023b Li al., 2023
transformer blocks.
2024
20.0
17.5
Tokan-wisa Abc
15.0
12.5
10.0-
7.5
5.0
In contrast, we show that this assumption does not hold for DiT models. First, we collect imput activations of PixArt-a across 20 timesteps, revealing that the activation range remains stable over time, as shown in Figure 5a We then analyze the Adaptive Layernorm (AdaLN) (Perez et al. 2018) in the Pix DiT model in Figure 2 Since the feed-forward scale directly influences the DřT block output range, we visualize the feod- forward scale in the first layer in Figgueebbandd the output of the 7th DiT block in Figure [5c| These figures demonstrate that the value of activations is primarily controlled by channels opposed to timesteps, and that the width of the activation range tends
2.5
5
10
18
Figure 6: The distribution of the ab- solute maximum for each token's activa- tion among 4096 tokens in the PixArt-a model. The distribution demonstrates strong patch dependency in the DiT acti- vation.
to remain constant, but shifts as function of time.
Further, we plot the token-wise activation range in Figure This plot visualizes the absolute maximum activation of each image patch (token) across time. The results indicate that the activation range varies significantly, even among tokens within the same timestep.
Wu al. 2023b propose an online token-wise activation quantization that yields superior results when quantizing transformer activations. Table applies this tech- nique to the DiT scenario. Specifically, we apply simple min- min-max quantization to reduce weight precision of PixArt-a to 4-bits (W4), then consider 8-bit (A8) and 6-bit (A6) activation quantization. In both sce- narios, we observe CLIP performance that closer to the full precision model using coken-wise activation quantization as opposed the traditional, temporally-awane scale calibration technique which is designed for U-Nets. As such, we consider token-wise activation quantization throughout, the remainder of this work by substituting it into U-Net basclines like Q-Diffusion (Li ct al. 2023 and TFMQ-DM (Huang et al. 2024
Recent works by Microsoft (Yao et al. 2022
4 Results
In this section we conduct experiments to verify the ef- ficacy of FP4DIT. We elaborate on our experimental setup and then compare FP4DIT to several baselines ap- proaches to highlight its competitiveness in terms of quan- titative metrics and qualitative image generation output. Specifically, we consider three text-to-imant text-to-image (T2I) mod- cls: PixArt-a (Chen et al. 2024b), PixArt-E (Chen et al.) 2024a) and Hunyuan (Li et al. 2024). We also conduct several ablation studies to verify the components of our method. Finally, we report several hardware metrics tab- ulating the cost savings and throughput of FP4DiT.
Method
Precision
CLIP
No Quantization
11
0.3075
Temporally-Awar Act. Quant.
W4A6
0.2012
Token-wise Act. Quant.
W446
0.3036
Temporally-Awan Act. Calib. Token-wise Act. Quant.
W4A8
0.2410
W4A8
0.3120
Table 1: CLIP score (Hessel al. 2021 reported when quantizing PixArt-a to 4-bit. weights (W4) and 8 6-bit activations (A8 and A6) using the online token-wise method and temporarally- aware scale calibration. Specifically, we generate 1 images per configuration using COCO [Lin et al. 2014a) prompts and compare against the validation set. Higher CLIP better.
4.1 Experimental Settings
We use the HuggingFace Diffusers library (von Platen et al. 2022) to instantiate the base DiT models in W16A16 bit-precision and consider the default values for
7

---

Published in Transactions on Machine Learning Research (11/2025)
Benchmark
HPSv2
MS-COCO
Model
Method
Precision
Animationt
Concept-arti
Painting!
Photo
Average!
FID
CLIP
IRT
Full Precision
W16A16
32.56
31.06
30.76
29.67
31.01
31.05
0.3075
0.7037
Q-Dilfusion
W4A8
24.18
23.43
22.91
22.15
23.17
52.20
0.3017
0,1855
TFMQ DM
W4A8
27.88
26.03
25.14
24.91
25.99
64.73
0.3006
0.4706
VIDIT-Q
W1A8
17.93
17.35
16.81
17.59
17:42
39.11
0,2900
0.1716
Q-DIT
W4A8
28.49
26.91
27.55
26.50
27.98
36.17
0.3002
0.4979
PixArt-a
FP4DIT (ours)
28.63
26.
27.59
26.72
27.43
30.37
0.3076
0.5509
Q-Diiffusion
12.63
13.39
13.32
10.65
12.50
0.2868
-0.1825
TFMQ-DM DM
W4A6
24.02
23.36
23.
23.29
0.3015
0.1299
VIDIT-Q
W1A6
16.71
16.28
16.23
16.56
16.44
56.54
0,2827
0.1296
Q-DIT
WLA6
24.66
22.59
29.29
23. 04
29.40
49.91
0.9015
0. 0.1333
FP4DIT (ours)
W4A6
24.57
23.
23.30
23.26
23.55
40.61
0.3031
0.1353
Full Precision
W16A16
33.07
31.58
31.51
30.19
31.67
36.91
0.3139
0.9223
Q-Diffusion
W4A8
27.30
26.11
26.06
25.06
26.13
36.89
0.3050
0,2285
TFMQ DM
W4A8
24.95
23.59
23.19
22.78
23.62
62.98
0.2975
0.1047
ViDEI-Q
W4A8
27.10
26.45
26.24
24.49
26.07
37.17
0.2562
0.2832
Q-DIT
W4A8
74
26.23
26.05
24.67
26.17
39. 03
0.3048
0,2679
PixArt-E
FP4DIT (oues)
W4A8
24.67
26.27
0.3064
0.2927
Q-Diffussin
24.07
21.72
74.83
0.3027
0.0828
TFMQ-DM
W4A6
19.30
18.46
18.85
17.50
18.53
154.13
0.2346
-0.228
VIDIT-Q
W4A6
24.84
23.23
23.48
21.94
23.37
87.47
0.2425
-0.1156
Q-DFT
W4A6
25.67
23.57
80.60
212. 63
23.87
79.90
0.2894
0. 7614
FP4DIT (ours)
W4A6
26.91
25.55
25.22
23.93
25.40
42,21
0.3040
0.2085
Full Precision
W16A16
33.72
31.84
31.52
31.24
32.08
59.08
0.3102
0.8701
Q Dillflusion
W4A8
26.00
24.74
24.84
24.26
24.96
82.43
0.3006
0.6874
TFMQ-DM
W1A8
25.77
27. 63
27.07
26.15
27.50
89.39
0.3075
0.6873
VIDIT-Q
W4A8
25.74
26.79
:20
27.31
80.22
0.5099
0.7423
Q-DIT
W4A8
91
27:27
27.15
26.72
27.52
85.33
0.3088
7539
Hunyuan
FP4DIT (ours)
W4A8
28.
27.47
26.94
0.3102
0.7669
Q-Dilfusion
W4A6
14.90
14.83
15.32
14.24
14.83
255. 69
0.2277
-1.9366
TFMQ-DM
W4A6
13.73
13.31
13.16
14.51
13.68
258.02
0,2520
-1.9007
VIDIT-Q
W1A6
15.11
15.20
15.41
14.17
14.97
265.52
0.2559
-1.8123
Q-DFT
W4A6
13.41
13.15
13.48
12.94
13.24
262.33
0.2312
-1.1822
FP4DIT (ours)
W4A6
15.23
14.94
15.57
14.27
15.00
255.06
0.2562
-1. 7850
Table 2: Quantitative evaluation results for PixArt-a, PixArt-Y and Hunyuan in terms HPSv2, FID, CLIP and ImageReward (IR) score. Specifically, for each configuration, we generate 10k images (5k for Hunyuan) using COCO 2014 validation set prompis. Best and second best results in bold and ilalics, respectively.
inference parameters like number denoising steps and classifier-free guidance (CFG) scale. We quantize weights 4-bit precision FP format. Specifically, we set the weight format for the first linear layer each pointwise feed-forward be E3M0. We quantize all other weights to E2M1 for PixArt-a and Hunyuan, and E1M2 for PixArt-E. Further details on this decision are provided in Session 4.3.1 Finally, weight quantization is group- wise (Frantar et al. 2022 group size of 128.
Park et al. 2022 along the output channel dimension with
We perform weight quantization calibration using our scale-aware AdaRound and BRECQ. Weight calibra- tion requires a small amount of calibration data. We use 128 (64 for Hunyuan) prompts from the MS-COCO 2014 train (Lin et al.] (2014a) dataset and calibrate for 2.5k iterations per DiT block or layer. Next, we perform activation quantization
8 6-bit precision using min-max token-wise quantization from Zero- Wu et al. 2023b We provide further hyperparameter details in the appendix.
Quant Yao et al. 2022
4.2 Main Results
In our experiment, we consider four baseline approaches: Q-Diffusion et al. 2023), TFMQ-DM (Huang et al. 2024), ViDiT-Q (Zhao al. 20244) and Q-DiT (Chen et al. 2025a). Note that while Q-Diffusion and TFMQ-DM are originally designed for U-Nets, we modify these approaches use the same online token-wise activation quantization FP4DiT per Table 1 while ViDiT-Q and Q-DiT uses this mechanism by default.
8

---

Published in Transactions on Machine Learning Research (11/2025)
Further, note that ViDiT-Q uses mix-precision meaning some of their layers are not quantized to 4-bits. For the sake of convenience, we use their mix-precision as a W4 baseline to conduct our experiments.
We generate 512 x 512 resolution images using PixArt-a and 1024 x 1024 for PixArt-E and Hunyuan. For evaluation, we primarily consider the Human Preference Score v2 (HPSv2) (Wu et al.) 2023a) benchmark and Ms-COCO 2014 (Lin et al.] 20144ab validation set. Specifically, HPSv2 considers four image categories: animation, concept-art, painting and photography, and estimates the human preference score of an image generated using prompt with respect to one category. Each category contains 800 prompts, requiring 3.2k images to be generated to fully evaluate. The final HPSv2 score is the average estimated human preference across all four categories. Additionally, for COCO 2014, we measure the FID (Heusel et al., 2017j, CLIP (Hessel et al. 2021) score and average IeageReward (IR) (Xu et al., 2023 score using prompts from the validation set.
Table 2 compares our method with the statod bascline approaches at the W4A8 and W4A6 precision levels. FPADIT consistently oulperforms all other methods across three different base models at every precision level in terms of each performance metric. On HPSv2, FP4DiT achieves higher performance across every model and bit-precision combination. Although other approaches may achieve slightly higher performance individual category, these gaps are small, while FP4DiT leads in terms of FID, and CLIP on the COCO 2014 validation set. Finally, we observe a modest amount of IR degradation for PixArt-a and Hunyuan compared to the W16A16 model, however this is most pronounced for PixArt-E where all quantization methods suffer a substantial IR score loss.
an
We believe this degradation is two-fold. First, PixArt-E generates larger images, which means there is more pixel space to introduce noisc. Second, IR estimates how much human would prefer image given the prompl, opposed to CLIP which measures prompt adherence by comparing the latent embeddings of prompt and image, meaning the errors introduced by quantization are likely to have a greater impact. Nevertheless, FP4DiT achioves the highest IR performance in most of the model and precision settings.
Next, we compare performance across activation bit-precision. For the PixArt-series DiT models, A6 preci- sion can cause substantial performance degradation for some baselines, while FPADiT still maintains high performance. This is specifically noteworthy PixArt-E as baseline approaches suffer a loss in terms of either FID and/or CLIP performance at A6 compared to A8, but this is not the case for FP4DiT. For the Hunyuan DiT model, A6 quantization remains highly challenging across the board, however, our method still achives the best results at this level except the IR score. Overall though, Table 2 demonstrates the efficacy of FPADiT in term of quantitative T2I compared to prominant bascline approaches.
Next, Figure 7 provides qualitative image samples on the PixArt models. Note the higher quality of the images generated by FP4DiT at both the A8 and A6 levels. Specifically, on PixArt-a, the puppies have more realistic detail and the generated image more closely aligns with the W16A16 model. This is especially true for the PixArt-E sample images, where FP4DiT accurately captures the intricate details of the portrait of a middle-aged Asian woman, including the fractured porcelain and paint splatters. In contrast, the other quantization methods either introduce siguificant artifacts, lose fine details, or fail to accurately represent the prompt's content, further highlighting the robustness and superior performance of the FP4DiT approach.
Further, Figure D Providpr image results on Hunyuan, where we again note the detail present in the FP4DiT image, while the baseline approaches are much noisier and have yellow backgrounds which are not prompt- adherent. Finally, additional visualization results can be found in the appendix.
4.2.1 User Preference Study
To further verify the utility of our method, we conduct several human user preference studies to qualitatively compare FP4DIT to existing basclinc approaches. Specifically, for each human user preference study, we have a set of prompts (i.e,, introduced in (Chen et al. 2024bb) as well as quantized variants of a single model (e.g.. Hunyuan- quantized to W4A8 using ViDiT-Q. Q-DiT and FP4DIT). Each quantized model generates one image per prompt. For cach prompt, we solicit the opinion of a human participant, by presenting them the set of images produced using the prompt by different methods, and ask them to select which image they prefer, in terms of perceived prompt adherence and general visual quality. We then tally up the number of
9

---

Published in Transactions on Machine Learning Research (11/2025)
O-Diffuaion W4A6
Q-Diffusion W4AB
TFMQ-DM W4A6
TFMO-DM W4A8
VIDIT-Q W4A8
VIDIT-Q W4A8
Q-DIT W4A6
Q-DIT W4AB
FP4DIT W4A6
FP4DIT
W16A16
W4AB
Mitter of galcen rotrivvor pupples playing the snow. Thair haads pap out of the enow, coveredin in. middleeageed Asian wornen, her silve- streaked hair fractured, painted
Figure 7: PixArt-a (up) and PixArt-E (down) images and comparison between FP4DIT and related work. Best viewed in color and zoomed in.
FP W16A16
Q-Diffusion W4A8
TFMQ-DM W4A8
VIDIT-Q W4A8
Q-DIT W4A8
FP4DIT W4A8
baby painter trying to draw very simple picture, white background
Figure 8: Hunyuan image comparison. Note the details like 'white background' and 'detailed hair texture'
for FP4DIT. Best viewed in color.
Model
Method
Prec.
Preference(P))
Model
Method
Prec.
Preference (%)
ViDIT-Q
W4A6
17.09
Q-Diffusion
W4A8
6.58
PixArt-E
Q-DřT
30.77
TFMQ-DM
W4A8
36.84
EP4DIT
52.14
FP4DiT
W4A8
56.58
Hunyuan
ViDIT-Q
W4A8
19.17
Q- Q-Diffusion
W446
32.84
Hunynan
Q-DIT
W4A8
35.83
TFMQ-DM
W446
28.36
FP4DIT
W4A8
45.00
FP4DIT
W4A6
38.81
Table 3: DIT PTQ user preference study between FP4DiT, ViDiT-Q, and Q-DiT on PixArt-E W4A6 precision and Hunyuan-DiT W4A8 precision.
Table 4: Ada.oond/ user preference study between FP4DIT, Q-Diffusion and TFMQ-DM on Hunyuan DiT with W4A8 and W4A6 quantization.
votes each method obtains and compute how often it is selected compared to its competitors, e.g., preference score (%), where higher is better.
Our first test focused on comparing FP4DIT to other methods explicitly designed for DiT PTQ: VIDIT-Q and Q-DiT. This study consists of 120 prompts and 17 human participants. We compare generated image quality for PixArt-E at W4A6 precision and Hunyuan at W4A8 precision. Table 3 reports findings. FP4DiT is preferred over 50% of the time for PixArt-E W4A6 precision level, followed by Q-DiT and then ViDIT-Q. This result aligns with Table 2 where FP4DiT achieves the best result and Q-DiT outperforms ViDIT-Q in this setting. For Hunyuan at W4A8 the preference vote share for bascline mothods grow, but they do not overtake FPADTT, which demonstrates the veracity of our approach.
Next, we conduct an additional study comparing FP4DiT baseline approaches originally designed with U-Net DMs in mind but which utilize advanced PTQ via AdaRound and BRECQ: Q-Diffusion and TFMQ- DM. Specifically,
consider 75 random prompts and 15 human participants. The baseline DM is Hunyuan quantized to either W4A8 W4A6 precision. Table reports our findings. At A8 precision FP4DÍT clearly outperforms the other methods with a majority of the images being favored, while it also obtains almost 40% of the votes at A6 precision.
10

---

Published in Transactions on Machine Learning Research (11/2025)
4.3 Ablation Studies
We conduct ablation studies on the PixArt-a model to verify the contribution of each component of FP4DÍT. The experiment scttings are consistent with Section 4.2 unless specified. Additional ablation studies can be found in the appendix.
PixArt a
PixArt
Hunyuan
E2M1
4.3.1 Effect of FP Format.
Recall the mixed format strategy for FPQ in FP4DiT: we apply E3M0, which allocates more value closer to the GELU sensitive interval, the first pointwise linear layer and a unified FP format from E2M1 and E1M2 for the rest layers. To choose better one between E2M1 and E1M2 while avoiding data leakage, instead of quantizing FP4DíT with two formats and selecting the better one based on the FID and CLIP, we only employ the basic min-max quantization scheme (without AdaRound tech- nique thereby avoiding the use of calibration data) and leave the activation un-quantize (e.g. W4A16). We use the PixArt prompts to generate images and apply user preference studies to determine the format.
E1M2
Figure 9: E2M1 (up) and E1M2 (down) min-max FPQ visualization results on PixArt and Hun- yuan DIT. E2M1 is preferred by PixArt-o and Hunyuan, while E1M2 is preferred by PixArt-E.
Figure shows the visualization results of E2M1 and E1M2 FPQ. For the PixArt-a, F2M1 demonstrates better suitability, as the cactus in E1M2 loses its texture, result- ing in a mismatch between the image and the prompt. For PixArt-E and Hunyuan, inappropriate FP format causes noise the generated image, leading to suboptimal per- formance. In conclusion, it is straightforward and un- ambignous to determine the unified FP format based on these visualization results.
Method
Precision
HPSv2
Full Precision
W16A16
31.01
Q-Diffusion
W4A16
25.22
Q Q-Diffusion-FPQ
W4A16
8.78
Croup Quant
W4A16
25.05
+ Scale-Aware AdaRound
W4A16
26.44
I Scusitive-aware Quant.
W4A16
Table 5: The effect of different methods proposed for weight quantization on W4A16 PixArt-a.
4.3.2 Effect of Weight Quantization
To evaluate the effectiveness of our weight quantization method, we perform an ablation study on onig quantization, e.g. W4A16. As depicted in Table 5 our method progressively improves the weight quantization: Initially, directly applying FPQ to Q-Diffusion results significant degradation. Our research then reveals that group quantization is necessary for FPQ. Notably, using the original AdaRound top of FPQ impedes its ef- fectiveness. Subsequently, our sonsitivo-aware mixed for- mat FPQ (E3M0 in pointwise linear) further improves the post-quantizatioe performance. Eventually, our scale- aware AdaRound advances the boundaries of optimal per- formance by considering the multi-scale nature of FP weight reconot.
INT ANT-AdaRuund
Scale-aware AdaRound
0.310
0.305
CLIP
0.300
0.295
0.290
2.5k
5k
10k
20k
Steps
4.3.3 Effect of
AdaRound
The performance of different AdaRound methods with different calibration budgets on W4A16 PixArt-a quantization gener- ating 1K images. The stars indicate the optimal budget for INT and scale-aware AdaRound.
Figure 10:
To further verify our scale-aware AdaRound for FP quan- tization, we compare our scale-aware AdaRound the origin AdaRound with INT quantization in the W4A16
11

---

Published in Transactions on Machine Learning Research (11/2025)
Layer
W4A8-FP (ms)
W4AG-FP (ma)
W4A8-INT (ms)
W16A16 (ms)
Foodforward Layer
303.42 (1.54x)
313.54 (1.4xx)
302.56 (1,34x)
465.99
Feedforward Tayer 2
313.31 (1.50)
318.67 (1.47x)
312.80 (1.50x)
469.91
Sell-ttention QKV Proj.
78.33 (1.51x)
79.94 (1.48%)
78.16 (1.51x)
118.28
Table 6: The latency and the speedup of different linear layers in PixArt-a/2 under different quantization precision generating a 1024 x 1024 image, e.g., batch size of 2, 4096 tokens, each with an embedding dimension of 1152. Result measured on RTX 5080 GPU using CUDA 12.8.
GPU
INT8
FP8
FP6
FP4
RTX 4090 IVIIIIA 2024a
660.6 TOPS
660.6 TFLOPS
H100 INVIDIA 20246
1979 TOPS
1979 TFLOPS
RIX 5090
VIDIA 20244
838 TOPS
838 TFLOPS
838 TFLOPS
1676 TFLOPS
HGX B100
NVIDIA
56 POPS
PFLOPS
56 PFLOPS
PFLOPS
HGX B200
VVDII
20246
POPS
PELOPS
72 PFLOPS
PFLOPS
Table 7: GPU throughput rates for different low-bit datatype formats. Horizontal line demarcates older Ada SivelceeHopper GPUs from statc-of-the-art Blackwell scries. Older scries do not support FP6 and FP4.
setting. Note that the calibration budget is crucial for the performance of weight reconstruction. BRECQ (Li et al. 2021 uses 20k as default and this setting is inherited by prior DM quantization research like Q-Diffusion and TFMQ-DM. Thus, we configured calibra- tion budgets at {2.5k, 5k, 10k, 20k3 to ensure a fair comparison. Figure 10 outlines the CLIP of different AdaRound methods on PixArt-a under different calibration budgets. Scale-aware AdaRound achieving opti- mal budget with times fewer calibration steps than INT AdaRound, highlights its effectiveness in reducing calibration costs without compromising reconstruction quality.
4.4 Hardware Cost Comparison
Finally, we measure the hardware latency impact of FPQ compared more traditional INT quantization and the W16A16. Specifically, consider the quantization latency cost to execute some of the common, yet costlier weiggh//ctiivttin operations in DiT, namely the self-attention mechanism and feedforward module. Specifically, the linear layers corresponding to the self-attention mechanism (Q. K, V and oulput projection) all share the same layers which expand and then contract the token/patch embedding dimension, respectively. Additionally, weights and activations may not share the same bit precision, imposing additional dequantization cost.
dimensions, while the feedforward consists of two linear
We develop a CUDA 12.8 kernel and measure hardware latency on an Nvidia 5080, Blackwell GPU which supports low-bit FP formats. We consider the self-attention weight/activatinn dimensions found in PixArt-a/E when generating 1024 1024 image. Table 6 reports our findings. The result indicates that, despite the general expectation that floating-point computations are more demanding than integer operations, the latency results are nearly identical across all tested quantization precisions. Specifically, although A6 precision imposes some additional overhead compared to A8 (either for FP or INT), this is quite minor, leading to an overall speedup around 1.5x compared to W16A16, much like the popular W4A8
integer quantization (Li al. 2023: Zhao et al. 2024 Chen et al. 2025a).
Leveraging this finding, FPQ method can achieve superior performance without sacrificing computational efficiency. In fact, this latency results align with the stated computation rate of different INT and FP formats across different Nvidia GPUs, which we report in Table 7 Specifically, GPUs can handle INT and FP quantization with similar computational Uhroughput, which ensures that FP-based quantization does not introduce additional computational overhead. Lastly, although FP6 shares the same compute throughput as 8-bit formats (see Table 7 it brings memory saving (25% smaller tensor) which is essential for low-bit quantization, as transformers often become memory-bound (Xia al. 2024) during token generation. As a result, FP6 enhances DiT's inference efficiency.
12

---

Published in Transactions on Machine Learning Research (11/2025)
Model
Precision
Model Size (MB))
TBOPs!
W16A16
610.86
35.72
W8A8
305.33
8.938
PixArt-a
W4A8
152.87
4.474
W4A6
152.87
3.358
W4AS-G128 (ours)
158.59
4.474
W4A6-G128 (ours)
158.39
3.358
Table The comparison of model size and Bit-Ops of different Precision on PixArt-a. G128 denotes group- wise weight quantization with group size of 128. Lower is better.
We also compare the hardware cost of the quantized FP4DiT model with the full-precision model in terms of memory and energy consumption using two metrics: model size and Bit-Ops (BOPs) (He et. al. 2024 Specifically, model size refers to the disk space required to store the model checkpoint weights and scales, while BOPs is quantization-aware extension of the MACs (Chu et al. 2022 Mills et al. 2023 metric which measures the compute cost of neural network forward pass. As shown in Table group-wise weight quantizationintrouues, only moderate overhead; nonetheless, our method still substantially reduces the model size and BOPs.
5 Conclusion
In this paper, we propose FP4DiT, a PTQ method that achieves W4A6 and W4A8 quantization on T2I DiT using FPQ. We use a mixed FP formats strategy based on the special structure of DiT and propose scale-aware AdaRound to enhance the weight quantization for FPQ. We analyze the difference between the activation of U-Net DM and DiT and apply token-wise online activation quantization based on the find- ings. Our experiments demonstrate that FP4DiT achieves performance improvement in terms of quantative benchmarks such as CLIP and ImageReward scores on the MS-COCO dataset and HPSv2 benchmark, while also obtaining qualitative visualization improvement at minimal hardware cost.
References
Salch Ashkboos, Amirkeivan Mohtashami, Maximilian Croci, Bo Li, Pashmina Cameron, Martin Jaggi, Dan Alistarh, Torsten Hoefler, and James Hensman. Quarot: Outlier-free 4-bit inference in rotated llms, Aduances in Newral Information Processing Systems, 37:100213-100241 2024.
Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufci Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, Clarence Ng, Ricky Wang, and Aditya Ramesh.
Video generation models as world simulators. 2024. URL https://openai httpp://ppenaa..ommrreeaaccc com/research/
nitle-geeerattor--oollsss------ simulatorss
Thibault Castells, Hyoung-Kyu Song, Bo-Kyeong Kim, and Shinkook Choi. Ld-pruner: Efficient pruning of latent diffussin models using lask-agnostic insights. In Proceedings of the IEEE/CVE Conference Computer Vision and Pattern Recognition, pp. 821-830, 2024.
Jerry Chee, Yaohui Cai, Volodymyr Kuleshov, and Christopher De Sa. Quip: 2-bit quantization of large language models with guarantces. Advances in Neural Information Processing Systems, 36:4396-4429, 2023.
Junsong Chen, Chongjian Ge, Euze Xie, Yue Wu, Lewei Yao, Xiaozhe Ren, Zhongdao Wang, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart-E: Weak-to-strome training of diffusion transformer for 4k text-to- image generation, 2024a.
Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xic, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart-o: Fast training of diffusion transformer for photorealistic text- to-image synthesis. In The Twelfth International Conference Learning Represendations, ICLR 2024,
13

---

Published in Transactions on Machine Learning Research (11/2025)
Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024b. URL bttpss//ppaameeiii.nn///ooorrrr eAXmQPe3m1]
Lei Chen, Yuan Meng, Chen Tang, Xinzhu Ma, Jingyan Jiang, Xin Wang, Zhi Wang, and Wenwu Zhu. Q-dit: Accurate post-training quantization for diffusion transformers. In Proceedings of the IEEE/CVF Conference on Compuler Vision and Patlern Recognition (CVPR), June 2025a.
Ruichen Chen, Keith G. Mills, Liyao Jiang, Chao Gao, and Di Niu. Re-ttention: Ultra sparse visual generation via attention statistical reshape. In Advances in Neural Information Processing Systems, 2025b.
Xiaojie Chu, Liangyu Chen, and Wenqing Yu. Nafssr: Stereo image super-resolution using nafnet. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 1239 1248, June 2022.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248 255. leee, 2009.
Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. Gpt3. int8 (): 8-bit matrix multiplication for transformers at scale. Advances in Neural Information Processing Systems, 35:30318-30223 2022.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Un- terthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words:
Transformers for image recognition at scale. arXiv preprint arXiv:2010. 1 11929, 2020.
Patrick Esser, Sumith Kulal, Andreas Blatimann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthosis. In Forty-first International Conference Machine Learning, 2024.
Gongfan Fang, Xinyin Ma, and Xinchao Wang. Structural pruning for diffusion models. In Advances in Newral Information Processing Systems, 2023.
Weilun Feng, Haotong Qin, Chuanguang Yang, Zhulin An, Libo Huang, Boyu Diao, Fei Wang, Renshuai Tao, Yongjun Xu, and Michele Magno. Mpq-dm: Mixed precision quantization for extremely low bit diffusion models. In Proceedings of the AAAI Conference on Artificial Intelligence, 2025.
Elias Frantar and Dan Alistarh. Optimal brain compression: A framework for accurate post-training quan- tization and pruning. Advances in Neural Information Processing Systems, 35:4475-444 4488,, 2022.
Elias Frantar, Saleh Ashkboos, Torsten Hoefter, and Dan Alistarh. Gptq: Accurate post-training quantization for generative pre-trained transformers. arXiv preprint arXiv:2210. 17323, 2022.
Yefei He, Jing Liu, Weijia Wu, Hong Zhou, and Bohan Zhuang, Efficientdm: Efficient quantization-awwree fine-tuning of low-bit diffusion models. arXiv preprint XXXi:23100002200 2023.
Yefei He, Luping Liu, Jing Liu, Weijia Wu, Hong Zhou, and Bohan Zhuang. Piqd: Accurate post-training quantization for diffusion models. Advances in Neural Information Processing Systems, 36, 2024.
Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free
evaluation metric for image captioning. arXiv preprint arXiv:2104.08718, 2021.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural infor- mation processing systems, 30, 2017.
Yushi Huang, Ruihao Gong, Jing Liu, Tianlong Chen, and Xianglong Liu. Tfmq-dm: Temporal feature maintenance quantization for diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3362-7371, 2024.
14

---

Published in Transactions on Machine Learning Research (11/2025)
Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko. Quantization and training of neural networks for efficient integer-arithnetic- only inference. In Proceedings of the IEEE conference on computer vision and pottern recognition, pp. 2704-2713, 2018.
William Kahan. leee standard 754 for binary floating-pons point arithmetic. Lecture Notes on the Status of IEEE, 754(94720-1776):11, 1996.
Dhiraj Kalamkar, Dheevatsa Mudigere, Naveen Mellempudi, Dipankar Das, Kunal Banerjee, Sasikanth Avancha, Dharma Teja Vooturi, Nataraj Jammalamadaks Jianyu Huang, Hector Yuen, et al. A study of bfloat16 for deep learning training. arXiv preprint arXiv: 1905. 12322, 2019.
Raghuraman Krishnamoorthi. Quantizing deep convolutional networks for efficient inference: A whitepaper. arXiv preprint XrXi::186..0000 08342, 2018.
Andrey Kuzmin, Mart Van Baalen, Yuwei Ren, Markus Nagel, Jorn Peters, and Tijmen Blankevoort. Fp8 quantization: The power of the exponent. Advances in Neural Information Processing Systems, 35:14651- 14662, 2022.
Black Forest Labs. flux. URL httppp///gihuu...oo//bbacefffffs
Changhun Lee, Jungyu Jin, Taesu Kim, Hyungjun Kim, and Eunhyeok Park. Owq: Outlier-aware weight quantization for efficient fine-tuning and inference of large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 13355-13364, 2024a.
Joonhyung Lee, Jeongin Bae, Byeongwook Kim, Se Jung Kwon, and Dongsoo Lee. To fp8 and back again: Quantifying the effects of reducing precision on Ilm training stability. arXiv preprint arXiv:2405.1877 18710, 2024b.
Xinyu Li, Yijiang Liu, Long Lian, Huanrui Yang, Zhen Dong, Daniel Kang, Shanghang Zhang, and Kurt Keutzer. Q-diffusion: Quantizing diffusion models. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pp. 17535-17545, 2023.
Yuhang Li, Ruihao Gong, Xu Tan, Yang Yang, Peng Hu, Qi Zhang, Fengwei Yu, Wei Wang, and Shi Gu. Breeq: Pushing the limit of post-training quantization by block reconstruction. ar Xiv preprint arXiv:2102, 2XX.:22022.05266 2021.
Zhimin Li, Jianwei Zhang, Qin Lin, Jiangfeng Xiong, Yanxin Long, Xinchi Deng, Yingfang Zhang, Xingchao Liu, Minbin Huang, Zedong Xiao, Dayou Chen, Jiajun He, Jiahao Li, Wenyue Li, Chen Zhang, Rongwei Quan, Jianxiang Lu, Jiabin Huang, Xiaoyan Yuan, Xiacxiao Zheng, Yixuan Li, Jihong Zhang, Chao Zhang, Meng Chen, Jie Liu, Zheng Fang, Weiyan Wang, Jinbao Xue, Yangyu Tao, Jianchen Zhu, Kai Liu, Sihuan Lin, Yifu Sun, Yun Li, Dongdong Wang, Mingtao Chen, Zhichao Hu, Xiao Xiao, Yan Chen, Yuhong Liu, Wei Liu, Di Wang, Yong Yang, Jie Jiang, and Qinglin Lu. Hunyuan-dit: powerful multi-resolution diffusion transformer with fine- gre-grd chinese understanding, 2024.
Haokun Lin, Haobo Xu, Yichen Wu, Jingzhi Cui, Yingtao Zhang, Linzhan Mou, Linqi Song, Zhenan Sun, and Ying Wei. Duquant: Distributing outliers via dual transformation makes stronger quantized llms. Advances in Neural Information Processing Systems, 37:87766 87800, 2024a.
Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. Awq: Activation-awar weight quantization for Ilm compression and acceleration. Tn ML,Sys, 2024b.
Tsung-Y Yi Lin, Michael Maire, Serge Belongic, Lubomir Bourdev, Ross Girshick, James Hays, Pictro Perona, Deva Ramanan, C. Lawrence Zitnick, and Piotr Dollár. Microsoft coco: Common objects in context. In Computer Vision ECCV 2014, pp. 740 755, Cham, 2014a 4a. Springer International Publishing.
15

---

Published in Transactions on Machine Learning Research (11/2025)
Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision- ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pp. 740 755. Springer, 2014b.
Shil-yang Liu, Zechun Liu, Xijie Huang, Pingcheng Dong, and Kwang- Ting Cheng, Llm-fp4: 4-bit floating- point quantized transformers. arXie preprint orXiv:2310. 16836, 2023.
Wenxuan Liu and Saiqian Zhang. Hq-dit: Efficient diffusion transformer with fp4 hybrid quantization. arXiv preprint 2iv:22400 19751, 2024.
Zechun Liu, Changsheng Zhao, Igor Fedorov, Bilge Soran, Dhruv Choudhary, Raghuraman Krishnamoorthi, Vikas Chandra, Yuandong Tian, and Tijmen Blankevoort. Spinquant: Llm quantization with learned rotations. Xrviv preprint orXiv22455164000 2024.
Christos Louizos, Max Welling, and Diederik P Kingma. Learning sparse neural networks through 0 regularization. Xi preprint arXi:: (al::171001111 2017.
Shuming Ma, Hongyu Wang, Lingxiao Ma, Lei Wang, Wenhui Wang, Shaohan Huang, Li Dong, Ruiping Wang, Jilong Xue, and Furu Wei. The era of 1-bit llms: All large language models are in 1.58 bits. ar preprinl ar Xiv:2402. 1 1774,, 2024.
Keith G. Mills, Di Niu, Mohammad Salameh, Weichen Qiu, Fred X. Han, Puyuan Liu, Jialin Zhang, Wei Lu, and Shangling Jui. Aio-p: Expanding neural performance predictors beyond image classification. Proceedings of the AAAI Conference on Artificial Intelligence, 37(8):9180 9189, Jun. 2023. doi: 10.1609/ aeaivv37i..26100 CRL https://ojs. www..iiooaaaaa.aaoooaaaoaaaai.aa org/ index. pttp:/AAt//atiiccee/eeee//////
Keith G. Mills, Mohammad Salameh, Ruichen Chen, Wei Hassanpour, Negar Lu, and Di Niu. Qua2sedimo: Quantifiable quantization sensitivity of diffusion models. Proceedings of the AAAI Conference on Artificial Intelligence, pp. 6153-6166 0163,, 2025.
Markus Nagel, Rana Ali Amjad, Mart Van Baalen, Christos Louizos, and Tijmen Blankevoort. Up or down? adaptive rounding for post-training quantization. In International Conference on Machine Learning, pp. 7197-7206. PMLR, 2020.
Markus Nagel, Marios Fournarakis, Rana Ali Amjad, Yelysei Bondarenko, Mart Van Baalen, and Tijmen Blankevoort. A white paper neural network quantization. arXiiv preprint arXi::2106.0888 08295, 2021.
Yury Nahshan, Brian Chmiel, Chaim Baskin, Evgenii Zheltonozhskii, Ron Banner, Alex M Bronstein, and Avi Mendelson. Loss aware post-training quantization. Machine Learning, 110(11):3245-3322, 2021.
NVIDIA. Nvidia rtx blackwell gpu architecture, 2024a. URL https://images: www.aaagge..viiiii..... www.dii.cioaaaeeeeeeee wttidiia....ammmmmmmmaee commm/ammaam appp-rrcchttctureepaa. Accessed: 2025-3-07.
en Batttt/oorsseoooe//ccaeeee////ec htr.aaaccccelaaaaarrrrrrrtttttrr
NVIDIA. Nvidia blackwell architecture tochnical overview, 2024b. URL http::://resourccs..vidit. com/
Accessed: 2024-11-07.
NVIDIA. Nvidia h100 tensor core gpu architecture overview, 2024c. URL http::/resourcee../// nvidia,
com/ en tor-uss-eeoo-ccrrrr Accessod: 2024-11-07.
Gunho Park, Bacscong Park, Minsub Kim, Sungjac Lec, Jeonghoon Kim, Beomscok Kwon, Se Jung Kwon, Byeongwook Kim, Youngjoo Lee, and Dongsoo Lee. Lut-gemm: Quantized matrix multiplication based on luts for efficient inference large-scale generative language models. arXiv preprint orXiv:2206.09557 2022.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IRFE/CVF International Conference Computer Vision, pp. 4195 -19554425, 2023.
16

---

Published in Transactions on Machine Learning Research (11/2025)
Ethan Perez, Florian Strub, Harm De Vries, Vincent Dumoulin, and Aaron Courville. Film: Visual rea- soning with a general conditioning layer. In Proceedingg of the AAAT conference on artificial intelligence, volume 32, 2018.
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. SDXL: improving latent diffusion models for high-resolution image synthesis. In The Twelfth International Conference on Leurning Represenlations, ICLR 2024, Vienna, Austria, May 7-11, 2024. Re, 2024. URL httpp::/ net/forum22dddd122288.
Enrico Reggiani, Renzo Andri, and Lukas Cavigelli. Flex-sfu: Accelerating dnn activation functions by non-uniform piecewise approximation. In 2023 60th ACM/IEEE Design Automation Conference (DAC), pp. 1-6. IEEE, 2023.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognttion (CVPR), pp. 10684-10695, June 2022.
Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. arXiv preprint wxXiv2222.00512 2022.
Yuzhang Shang, Zhihang Yuan, Qiang Wu, and Zhen Dong. Pb-lim: Partially binarized large language models. arviv preprint or orXiv:2310.0000 0.00034, 2023a.
Yuzhang Shang, Zhihang Yuan, Bin Xie, Bingzhe Wu, and Yan Yan. Post-training quantization on diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1972-1981, 2023b.
Haihao Shen, Naveen Mellempudi, Xin He, Qun Gao, Chang Wang, and Mengni Wang. Effficient post-training quantization with fp8 formats. Proceedings of Machine Learning and Systems, 6:483 498, 2024.
Junhyuk So, Jungwon Lee, Dachyun Ahn, Hyungjun Kim, and Eunhyeok Park. Temporal dynamic quanti- zation for diffusion models. Advances in Neural Information Processing Systems, 36, 2024.
Yang Sui, Yanyu Li, Anil Kag, Yerlan Idelbayev, Junli Cao, Ju Hu, Dhritiman Sagar, Bo Yuan, Sergey Tulyakov, and Jian Ren. Bitsfusion: 1.99 bits weight quantization of diffusion model. Advances in Neural Information Processing Systems, 37, 2025.
Mart van Baalen, Andrey Kuzmin, Suparna Nair, Yuwei Ren, Eric Mahurin, Chirag Patel, Sundar Sub- ramanian, Sanghyuk Lee, Markus Nagel, Joseph Soriaga, al. Fp8 versus int8 for efficient deep learning inference. arXiv preprint Xix:22900 17951, 2023.
Patrick von Platen, Suraj Patil, Anton Loshkov Pedro Cuenca, Nathan Lambert, Kashif Rasul, Mishig Davaadorj, Dhruv Nair, Sayak Paul, William Berman, Yiyi Xu, Steven Liu, and Thomas Wolf. Diffusers: 2022.
State-of-tho-art diffusion models. http:://ggthabb.cc//ugugggggggge htwp:///.wo.ooggggggga//a////aar
Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis. arXiv preprinl ar Xix:222066.00 09341, 2023a.
Xiaoxia Wu, Zhewei Yao, and Yuxiong He. Zeroquant-lp: A leap forward in llms post-training w4a8 quan- tization using beating-point formats. arXiv preprint 20f::230700988. 2023b.
Haojun Xia, Zhen Zheng, Xiaoxia Wu, Shiyang Chen, Zhewei Yao, Stephen Youn, Arash Bakhtiari, Michael Wyatt, Donglin Zhuang, Zhongzhu Zhou, et al. (Quant-L.LMI] Accelerating the serving of large language models via (eP6-Centric) AAlgorithm-Systeen Systemm { (Cs-innggn] on modern {GPUs). In 2024 USENIX Annual Technical Conference (USENIX ATC 24), pp. 699 713, 2024.
Guangxuan Xiao, Lin, Mickael Seznee, Hao Wu, Julien Demouth, and Song Han. Smoothquant: Accurate and efficient post-training quantization for large language models. In International Conference on Machine Learning, pp. 38087-38099. PMLR, 2023.
17

---

Published in Transactions on Machine Learning Research (11/2025)
Enze Xie, Junsong Chen, Junyu Chen, Han Cai, Haotian Tang, Yujum Lin, Zhekai Zhang, Muyang Li, Ligeng Zhu, Yao Lu, and Song Han. Sana: Efficient high-resolution image synthesis with linear diffusion transformer, 2024. URL https::/arxii: wwtp:/.rrgabbb////100 10629
Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Lí, Ming Ding, Jie Tang, and Yuxiao Dong. Imagcreward: Learning and evaluating human preferencos for text-to-image generation. Advonces in Newral Information Processing Systems, 36:1590) 15935, 2023.
Zhewei Yao, Reza Yazdani Aminabadi, Minjia Zhang, Xiaoxia Wu, Conglong Li, and Yuxiong He. Zero- quant: Efficient. and affordable post-training quantization for large-scale transformers. Advances in Newral Information Processing Systems, 35:27168-2718 2022.
Tianwei Yin, Michaël Gharbi, Taesung Park, Richard Zhang, Eli Shechtman, Fredo Durand, and Bill Free- man. Improved distribution matching distillation for fast image synthesis. Advances neural information processing systems, 37:47455 47487, 2024.
Aozhong Zhang, Naigang Wang, Yanxia Deng, Xin Li, Zi Yang, and Penghang Yin. Magr: Weight magnitude reduction for enhancing post-training quantization. Advances in neural information processing systems, 97:85109-851333 2024.
Aozhong Zhang, Zi Yang, Naigang Wang, Yingvong Qi, Jack Xin, Xin Li, and Penghang Yin. Comq: A backpropaggaioon-free algorithm for post-training quantization. IEEE Access, 2025.
Tianchen Zhao, Tongcheng Fang, Enshu Liu, Wan Rui, Widyadewi Soodarmadji, Shiyao Li, Zinan Lin, Guobao Dai, Shengen Yan, Huazhong Yang, et al. Vidit-q: Efficient and accurate quantization of diffusion transformers for image and video generation. arXiv preprint arXiv:2406. 02540, 2024.
18

---

Published in Transactions on Machine Learning Research (11/2025)
A Appendix
A.1 Proof for
AdaRound
AdaRound (Nagel al., 2020 and BRECQ (Li et al., 2021) use gradient descent to updale the rounding mask. According to Eqs. and 6 ignoring the regularizer freg (V), we have following theorem which provides theoretical evidence that the origin AdaRound is imbalance across different scales s.
Theorem 1.
Let be the quantization scale corresponding the rounding mask V. Then for gradient descent, given as Vn+1 = VV, aVF(V,), the subtraction (V) is dependent the scalor s.
Proof. Refers Eqs.
and gives:
- Vn()
(9)
OV,
Refers to Eq. and gives:
(v (5-7) oo() (1-o(V))
(10)
Therefore, combining Eq. 999 and
10
the subtraction FV(),) is scaled by 8.
This theorem indicates that the imbalance gradient descent occurs if applying origin AdaRound to FPQ, shown in Figure 4b
In the section we propose a scale-aware version of AdaRound Eq. and Ignoring the regularizer Sreg, we have the following theorem which provides theoretical evidence that the scale-aware AdaRound has balanced gradient descent's update across different scales
Theorem 2. Let 8 be the quantization scale corresponding to the rounding mask V Then for gradient descent, given as V V'a41 V' n - aVF((V w), the subtraction F(V',,) is independent of the scalar
Proof. The learning objective does not change for scale-aware AdaRound. Hence, from Eq. and we give:
oh'(V',)
VF(V',)) v |X - W 8. 0V n
(11)
Refer to Eq.
8 and give:
Øh (h'(V')
(C (<-2) (V-')((-o(VV))
(12)
OV
Combining Eq. and 12
get:
VF(V'.) = V V Wx Wx| -
(VV) (1 (V'))
This result shows that subtraction V VV(V).) is independent of scale
Therefore, the gradient descent is balanced and normalized among different scale with scale-aware AdaRound, as depicted in Figure 4c
A.2 Additional Ablation Studies
In addition to the ablation studies in the main text, we include multiple ablation studies here to further demonstrate the design of FP4DiT.
A.2.1 Effect of Group Size.

---

Published in Transactions on Machine Learning Research (11/2025)
Algorithm MinMax Quantization for FP Format
Require: Full-precision array App, number of bits n, exponent bits ney mantissa bits nm clipping value
marval
1: Aabs -- abs(Apr) App)
2: bias f- 2 - logz (Aabs) + 1ogg 2-)))) )-m) -1
Actip +- min(max(( ApP, marval)) marval)
Slog clamp
(lamp(Logg200 (Loggg (abs (Aclp)) bias,
P9 S --.0(Stng -Mm bias)
result. t- (Annd- x
return result
Model
Group Size
FID $
We compare the group-wise weight quantization result with different group sizes in Table 9 Decreasing the group size g for weight quantiza- tion consistently improves the quantization performance down
128
96.58
256
100.60
PixArt-a
128. According to Park al. (0022), group size g such 128 can result. in sub- stantial improvement while maintaining low latency.
512
113.78
1024
174.38
Table 9: The quantization re-
sults for different. group sizes with PixArt-a.
A.2.2 Effect Calibration Dataset
We investigate FP4DiT's effectiveness under different calibra- tion dataset. sizes N. Table TO reports the HPSv2 score of
N-64
N
N-64
N-64
the W4A8 PixArt-a quantized by our method with N 64, 128, 256, and 512. The results show steady improvements as N increases, with performance saturating beyond N = 256.
25.09
27.43
28.21
28.20
In particular, FP4DIT achieves a strong score of 27.43 with only 128 samples, and further gains to 28.21 at N = 256, while larger calibration does not bring additional benefits. This demonstrates that FP4DiT requires only relatively small cal- ibration set to achieve near-optimal performance, making it both efficient and effective compared to existing approaches.
The quantization results for different sizes of calibration dataset with PixArt-a.
Table 10:
A.2.3 Classifier-Fres Guidance (CFG)
We investigate FP4DiT's effectiveness under differ- ent CFG scales. Table 11 compares FP4DiT's quan- tization performance with PixArt-a for three differ- ent CFG scales, which are 3.0, 4.5 (default), and 7.0. Although FP4DiT is calibrated at CFG = 4.5, it re- mains robust and exhibits only minor degradation under different CFG scales, which is still outstand- ing among the baseline methods as shown in Table
Precision
CFG=3.0
CFG=4.5
CFG=7.0
W4A8
27.22
27.43
26.74
Full Precision
30.97
31.01
31.26
The quantization results for different CFG scales with PixArt-a.
Table 11:
A.3 Extended Experimental Settings
A.3.1 Floating Point Quantization Scheme
Since Floating Point Quantization (FPQ) is not as straightforward as INT quantization, there has not been a simple and unified algorithm for performing FPQ yet. In this paper, we apply Algorithm [ from (Kuzmin ctt al. p perform the FPQ. Unlike (Kuzmin al., (022) learning the clipping value marval and bit allocations between mantissa and exponent part, we use the absolute maximum of the tensor as mazval and perform FPQ with predetermined FP format.
2

---

Published in Transactions on Machine Learning Research (11/2025)
In addition to the FPQ algorithm and the group-wise/token- quantization, provide our quantization hyperparameters in Table 12 Note that the calibration size refers to the number of images sampled from MS-COCO. For each image, we sample its input latent noise across 20 timesteps (50 for Hunyuan) as the calibration data for AdaRound. The activation FP format is for W4A6/ W4A6/W4A3 /W4A8 respectively.
A.4 Evaluation Settings
For CLIP score (Hessel ct al. 2021), we apply the openai-clip library to mea- sure the CL.P score between prompts and generated images. We employ ViT- B/32 as the CLIP model. To measure HPSv2 (Wu et al. 2023a score, we gen- Table 12: Quantization calibration hyperparameters for FP4DIT. crate 3.2k images across the four cate- gories (800 images per calegory) and use the provided human preference predictor to estimate the performance. For the Fréchet Inception Distance (FID) (Heusel ct al. 2017) measurement, we apply clean-fid library to mcasure the FID.
Model
Cali. Size
Cali. Step
Weight Format
Act. Format
PixArt-a
128
2500
E2M1
E2M3/E3M4
PixArt-E
128
2500
E1M2
E2M3/E3M4
Hunyuan
64
2500
E2M1
E2M3/E3M4
A.5 Hardware and Software Resources
We execute our FP4DiT and baseline experiments on two rack servers. The first is equipped with 2 Nvidia A100 80 GPUs, an AMD SPYC-Rome Processor and 512GB RAM. The second server has 8 Nvidia V100 32GB GPUs, an Intel Xeon Gold GPU and 756GB RAM.
Our code is rumning under Python 3 using Anaconda virtual environments and open-source repository forks based on Q-Diffusion (Li et al.) 2023). We modified the code to implement our FP4DiT and enable the interface between Quantization scripts and the Hugging Face Diffusers 3 library. The hardware cost measurement conducted using the ptflops library. We provide a code implementation with README listing all necessary details and sleps.
A.6 Additional Visualization Results
In this section, we present the random samples derived from full-precision and W4A6/W4A8 PixArt-a, PixArt-E, and Hunyuan that are quantized by baselines and FP4DÎT. As depicted by Figures [I 12
14 15 and [6] FP4DiT generates results of impressive visual content. FP4DiT consistently shows superior performance across various DiT models. We list detailed comparison between FP4DiT and the baselines:
1. PixArt-o: On W4A8 (Fig. 11, FP4DiT generates more fine-grained images, e.g., macaroon and wave details. Besides, the basclincs fail to generate a hodgehog while FP4DiT correctly depicts it. On W4A6 (Fig. 12), our method shows near W4A8 performance, unlike the noisier baselines.
PixArt-E: On W4A8 (Fig. 13 there is almost no noise on FP4DiT's generated images, which demonstrates the improvement of our method. As for the image details, FP4DiT has a more natural yoga posture, more fine-grained model face, and more detailed cat hair and whiskers. Similar to PixArt-a, FP4DiT's W4A6 (Fig. 14 images have the least noise and best image quality.
3. Hunyuan: On W4A8 (Fig. 15 FP4DiT's generated images closely align with the full precision model. For example, in the first image, FP4DiT has the same color right face while Q-Diffusion and TMFQ- DM alter the face's color. In the third image, FP4DiT generates the most similar face decoration. In addition, FP4DiT also has the least. noise on Hunyuan DiT. On W4A6 (Fig. 16 all met generate blur images. However, the contour of images identified from FP4DiT's visualization results matches the full precision images, while other baselines fail to produce a prompt-adheeent contour.
https://github. http:iiwbb.coooooaaaaaaaaa /CLII CLLIP
https://github.com
http://tthubbccoo//aPaaaaaaa www.o////aaaaaa//aae.aaoeaaaaeee fid
http:://uuggingfacccco////
http:///gthabb.ccm//svraaassss
pytorch
3

---

Published in Transactions on Machine Learning Research (11/2025)
Full Precision
Q-Diffusion W4A8
TFMQ-DM W4A8
VIDIT-Q W4A8
Q-DiT W4A8
FP4DIT W4A8
Figure 11: More visualization result for W4A8 PixArt-a. Prompts: 'A blue jay standing on a large basket of rainbow macarons!;; 'Frog, in forest, colorful, no watermark, no signature, in forest, 8k'; DDrone view of waves crashing against the rugged cliffs along Big Sur's Garay Point beach. The crashing blue waters create white-tipped waves, while the golden light of the setting sun illuminates the rocky shore.; 'An ink sketch style illustration of a small hedgehog holding piece of watermelon with its tiny paws, taking little bites with its eyes closed in delight. Photo of a lychee-inspired spherical chair, with a bumpy white exterior and plush interior, set against a tropical wallpaper.'

---

Published in Transactions on Machine Learning Research (11/2025)
Full Precision
Q-Diffusion W4A6
TFMQ-DM W4A6
VIDiT-Q W4A6
Q-DIT W4A6
FP4DIT W4A6
Figure 12: More visualization result for W4A6 PixArt-a. Prompts: 'A blue jay standing on a large basket of rainbow macarons!;; 'Frog, in forest, colorful, no watermark, no signature, in forest, 8k'; 'Drone view of waves crashing against the rugged cliffs along Big Sur's Garay Point beach. The crashing blue waters create white-tipped waves, while the golden light of the setting sun illuminates the rocky shore.; 'An ink sketch style illustration of a small hedgehog holding piece of watermelon with its tiny paws, taking little bites with its eyes closed in delight. Photo of a lychee-inspired spherical chair, with a bumpy white exterior and plush interior, set against a tropical wallpaper.'

---

Published in Transactions on Machine Learning Research (11/2025)
Full Precision
Q-Diffusion W4A8
TFMQ-DM W4A8
VIDIT-Q W4A8
Q-DiT W4A8
FP4DIT W4A8
Figure 13: More visualization result for W4A8 PixArt-E. Prompts: 'A very attractive and natural woman, sitting on a yoka mat, breathing, eye closed, no make up, intense satisfaction, she looks like she intensely relaxed, yoga class, sunrise, 35mm'; 'Raalsstcc oil painting of a stunning model merged in multicolor splash made of finely torn paper, eye contact, walking with class street."; 'A cute orange kitten sliding down an aqua slido. happy excited. 16mm lens in front. we see his excitement and scared in the eye. vibrant colors. water splashing the lens.'

---

Published in Transactions on Machine Learning Research (11/2025)
Full Precision
Q-Diffusion W4A6
TFMQ-DM W4A6
VIDIT-Q W4A6
Q-DIT W4A6
FP4DIT W4A6
Figure 14: More visualization result for W4A6 PixArt-E, Prompts: 'A very attractive and natural woman, sitting on a yoka mat, breathing, eye closed, no make up, intense satisfaction, she looks like she is intensely relaxed, yoga class, sunrise, 35mm'; 'Raaisttc oil painting of a stunning model merged multicolor splash made of finely torn paper, eye contact, walking with class street."; 'A cute orange kitten sliding down an aqua slido. happy excited. 16mm lens in front. we see his excitement and scared in the eye. vibrant colors. water splashing on the lens.'

---

Published in Transactions on Machine Learning Research (11/2025)
Full Precision
Q-Diffusion W4A8
TFMQ-DM W4A8
VIDIT-Q W4A8
Q-DIT W4A8
FP4DIT W4A8
More visualization result for W4A8 Hunyuan. Prompts: 'nature vs human nature, surreal, UHD, 8k, hyper details, rich colors, photograph.; 'A transparent sculpture of duck made out of glass. The sculpture is in front of a painting of landscape.'; 'Steampunk makeup, in the style of vray tracing, colorful impasto, uhd image, indonesian art, fine feather details with bright red and yellow and green and pink and orange colours, intricate patterns and details, dark cyan and amber makeup. Rich colourful plumes. Victorian style.'
Figure 15:

---

Published in Transactions on Machine Learning Research (11/2025)
Full Precision
Q-Diffusion W4A6
TFMQ-DM W4A6
VIDIT-Q W4A6
Q-DIT W4A6
FP4DIT W4A6
More visualization result for W4A6 Hunyuan. Prompts: 'nature vs human nature, surreal, UHD, 8k, hyper details, rich colors, photograph.; 'A transparent sculpture of duck made out of glass. The sculpture is in front of a painting of landscape:'; 'Steampunk makeup, in the style of vray tracing, colorful impasto, uhd image, indonesian art, fine feather details with bright red and yellow and green and pink and orange colours, intricate patterns and details, dark cyan and amber makeup. Rich colourful plumes. Victorian style.'
Figure 16:


Reviews:
[{"review_id": "DihuzqkDVG", "reviewer": "Reviewer_nNHa", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "FP4DiT adapts PTQ to Diffusion Transformers with per-layer mixed FP4 formats, a scale-aware AdaRound, and token-wise activation quantization, yielding near-lossless W4A8, competitive W4A6 on three DiTs, and ~1.5× layer-level speedups with lower memory on Blackwell GPUs.\n\nStrengths:\n1. FP4DiT adapts FP formats to DiT by introducing mixed FP4 formats (E3M0/E2M1/E1M2 per layer), which better capture the sensitive range of GELU activations.\n2. FP4DiT presents an enhanced rounding and scaling strategy designed for FP quantization.\n3. Experiments span three representative DiT models with standard T2I benchmarks (e.g., FID, CLIP, HPSv2), include ablations on calibration budget and format choice, and report real-hardware measurements on Blackwell GPUs showing ~1.5× speedups.\n\nWeaknesses:\n1. The novelty is limited. The contribution primarily adapts established FPQ to DiT; the main new component is scale-aware AdaRound.\n2. Human eval is small-scale: Preference studies (15–17 participants) lack statistical significance tests, inter-rater agreement, and detailed blinding/randomization protocols。\n3. While W4A8 is near-lossless, W4A6 shows larger drops on challenging setups.", "claims_and_evidence_yn": "Yes", "claims_explanation": "1. On three DiT models, FID/CLIP/HPSv2 say W4A8 is basically lossless.\n2. The improvements mainly come from scale-aware AdaRound with mixed FP4, as confirmed by the calibration/format ablations", "audience_yn": "Yes", "audience_explanation": "Given the scarcity of prior work on applying FPQ to DiT architectures, this paper fills a timely gap and should serve as a useful reference for subsequent efforts in DiT quantization.", "requested_changes": "1. More ablation study about block-wise optimization will enhance this paper.\n\n2. End-to-end latency analysis will better than per-layer latency analysis.\n\n3. Additional DiT families and harder settings (higher resolutions, longer/complex or multilingual prompts) will demonstrate generality. (not necessary)\n\n4. Deepen the quantization discussion will be better. This paper lacks many quantization references, such as OBQ [1], COMQ [2], QuIP [3], SpinQuant [4], MagR [5].\n\n[1] Frantar, Elias, and Dan Alistarh. \"Optimal brain compression: A framework for accurate post-training quantization and pruning.\" Advances in Neural Information Processing Systems 35 (2022): 4475-4488.\n\n[2] Zhang, Aozhong, et al. \"Comq: A backpropagation-free algorithm for post-training quantization.\" IEEE Access (2025).\n\n[3] Chee, Jerry, et al. \"Quip: 2-bit quantization of large language models with guarantees.\" Advances in Neural Information Processing Systems 36 (2023): 4396-4429.\n\n[4] Liu, Zechun, et al. \"Spinquant: Llm quantization with learned rotations.\" arXiv preprint arXiv:2405.16406 (2024).\n\n[5] Zhang, Aozhong, et al. \"Magr: Weight magnitude reduction for enhancing post-training quantization.\" Advances in neural information processing systems 37 (2024): 85109-85130.", "broader_impact_concerns": "No", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "eK4FY6plNL", "reviewer": "Reviewer_cbZf", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work focuses on the post-training quantization (PTQ) of text-to-image (T2I) diffusion models (DMs) and proposes a floating-point quantization framework, FP4DiT. Considering the non-uniform distribution of weights and activations in DMs, floating-point quantization can reduce the quantization error compared to the existing INT-based framework. The author also fully investigates the GELU activation function and locates that sensitive intervals are important. The proposed FP4DiT is built upon AdaRound, and the author improves it by introducing a scaling tailored for FP quantization and an effective online activation quantization scheme. The proposed FP4DiT is verified across two benchmarks (HPSv2 and MS-COCO) and various DMs (PixArt-$\\alpha$, $\\Sigma$, Hunyuan), demonstrating its effectiveness and efficiency.", "claims_and_evidence_yn": "Yes", "claims_explanation": "This paper conducts a full investigation and reveals some important challenges and bottlenecks of DM PTQ. Both empirical results and theoretical analysis show that FP4DiT would be a solution to the challenges. The theoretical proof of Scale-Aware AdaRound (in the appendix) and hardware cost comparison are particularly impressive. The design of ablation experiments is reasonable, showing how the proposed components of FP4DiT improve the naive FPQ of DMs. In addition, the presentation quality of this work is good, whole paper is easy to follow with clear figures and tables.", "audience_yn": "Yes", "audience_explanation": "The topic of Diffusion Model quantization is important and under-investigated compared to traditional small-scale models (CNNs, ViTs) and LLMs. I believe TMLR's audience working on the efficiency of GenAI will definitely be interested in this paper.", "requested_changes": "- Although the proposed FP4DiT is designed for DiT-based DMs, it seems that it could also be directly applied to UNet-based DMs without much modification. The results on UNet-based DMs, especially SD series (SDXL, SD3), would be interesting and helpful as they are more widely adopted in real applications.\n- While cfg changes, what will be the performance of FP4DiT? Please include related discussions and experiments since this is an important setting in DMs.\n- More recent and widely-used benchmarks and metrics could be included in experiments, such as GenEval [1], DPG-bench [2], and Image Reward [3].\n\n[1] GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment\n\n[2] ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignment\n\n[3] ImageReward: Learning and Evaluating Human Preferences for Text-to-image Generation, NeurIPS 2023", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "4AEN615zqd", "reviewer": "Reviewer_JPJW", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a method to quantize diffusion transformers, covering both weight and activation quantization. For weighs, the authors propose a scale-aware adaptive rounding method to achieve a 4-bit floating-point quantization applied heterogeneously throughout the denoising networking. For activations, the authors consider both 8-bit and 6-bit floating-point formats with per token scaling. Evaluation is conducted on two PixArt models and Hunyuan, using HPSv2, MS-COCO with FID and CLIP, and human evaluation. Overall, the method achieves a good speedup of 1.5x while improving generative image quality over prior methods.\n\nStrengths:\n- The methodology is well motivated by compelling evaluation of the DiT’s time-step dependent activation behavior, and shortcomings of prior uniform quantization methods.\n- The idea of using E3M0 for the input projection before the GELU activations is interesting, and well-reasoned.\n- The authors perform comprehensive evaluations, covering multiple models and metrics. \n\nWeaknesses:\n- While surpassing prior methods in many cases, the image quality is still lacking with visible quantization artifacts in W4A6 and in some W4A8 cases.\n- Several of the claims are overstated, such as superior performance, where the metric improvements may not be statistically significant.\n- Some of the experimental setups are not well described, requiring inspection of source code for reproduction.", "claims_and_evidence_yn": "No", "claims_explanation": "Many of the quality claims have over-emphasized significance, where they instead demonstrate matching or slightly improved behavior compared to baselines. For example, it is not clear that an overall HPSv2 score of 27.78 vs 27.52 is statistically significant. \nAdditionally, the evaluation methodology on FID and CLIP deviates from standard practices. FID on MS-COCO is typically done with 30k samples to improve the reliability and stability of the score. CLIP typically uses VIT-B/16 on 30k samples as proposed by GLIDE. Notably, the authors failed to reproduce the FID result of PixArt-$\\alpha$ (34.05) as reported in the original paper (7.32).", "audience_yn": "Yes", "audience_explanation": "This paper explores several behavior characteristics of diffusion transformers and their impact on weight quantization. The novel method of scale-aware quantization, and the improved results at W4A6 (despite the image quality loss) are also interesting and will be helpful to future quantization works.", "requested_changes": "For acceptance:\n- Improve the accuracy of the FID and CLIP metrics with more samples. The full precision approaches should reproduce (or come close to) the previously reported metrics or explain why a deviation occurs. \n- Soften the claims of superior performance. The FID and CLIP results are better, while the HPSv2 results are marginal.\n- Reduce the computational improvement claim or better position the motivation in the introduction and related works. Currently, it reads as quantization being the only viable approach, yet FP4DiT only achieves a 1.5x speedup. Timestep distillation and model scale distillation / pruning are orthogonal methods that can have greater speedups on edge devices.\n- Improve the experimental setup information. What precision are the MACs done in? What precision do the group and token scale coefficients use? How were the results in Table 6 computed (e.g., sample size, batch size, resolution, etc.)?\n\nTo Strengthen:\n- Understanding why the speedup is only 1.5x for W4A8 would be helpful. Is it truly computationally bound since the memory movement is reduced by 4x for the weights and 2x for the activations? Which layer is the bottleneck?\n- When looking at the CLIP impact vs calibration steps, the curve appears to be increasing from 10k to 20k. Could 40k potentially outperform 2.5k samples? \n- Is there a dependency on the calibration set used? MS-COCO is lower-resolution compared to PixArt-$\\Sigma$ and Hunyuan and lacks the variability of the pre-training datasets.\n- Investigation into FP4DiT’s performance on timestep distilled models would also be appreciated (e.g., PixArt-$\\delta$).\n- Understanding why Hunyuan experienced such a large FID and CLIP degradation when moving to W4A6 could be impactful, since the same extreme was not observed in the PixArt models.", "broader_impact_concerns": "No concerns.", "additional_comments": "Questions:\n- The motivation for E3M0 prior to the GELU is to capture the negative output region near zero. Would this still be necessary if the GELU FFNs were hardened with ReLU as is sometimes done with vision transformers for edge devices? And would models that utilize SiLU prefer a different quantization?\n- Given the degradation of W4A6 vs W4A8, and the lack of existing hardware support, is there a reason to prefer the 6-bit activations over 8-bit? It seems to me that the tradeoff is not worthwhile. \n\nSuggestions:\n- There should be an explanation (including the abstract) of what W4A8 means; it may be unclear to readers unfamiliar with the notation.\n- Replacing the Northern Lights image example in Figure 7 may make the quantization results easier to observe – Landscapes are not ideal for visual evaluation, as they are relatively easy for even small models to generate convincingly.\n- Figure 4 and Figure 5 should have subfigure labels to easily see what each represents without reading the full caption.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)