ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generation
Authors: Tianchen Zhao
Tongcheng Fang
Haofeng Huang
Rui Wan
Widyadewi Soedarmadji
Enshu Liu
Shiyao Li
Zinan Lin
Guohao Dai
Shengen Yan
Huazhong Yang
Xuefei Ning
Yu Wang
Venue: ICLR 2025 Poster (2025)
Keywords: video generation
low-bit quantization
diffusion model
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We design the quantization scheme ViDiT-Q tailored for Video & Image Diffusion Transformer Quantization. It achieves W4A8 quantization with negligible performance loss,  which brings 2x memory and 1.5x end2end latency speedup.

Abstract: Diffusion transformers have demonstrated remarkable performance in visual generation tasks, such as generating realistic images or videos based on textual instructions. However, larger model sizes and multi-frame processing for video generation lead to increased computational and memory costs, posing challenges for practical deployment on edge devices. Post-Training Quantization (PTQ) is an effective method for reducing memory costs and computational complexity.
When quantizing diffusion transformers, we find that existing quantization methods face challenges when applied to text-to-image and video tasks. To address these challenges, we begin by systematically analyzing the source of quantization error and conclude with the unique challenges posed by DiT quantization. Accordingly, we design an improved quantization scheme: ViDiT-Q (**V**ideo \& **I**mage **Di**ffusion **T**ransformer **Q**uantization), tailored specifically for DiT models. We validate the effectiveness of ViDiT-Q across a variety of text-to-image and video models, achieving W8A8 and W4A8 with negligible degradation in visual quality and metrics. Additionally, we implement efficient GPU kernels to achieve practical 2-2.5x memory optimization and a 1.4-1.7x end-to-end latency speedup.

Published as a conference paper at ICLR 2025
VIDIT-Q: EFFICIENT AND ACCURATE QUANTIZA- TION OF DIFFUSION TRANSFORMERS FOR IMAGE AND VIDEO GENERATION
Tianchen Zhao Tongeheng Fang"? 12, Haofeng Huang', Rui Wan', Widyadewi Soedarmadji', Enshu Liu' Shiyao Li", Zinan Lin", Guohao Dai Shhnngnn Yan', Huazhong Yang', Xuefei Ning', Yu Wang"" TTsinghua University, "Ifinigencce AI, Microsoft, Shanghai Jiaotong University
ABSTRACT
Diffusion transformers have demonstrated remarkable performance in visual gen- eration tasks, such as generating realistic images or videos based on textual in- structions. However, larger model sizes and multi-frame processing for video generation lead to increased computational and memory costs, posing challenges for practical deployment on edge devices. Post-Training Quantization (PTQ) effective method for reducing memory costs and computational complexity. When quantizing diffusion transformers, we find that existing quantization meth- ods face challenges when applied to text-to-image and video tasks. To address these challenges, we begin by systematically analyzing the source of quantiza- tion error and conclude with the unique challenges posed by DiT quantization. Accordingly, we design an improved quantization scheme: ViDiT-Q (Video Image Diffusion Transformer Quantization), tailored specifically for DiT models. We validate the effectiveness of ViDiT-Q across a variety of text-to-image and video models, achieving W8A8 and W4A8 with negligible degradation in visual quality and metrics. Additionally, we implement efficient GPU kernels to achieve practical 2-2.5x memory saving and 1.4-1.7x end-to-end latency speedup.
ViDiT-Q: Video & Image Diffusion Transformer Quantization
"Accurate" & L "Efficciett
FP16
2.5x
FP16
Basefine
VIDITQQ
Baseline
VIDIT-Q
2-2.5x
Memory Opt.
W8A8
W8A8
W448
W448
1.4-1.7x Latency Opt.
Figure I: ViDiT-Q addresses the challenges that existing quantization methods face in text-to-image and video generation. It achieves quantization with negligible performance loss, delivering 2-2.5x memory savings and 1.4-1.7x latency reduction.
INTRODUCTION
Diffusion Transformers (DiT's Peebles & Xie 2023) and video generation tasks Singer et al. (2022) have garnered significant research interest since the impressive performance of SORA OpenAl
2024). However, the increasing model size poses challenges for application and deployment on "Corresponding Authors

---

Published as a conference paper at ICLR 2025
edge devices. In the realm of video generation, processing multiple frames imposes significant burden on both memory and computation. For example, the OpenSORA HPC-AI (2024) model consumes over 10 GB of GPU memory to generate single 512x512 resolution video with only 16 frames, taking about 50 seconds on an Nvidia A A100 GPU.
Model quantization Jacob et al. (018)) has proven to be effective compression method, and is compatible with diffusion efficieccy improvement techniques from other perspectives such as efficient sampling Liu al. (2023a b); Ni al. (2024); Yuan et al. (2024a) and caching Ma et al.
22033; Zhang et al. (2024a 2025a): Zou et al. 2024) By compressing high bit-width floating-
point (FP) data into lower bit- bit-w integers, the computational and memory costs can be effectively reduced. The guantization of DiT models remain muder-explored. While some prior studies [Wu] et al. (2024): [Cen et al. (2024) explores DiT quantization for class-conditioned generation, we empirically obeserve challenges when applying them to more challenging text to image and video generation tasks with larger-scaled model (as seen Fig. T.. Another line of recent literature /Zhang al. c b 2025b); Yuan et al. (2024b); Pu et al. (2024) focus optimizing the attention computation, while we focus on quantizing the linear layers.
To address this challenge, we begin by analyzing the sources of quantization error and conclude the primary issue stems from improperly large quantization range caused by high data variation within quantization groups. Next, investigate the unique challenges in the specific application. For DiT models, we observe significant variation in multiple dimensions. For visual generation task, we find that merely reducing quantization error is insufficient to preserve the multi-faceted generation quality, such as textual alignment Wu et al. (2021) and temporal consistency Liu et al. (2023d).
In light of the above findings, we further investigate the reason for the failure of existing methods and introduce corresponding modifications. First, to handle data variation in multiple dimensions, we carefully examine the limitations of quantization grouping of existing methods from the perspectives of both algorithm performance and hardware efficiency, and highlight the need for fine-gr fine-grained and dynamic quantization parameters. Second, in response the unique time-varying channel imbal- ance problem, we analyze the shortcomings of existing scaling and rotation based channel balancing techniques, and design "static-dynamic" channel balancing technique that combines the strengths of both approaches. Finally, to preserve multiple aspects of generation quality under lower bitwidth, we introduce a metric-decoupled mixed precision scheme, which "decouples" the effects of quanti- zation across different dimensions for sensitivity analysis.
We summarize our contributions as follows:
1. We conduct extensive analysis and identify the major source of quantization error and unique challenges for quantizing the DiT model and visual generation task.
2. We design improved quantization scheme ViDiT-Q, tailored for DiT models, containing techniques accordingly to address these challenges.
3. We validate the effectiveness of ViDiT-Q on extensive DiT models for both image and video generation, and further implement efficient GPU kemnlss to achieve practical hard- ware savings and acceleration.
2 RELATED WORKS
2.1 DIFFUSION TRANSFORMERS FOR IMAGE AND VIDEO GENERATION
Diffusion Transformers (DiTs), which employ Transformers Vaswani et al. 2017 replace the CNN-based diffusion backbones (U-Net M.NeeeRoonbbbrgee et al. (2155) in prior research Rombach et al.
2022), have achieved remarkable performance visual generation. Image Generation: DiT Pee- bles & (023) and UVIT Bao al. (2.22) pioneer the use of transformers as diffusion backbones.
PixArt-a Chen et al. 20033 explores text-to-image generation with DiTs. Video Generation: Early
video generation models Ho et al. (2022b a); Guo et al. (2023) mainly adopted CNN backbones.
Latte Ma et al. (024)) pioneer the use of transformers for text-to-videe video generation. The success of
Spen (2024) inspire the development of video diffusion transformers such OpenSORA
HPC-AI (2024). Both high-resolution image generation and colti-frame video generation add to
hardware costs, necessitating efficiency improvements.
2

---

Published as a conference paper at ICLR 2025
2.2 IMAGE AND VIDEO GENERATION EVALUATION METRICS
Visual generation can be evaluated from multiple aspects, and many metrics are introduced accord- ingly. Image Metries: FID Heusel et (0177) and IS Salimans et al. (2016) are two commonly adopted metrics for measuring the Inception network feature difference between generated and ref- erence images for quality and fidelity assessment. ClipScore HrpScore|Hessell [Hsssll et al. (202T) evaluates how well the generated image follows the prompt instruction (text-image alignment), while ImageReward Xu al. (2023), HPS Wu al. (2023b) incorporates human preference by collecting actual user data to train the reward model. Video Metries: FVD extends the feature-based metric FID to the video domain. CLIPSIM Wu et al.
(2021) estimates the similarity between video and text instructions. CLIP-temp Esser et al. (2023) measures the semantic similarity between video frames. Flow-score proposed as part of the video evaluation benchmark EvalCrafter Liu et al. (2023c) to assess mo- tion quality. EvalCrafter also adopts DOVER Wu et al. (2023a) for video quality assessment. These metrics from multiple aspects should be considered when evaluating the effect of quantization.
2.3 MODEL QUANTIZATION
Post Training Quantization (PTQ) has proven to be an efficient and effective model compression method Nagel et al. (202T) Diffusion Model: Focusing the unique timestep dimension, prior research Q-Diffusion Li et al. (2023) and PTQ4DM (Shang et al. (2023) collects timestep-wise ac- tivation data determine quantizaiton parameters. Transformer: Prior research made significant progress in quantizing transformers for both ViTs Liu et al. (2021) and language models Yao et al. 22022) One major focus is addressing the channel imbalance issue. SmoothQuant Xíao et al. (2024) introduces channel -wise scaling to balance the difficulty of weight and activation quantization, while et al. (2024) employs orthogonal matrix rotations to distribute values more evenly across channels. DiT: D-DiTCChen et al. (2024) tackles channel-wisg imbalance by assigning differ- ent quantization parameters to different channels. PTQ4DiT et al. (2024) addresses time-varying channel imbalance by designing fixed channel balance mask that fits all timesteps. While these methods improve quantization from various angles, directly applying them to the more chal- lenging task of text-to-image/viden generation in DiT models results in notable performance degradation. In Sec. 4 we thoroughly discuss their limitations and propose novel techniques to overcome these challenges.
PRELIMINARY ANALYSIS
3.1 QUANTIZATION ERROR ANALYSIS
Consider the quantization problem seeking the optimal quantization strategy minimize the difference between the quantized model and the floating-point model. An usual approach to
surrogate this task into minimzing the layer-wise quantization error for weight W and activation X:
min Ltask(frr) (frr.
min Wy
- -(W0- 1 (1) (1)
where fFP, fa denotes the network with layers. The Wg, X, represents quantized weight and activation. The weight and activation are quantized within each group G (e.g., tensor-wise, channel- wise). The quantization process approximates the full-precision x with integer 2mm and quantization parameters (scaling factor 8, zero point z): x RJ t s(Zim z). The elements within certain group of size y. represented by vector x R2 shares the same quantization parameters (s and z). The quantization operator Q with b bits is described as:
Fin Q( 8, 2, clamp (1 + 2, -1)
(2)
The function clamp(x; c) clamps the values into range [a, c). the |.1 is the mund-to-nearce oper- ator. As discussed in prior literature Nagel et al. (202T). the quantization error mainly consist two trade-off, the clamping error could
parts, the clamping error and the rounding error. They
3

---

Published as a conference paper at ICLR 2025
be reduced with larger scaling however, this in turn increases the rounding error, which lies in range In the minmax quantization scheme adopted by most recent literature and deploy- ment tools, the scaling = (max(r)- - min(x))/(2* 1) are chosen to set the quantization range in [max( min(x)), which avoids the clipping error. Therefore, the major source of the quantiza- tion error arises from the rounding error with large when large data variation exists within the group. For example, when the group size is large (i.e.. tensor-wise), the range are determined by small portion of large values, making the quantization range unnecessarily large for the majority of elements, thus resulting in larger rounding error. Recent literature Chee et al. (2024) introduces the idea of "incoherence processing" echoes this finding. The data group x E R is defined to be j-coherent if: max(x) 2 |l|lW|| /g. where || || indicates the Frobenius norm, and is the number of elements. The data group with higher incoherence are harder to quantize, since the largest element is an outlier relative the average magnitude. Additional incoherence processing to ensure balanced data distirbution within group is essential to reduce the quantization error.
3.2 UNIQUE CHALLENGES FOR DITS AND VISUAL GENERATION
Challenges for DiT model: As mentioned above, data variation within group incur large quanti- zation error. We conduct comprehensive analysis for DiT data distribution, and discover that DiT model witness high data variation in multiple levels as presented in Fig. 2
Token-wise Variation: We observe notable variation between the visual tokens. Specifi- cally, for vidco DiTs, the variations exist both along the spatial and temporal dimension.
Condition-wise Variation: For conditional generation, the classifier- featiier-rrr-guidanneee Ho (2022) (CFG) conducts two separate forwards with and without the control signal (often implemented with batch of 2). We observe notable difference between the conditional part (red square) and the unconditional part (blue square).
Timestep-wise Variation: Diffusion method iterates the model for multiple timesteps. We observe notable variation in activation for the same layer across timesteps.
For both the weight and activation, we witness significant differ- ence across different channels. Specifically, the activation channel variation demonstrate time-varying characteristics.
Channel-wis Variation:
Challenges for visual generation task: As described in eq. 22. the minimization of quantization error is often adopted as the proxy task for quantization. However, for visual generation, the gen- eration quality could be evaluated from multiple perspectives (e.g., aesthetic, alignment). Simply regularizing the absolute error may not be sufficient for assessing the quantization's effect on visual generation. (discussed in more details in Sec. (43) For video generation, more aspects related to the temporal dimension should be included (e.g., temporal consistency, temporal flickering..
4 VIDIT-Q: QUANTIZATION SCHEME TAILORED FOR DITS
As presented in Fig. 2 address the aforementioned challenges, we design ViDIT-Q. Firstly, we highlight the importance of choosing fine-grained and dynamic quantization parameters to avoid data variation in large group. (Sec. [.T). Secondly, we design static-dynamic channel balance technique to handle the unique viry-igrying channel-wise data variation within group (Sec. 4.2) Finally, considering the quantization's effect of multiple aspects on generation quality, we design metric decoupled mixed precision method to preserve performance under lower bitwidths (Sec.
4.1 FINE-GRAINED GROUPING AND DYNAMIC QUANTIZATION
As discussed Sec. 3.1 high data variation within the quantization group (i.e., high incoherence) is major source of quantization error. Adopting coarse-grained quantization grouping with larger group size is more likely to include data with higher variation. For instance, in densor-wise quanti- zation groupings, used in prior research Wu al. (2024), the group contains all tokens, leading to high variation (as shown in Fig. 2). This suggests that finer groupings should be used as long as they do not impede efficient hardware implementation. In hardware implementations, the data
4

---

Published as a conference paper at ICLR 2025
When adopting minmax quantization scheme, The major of quantization the rounding error due large data variation within group.
Use Mininmizing quantization error as objective, may not be adequate for visual generation task
Quantization Error Analysis (Sec. 3.1)
Xtst @.: ciampl (+) 2,0,2"10
lesse (frr.fa)
mee Σ |1000 - o (( f2 200%)l
T
Text Alignment
Unique Challenges (Sec. 3.2)
Wsual Quality (Fidelity)
Time Consistency
Token--wire Variation
CFG wise Variation
Timestep-wise Variation
14] Time-varying Channel-wise Variation
Quantisation's Effect on multiple aspects
ViDiT-Q's Methodology (Sec. 4)
Resolve
Fine-grained grouping and Dynamic Quantization (Sec 4.1)
Static-Dynamic Channel Balance (Sec 4.2)
Mixed Precision (Sec 4.3)
Figure 2: The overall framework of ViDIT-Q. We begin by analyzing the sources of quantization error and identifying the unique challenges faced by DiT models and visual generation tasks. Cor- respondingly, develop specialized techniques to address these challenges.
summed together should share the same quantization parameters (i.e., belong to the same quanti- zation group) to avoid the overhead of casting integer values to floating point for summation. In transformer quantization, where the majority of computation occurs in the Linear layers, summation occurs along the input-channel dimension of the weights and activations. Therefore, despite the "channel-wiss activation quantization in Q-DiT Chen et al. (2024) enhances performance, it brings difficulty in hardware acceleration. Differently, we adopt the hardware- friendly "channel-wise and "token-wise" quantization groupings for weights and activation. This approach compresses the group size for activation quantization to the number of channels, introducing negligible overhead compared to coarse grained groupings, and is supported by mainstream inference frameworks Zhao (et al. 2024b); Lin et al. (2024).
For diffusion model, two additional dimensions, "condition-wise" and "timestep-wise" variation are introduced. Using static quantization parameters across all timesteps and conditions results in equivalent larger group sizes with larger data variation. For example, PTQ4DiTWWu et al. (2024) adopts the tensor-wise static activation quantization grouping, and fails to hanlde the high variation in the token, timestep dimensions. To address timestep-wist variation, previous methods [So et al. (204)) adopt timestep-wise static quantization parameters. However, the determination of these quantization parameters are costly (requires iterative training) and face difficulties when general- izing across solvers. In contrast, we propose using "dynamic" quantization parameters, which are computed online and naturally adapt varying timesteps and conditions. This approach acts as the upper bound of algorithm performance for resolving the timestep-wist variation issue. The addi- tional hardware cost is negligible, as it only requires determining the max and min of the data group and can be fused with previous operations to further minimize overhead. More detailed profiling and analysis are presented in Sec. (5.3
4.2 -YNIMIDD CHANNEL BALANCING
As mentioned above, by incorporating fine-grained grouping and dynamic quantization, the data group is reduced to vector with C channels. Consequently, reducing data variation within the group (i.e., channel balancing) is crucial for minimizing quantization error. As illustrated in Fig. 2 channel-wise data variation is evident in transformer models. Specifically, for DiTs, the de- gree of channel imbalance varies significantly across timesteps. Existing channcl-scaling or rotation based techniques struggle with the unique "time-varying channel imbalance". We investigate the reasons for their failure and design a specialized "static-dynamic" channel balance technique.
Scaling based methods Xiao et al. (2024) introduce per-channel balancing mask R By dividing the activation with and multiplying s with weights, shifts the quantization difficulty from activation to weights, and vice versa. The mask could be calculated follows:
5

---

Published as a conference paper at ICLR 2025
Existing Channel Balance's Challenges
Dynami-D Channel Balance (Sec. 4.2)
Very Stgiificntt Belatheryy om iméalance
Timespp
Scale Skit Table (Trinable Panimnned
1
Consaane Iiteros
I scau 2053
Scaling-baser based: Single a could not fit time varying distribution
Max/Menn-4075
Attnn
AGD
"Seatic Initial Channel Imbatance" "Dynamic Timestop-wise Variation"
Boco) W
Cumbirn
Rotation-based: Outlier still exists after rotation, 12.8x hard for 4-bit(16 levels)
Scale e based Method
+
Rutation-lae -eased Methadi
The illustration of static-dynamic channel balancing. Left: the limitations of existing rotation and scaling based channel balancing techniques. Right: the reason for time-varying imbal- ance could be decomposed into the static and the dynamic part.
Figure 3:
Y (Xdiag(s) (((ig(())))(((da(((()))))))) X W; s; max (axx(( /mxx))/mmxx((( (
(3)
where X, Y. W represents the input activation output activation, and weights. The s is a channel- wise balancing mask, is hyperparameter. Channel balancing could effectively alleviate the input thannel-wise variation. However, we empirically discover that it is sensitive to a choices. For dif- ferent timesteps, the degree of activation channel imbalance changes, suitable also changes. Employing the same a for earlier stages may shift too much difficulty from weights to activations, harming the activation quantization, and vice versa for latter stages. Introducing multiple as for different timesteps can resolve this issue. However, it necessitates different versions of weights for various timesteps. Optimizing for the optimal a cross all timesteps is alos challenging.
Rotation based methods Ashkboos et al. 2024): Liu et al. (2024) introduces an orthogonal rotation matrix Q, such that QQ
and [Q] 1. Multiplying the matrix Q on the left and right of the data preserves computational invariance Y = XW" (XQ)(Q"W) The rotation matrix makes the data values more evenly distributed along channels. Quantizing the rotated matrix XQ with less incoherence could reduce quantization error. The rotation based method requires no parameter tuning and naturally adjust to varying degree of channel imbalance across timestep. However, as seen in Fig. 3 some channels are still prominently larger than others after the rotation.
To overcome these limitations, we analyze the data distribution of DiT models and discover that the time-varying channel imbalance phenomenon arises from the "feature modulation" that aggregates the timestep embedding with the feature (as shown in Fig. 3 This phenomenon can be decomposed into two parts: the "static" initial activation distribution orginating from the pretrained "scale shift table" and the "dynamic" variation introduced by the time embedding. Inspired by these findings, we propose combining scaling and baaed channel balaneing methods to leverage the strengths of both. The scaling-based method addresses the "static" channel imbalance at the initial denoising stage, avoiding the need for multiple as for varying distributions. The rotation-based method is then utilized to address the "dynamic" varying distribution. Since the scaling method has already alleviated extreme channel imbalance, the rotation method ensures balanced distribution.
4.3 METRIC DECOUPLED MIXED PRECISION DESIGN
The aforementioned techniques can effectively reduce the "incoherence" of data distribution, thereby decreasing quantization error. However, still observe notable quality degradation with lower bit-widths (W4). Upon investigating the reasons for this issue, we find that some layers, de- spite exhibiting relatively low quantization error, significantly impact overall quantization. This suggests that layers have varying quantization sensitivity, and quantization can "bottlenecked" by certain highly sensitive layers. This aligns with the discussions in Sec. 3.1 and Sec. 3.2 Merely

---

Published as a conference paper at ICLR 2025
Metric- Decruppe Mixed Precision (Sec. 4.3)
20 steps denoising
G-10
ntorade cach range
gu hobal
Lapen senaitiiity
Step1: Assign furige for tach Group
Tamohlis
5.54
Aug. Bitwidht:
Deuriie over
F916 wthh sareiee
5.00
Step2: Metric- Deccupled Senskinilty Ansepiis
Inter Blick
Intra Bicck
norrss for
nuch group
andize Quuell my Digradattin
Snttrrten Change
Stepl:: Bi--Withh
Quanttition effects are highly correlated with layer types
Allocation
Parameter Budget
Figure 4: The illustration of metric decouple mixed precision. Left: The findings of layer sensi- tivity and quantization's 's effect. Right: the overview of metric decoupled mixed precision.
minimizing quantization error may not be sufficient, especially for visual generation tasks. Quanti- zation's effects on multiple aspects of generation quality should be considered.
To address the "bottleneck" phenomenon, straightforward solution "protect" these sensitive layers. The main challenge in mixed precision allocation lies in accurately identifying these sensitive layers. Prior literature Yang et al. (2023) measures sensitivity by quan- tizing specific layers and calculating the Mean Squared Error (MSE) with the floating- point output. However, MSE alone does not accurately reflect overall generation performance. Consistent with previous studies Sui et al. (2024), we discover that MSE tends to overemphasize content changes at the expense of visual quality degradation and temporal consistency.
assign higher bit-widths to
Inspired by prior research Zhao et al. (2024a), we find that quantization's 's impact on various as- pects of generation quality is strongly correlated with layer types. Specifically, cross-attention layers contribute significantly more to "content change" than other layers. In contrast, "visual qual- ity" is primarily influenced by spatial attention and feedforward network (FFN) layers, while "tem- poral consistency" is chiefly affected by temporal attention layers. Given these diverse impacts generation, the quantization effects of these layers should not be directly compared.
Building this finding, propose to "disentangle" the mixed influences of quantization multiple aspects and develop a metric-decoupled mixed precision method as presented in Fig. To account for sensitivity variation across timesteps, we divide the denoising process into four equal ranges and conduct sensitivity analysis for them respectively. Given a target bit-width budget, we categorize layers into three groups and allocate the budget based the MSE error between FP16 generation and each group when quantized. For each group, we use specific metrics (VQA. Clip- Score, and FlowScore) sensitivity measures for the layers within that group. Finally, we iteratively assign higher bit- widths to the most sensitive layers within each group until reaches the budget.
EXPERIMENTS
5.1 IMPLEMENTATION DETAILS AND EXPERIMENTAL SETTINGS
Video Genration Evaluation Settings: We apply ViDiT-Q OpenSORA HPC-AI (2024). the videos are generated with 100-steps DDIM with CFG scale of 4.0. The mixed precision is only adopted for the challenging W4A8. The evaluation contains two settings.
(1) Benchmark suite: We evaluate the quantized model on VBench Huang et al. (023) to provide comprehensive results. Following prior research Ren et al. (2024). we select major dimensions from Vbench. (2) Multi- aspects metrics: We select representative metrics, and measure them OpenSORA prompt sets. Following EvalCrafter Liu et al. (2023c), we select CLIPSIM and CLIP- Temp to measure the text- video alignment and temporal semantic consistency, and DOVER Wu al. (2023a) 's video quality assessment (VQA) metrics to evaluate the generation quality from aesthetic (high-level) and technical (low-level) perspectives, Flow-score and Temporal Flickering consistency. We also present results on the UCF- 101oooroo et al. (2012). adopting FVD|Unterthiner et al. (019) the metric for OpenSORA and Latte Ma al. (024 in the Appendix Sec. D.I
used for evaluating the temporal
7

---

Published as a conference paper at ICLR 2025
Table 1: Performance of ViDiT-Q text-to-video generation on VBench evaluation benchmark suite. The bit-width "16" represents FP16 without quantization. We omit some baselines that fails produce readable content under W4A8. The mixed precision are applied for ViDiT-Q W4A8.
Bit-width (W/A)
Imaging Quality
Aesthetic Quality
Motion
Dynamic Degree
BG.
Subject Consist.
Soene
Overall
Method
Smooth.
16/16
57.12
96.28
56.94
26.21
Q-Diffusion
8/8
60.38
55.15
94.44
68.05
94.17
87.74
36.62
25.66
Q-DiT
8/8
60.35
$5.80
93.64
68.05
94,70
86,94
32.34
26.09
PTQ4DFT
55.53
95.89
96.02
91.26
34.52
25.32
SmoothQuant
8/8
62.22
55,90
95.96
68.05
94.17
87.71
36.66
25,66
Quarot
8/8
60.14
53.21
94,98
66.21
95.03
85,35
35.65
25,43
ViDiT-Q
8/8
63.48
56.95
96.
61.11
95.84 84
90.24
38.22
26.06
Q-DIT
4/8
23.30
29.61
97.89
4.166
97.02
91.51
01.00
4,985
PTQADET
4/8
37.97
31.15
92,56
9.722
98.18
93.59
3,561
11.46
SmoothQuant
4/8
46.98
44.38
94.59
21.67
82.79
26.41
18.25
Quarot
4/8
44.25
43.78
92.57
66.21
94.25
84.55
28.43
18.43
VIDIT-Q
4/8
61.07
55.37
95.69
58.33
95.23
88.72
36.19
25.94
Bit-width (W/A)
VQA- Aesthetic
VQA- Technical
A Flow
Method
CLIPSIM
CLIP-Temp
Score. (L)
FP16
16/16
0.1797
0,9988
63.40
50.46
O-Diffusion
0,1781
0,9987
51.68
38.27
0.328
Q-DiT
878
0.1788
0.9977
61.03
34,97
0.473
PIQ4DiT
8/8
0,1836
0.9991
54.56
53.33
0.440
SmoothQuant
8/8
0.1951
0.9986
59.78
$1.53
0.331
VDT-QQ
Quarol
0,1949
58.73
52.28
0.215
ViDI-Q
8/8
0,1950
0.9991
60.70
54.64
0.089
Q-DIT
6/6
0.1710
0.9943
11.04
1.869
41.10
PTQ4DIT
6/6
0.1799
0.9976
59.97
43.89
0.997
SmoothQuant
0,1807
0,9985
56.45
48.21
29.26
Q.Di7
Quarot
0,1820
0.9975
61.47
$3,06
0.146
ViDiT-Q
6/6
0.1791
0,9984
64,45
51,58
0.625
Q-DiT
4/8
0.1687
0,9833
0.007
0.018
3,013
PTQ4DIT
4/8
0.1735
0.9973
2,210
0.318
0.108
PIQ4DIT
SmoothQuant
0,1832
0,9983
31.96
Quarot
4/8
0,1817
0.9965
47.36
33.13
0.326
VIDITQQ
4/8
0.1809
0.9989
60.62
49,38
0.1.3
Figure 5: Performance of text-to-video generation OpenSORA prompt set. Left: The com-
parison of generation quality for different quantization methods under different bitwidths. The Q- diffusion for W6A6 and W4A8 are omitted since it fails generate readable content. Right: Visu- alization of generated videos for DiT quantization methods under W4A8.
Image Evaluation Settings: We apply ViDiT-Q to PixArt-a model, the images are generated with 20-steps DPM-solver with CFG scale of 4.5, No mixed precision adopted for W4A8. We choose FeDee et al. (2017) for fidelity evaluation, Clipscore Hessel et al. (202I) for text-image align- ment, and Rewad Xu al.) (2023) for human preference. These metrics measured on the first 1024 prompts COCO annotations.
Hardware Implemention Settings: We implement efficient quantized GEMM CUDA kernels for practical resource savings. Following SmoothQuant [iaao et al.) (2024), the ocaling-based chan- nel balance factors are fused into the previous layer. Kernel fusion also adopted by integrating the quantization operation and Hadamard transformation into the previous LayerNorm, GeLU, and residual operations, thereby minimizing the quantization overhead. We measure the latency and memory savings the Nvidia A100 GPU using CUDA12. 1, the memory usage is measured with the Py Torch Memory Management APIs PoTch (202),, and the latency is profiled with Nsight tools NVIDIA The profiling is conducted with batch size of 1, and 20 denoising steps.
8

---

Published as a conference paper at ICLR 2025
Bit-width (W/A)
Method
FID()
CLIP()
IR(1)
16/16
73.34
0.258
0.901
8/8
96.54
0.239
0.186
Q- Q-Diffusion
4/8
91.95
0.228
-0.224
FP16
ViDiT-Q
8/8
73.60
0.256
0.854
Q-DiT
4/8
475.8
0.127
-2.277
8/8
127.9
0.217
-1.216
PTQ4DiT
4/8
171.9
0.177
-2.064
8/8
75.61
0.259
0.917
ViDiT-Q
4/8
74.33
0.257
0.887
Q-DiT
PTQ4DiT
Figure 6: Performance of ViDiT-Q text-to-image generation on COCO. Left: The metric scores of PixArt-a quantization. Right: Generated images comparison of W4A8 quantization.
5.2 MAIN RESULTS
Text-to-video generation on VBench and OpenSORA prompt set: As presented in Tab. [ and Fig. [] existing diffusion quantization methods (e.g., O-Diffusion) designed for U-Net-based mod- els incur notable quality degradation (VQA in Fig. 5 and mutliple aspects in Tab. D even at W8A8. These methods often fail to generate readable content (resulting blank images or noise) at lower bit-widths, which are omitted in the table. The primary reason for their failure is the use of coarse-grained and static quantization parameters. The baseline DiT quantization methods (Q-DiT, PTQ4DiT) achieve acceptable performance at W8A8 and W6A6. However, for W4A8, they fail to manage channel imbalance, producing unreadable content, as shown in the right part of Fig. 5 For Q-DiT, its grouping mechanism struggles to handle the large output channel variation under W4. For PTQ4DiT, the use of fixed channel balancing scaling is insufficient to manage the large varia- tion across timesteps. Language model quantization techniques (e.g., SmoothQuant Quarot) also perform comparably at W8A8 and W6A6. However, significant degradation is observed at the more challenging W4A8, highlighting the importance improved channel balancing.
Text-to-image generation on COCO: Similar to video generation, as seen in Fig. 6 existing quantization schemes, which employ fine-grained static quantization parameters (Q-Diffusion and PTQ4DM), encounter challenges even with W8A8 configurations. These difficulties arise from the improper handling of activation data variation across multiple dimensions. While Q-DiT achieves satisfactory performance under W8A8, it struggles under W4A8 due to output channel-wise imbal- ance. contrast, ViDiT-Q consistently maintains performance across bitwidths.
5.3 HARDWARE RESOURCE SAVINGS
Memory footprint reduction. Fig. (a) shows the GPU memory usage of ViDiT-Q and the FP16 baseline. ViDiT-Q can reduce the memory from two aspects: (1) Weight quantization reduces the allocated memory for storing model weights. (2) Activation quantization reduces allocated memory to store intermediate activations. Combining the two benefits, ViDiT-Q can effectively reduce the peak memory footprint by 1.99x for WSA8.Due the adoption of mixed precision, the memory savings under W4A8 are slightly less than the theoretical value, achieving a 2.42 reduction.
Latency speedup. We present the latency speedup Fig. (b). Replacing FP16 layers with our ef- ficient INT8 kernels achieves approximately x acceleration. Taking into account the unquantizable layers (e-g., norms, non-linears, attention) and the overhead from quantization (FP to INT conver- sion, channel balancing scaling, and rotations), the overall speedup is 1.71 x. We also compare our quantization method with a standard baseline ("Naive W8A8") that lacks dynamic and fine-grained quantization well channel balancing techniques. As shown, the incorporation of our methods significantly improves generation quality while introducing only minimal hardware overhead (from 1.73x to 1.71x). For W4A8, since current DiT computation exhibits "compute- -ompute-bound" charac- teristic, the W4A8 CUDA kernel primarily saves memory without enhancing efficiency. Due to the of mixed precision, the overall latency speedup is smaller (1.38x) compared to W8A8.

---

Published as a conference paper at ICLR 2025
10
25
Bit-width (W/A)
Memory Opt.
Latency Opt.
20
1.71x
1.58x
1.98X
Litencyyo
15
16/16
1.00x
1.00x
8/8 (naive)
1.99x
1.74x
1.0
(ours)
x
x
4/8 (ours)
2.42x
1,38x
05
0.
Isurat
Icurl
Figure 7: The illustration of ViDIT-Q's hardware resource savings. The table and figures present
memory savings and end-to-end latency speedup ViDiT-Q and naive quantization scheme.
Ablation studies of ViDiT-Q techniques. The comparison of OpenSORA W4A8 perfor- mance by gradually incorporating ViDiT-Q techniques.
Table 2:
Bit-width
CLIPSIM
CLiP-Temp
VQA-
VQA- Technical
A
Quant Parama
Channel Ralance
Mixed Precision
(WAA)
Aesheticc
Score.
16/16
0.180
0.998
64.198
51.904
Static & Tensor-wise
4/3
0.201
0.997
0.178
0.086
0.603
Dynamic Token-wise
418
0.196
0.998
32.217
10.994
0.109
Dynamic Token wise
Scaling based
4/8
0.191
0.999
31.963
22.847
0.415
Dynamic Token-wise
Rotation-bust based Static-Dynamis Static Serii--Dynmmia Static-Dyname
418
0.181
0.999
47.356
33.128
0.326
Dynamic Token-wise
0.181
0.999
60.216
42.257
0.151
Dynamic Token-wise
MSF-hased Metri Decoupled
4/8
0.179
0.999
53.338
38.729
0,258
Dynamic Token-wise
4/8
0.199
0.999
60.616
49.383
0.334
5.4 ABLATION STUDIES
We present ablation studies that gradually incorporate ViDiT-Q's techniques for W4A8 quantization, as shown Tab. 2
Effectiveness fine-grained grouping and dynamic quantization param- eter: Replacing static, tensor-wise quantization parameters with dynamic, token-wise ones signif- icantly improves generation, transforming it from near failure (with VQA scores close to zero) to producing readable content. Effectiveness of static-dynamic channel balancing: The scaling and rotation-based channel balancing technique results in notable quality degradation, whereas static- dynamic balancing improves generation quality a level comparable to FP. Effectiveness of mixed precision: Applying metrie-decoupled mixed precision further enhances generation quality, while MSE-based mixed precision negatively impacts performance.
Static Dynamic Balanced (static scaling only)
Static Dynamic
Original Weight locoherance: 40.75
Quarot
Channel Balanced
Incotorence:
62
40
Figure 8: The channel distribution of blockImll..fll weights with different channel balancing techniques. Directly adopting Quarot still results in relatively large incoherence. However, imple- menting static-dynamic channel balance reduces incoherence acceptable level..
CONCLUSION AND LIMITATIONS
We design ViDiT-Q, a quantization method that addresses unique challenges of DiTs. It achieves W4A8 quantization with minimal performance degradation for popular video and image generation models. We further implement CUDA kernels to achieve 2-2.5x memory savings, and 1.4-1.7x latency speedup. Despite achieving good performance, the mixed precision design is still worth polishing, and lower activation bit-width is essential for fully utilizing the acceleration potential of b-t weight. We aim to address these issues and further improve ViDiT-Q.
10

---

Published as a conference paper at ICLR 2025
REPRODUCIBILITY STATEMENT
All findings presented in this paper are fully reproducible. We have provided anonymized code in the supplementary materials, along with the videos shown in the figures. Detailed information about our experiments, including hyperparameters, training protocols, and evaluation methods, is available in the Experiments section. We are confident that, with the provided resources, readers will be able to reproduce all of the results presented.
ACKNOWLEDGEMENT
This work supported by National Natural Science Foundation of China (No. 62325405, 62104128, U19B2019, B21310 61832007, 62204164), Tsinghua EE Xilinx Al Research Fund, and Beijing National Research Center for Information Science and Technology (BNRist). We thank for all the support from InfinigenceAl.
REFERENCES
Saleh Ashkboos, Amirkeivan Mohtashami, Maximilian L Croci, Bo Li, Martin Jaggi, Dan Alistarh, Torsten Hoefler, and James Hensman. Quarot: Outlier-free 4-bit inference rotated Ilms. arXiv preprint arXiv:2404.00456 2024.
Fan Bao, Shen Nie, Kaiwen Xue, Yue Cao, Chongxuan Li, Hang Su, and Jun Zhu. All are worth words: A vit backbone for diffusion models. In CVPR, 2023.
Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis, Align your latents: High-resolution video synthesis with latent diffusion mod- els. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22563-22575, 2023.
Jerry Chee, Yaohui Cai, Volodymyr Kuleshov, and Christopher M De Sa. Quip: 2-bit quantization of large language models with guarantees. Advances in Neural Information Processing Systems, 36, 2024.
Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang. James Kwok, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart-a: Fast training of diffusion transformer for photorealistic text-to-image synthesis, 2023.
Lei Chen, Yuan Meng, Chen Tang, Xinzhu Ma, Jingyan Jiang, Xin Wang. Zhi Wang, and Wenwu Zhu. Q-dit: Accurate post-training quantization for diffusion transformers. arXiv preprint arXiv:2406.17344 2024.
Patrick Esser, Johnathan Chiu, Parmida Atighehchian, Jonathan Granskog, and Anastasis Germani- dis. Structure and content-guide guiden video synthesis with diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7346 7356, 2023.
Yuwei Guo, Ceyuan Yang, Anyi Rao, Yaohui Wang, Yu Qiao, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized -ext-to-image diffusion models without specific tuning. arXiv preprint 00Xiv:2307.047255 2023.
Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Bras, and Choi Yejin. Clipscore: A reference- free evaluation metric for image captioning. pp. 7514-7528, 01 2021. doi: 10. 10.18663/v1/2021. emnlp-mane 595.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by two time-scale update rule converge local nash equilibrium. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS' 17, pp. 6629 6629-6640, Red Hook, NY, USA. 2017. Curran Associates Inc ISBN 97815
Jonathan Ho. Classifier-free diffusion guidance. ArXiv, abs/2207. 2022. URL https 7/api semant wachoocchhoooooo wtt.hholl....g///rpptttttt

---

Published as a conference paper at ICLR 2025
Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P Kingma, Ben Poole, Mohammad Norouzi, David J Fleet, et al. Imagen video: High definition video generation with diffusion models. arXiv preprint arXiv:2210.025003, 2022a.
Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David Fleet. Vidco diffusion models. Advances in Neural Information Processing Systems, 35:8633- 8646, 2022h.
HPC-AL Open-Sora. httpss//gggt..u....///o/aaaeeeee
2024.
Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianx- ing Wu, Qingyang Jin, Nattapol Champaisit, et al. Vbench: Comprchensive benchmark suite for video generative models. arXiv preprint arXiv:2311.17982 2023.
Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko. Quantization and training of neural networks for efficient pnteger-aiitmmetic-onn inference. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 2704-2713, 2018.
Xiuyu Li, Yijiang Liu, Long Lian, Huanrui Yang, Zhen Dong, Daniel Kang, Shanghang Zhang, and Kurt Keutzer. Q-diffusion: Quantizing diffusion models. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 17535-17545 2023.
Yuhang Li, Ruihao Gong, Xu Tan, Yang Yang. Peng Hu, Qi Zhang, Fengwei Yu, Wei Wang, and Shi Gu. Breeq: Pushing the limit of post-training quantization by block reconstruc-
tion. ArXiv, abs/2102.05426, 2021. URL https: /ttp:///api...e www.estteatnishhhccceee
Yujun Lin, Haotian Tang, Shang Yang. Zhekai Zhang, Guangxuan Xiao, Chuang Gan, and Song Han. Qserve: W4a8kv4 quantization and system co-design for efficient Ilm serving. arXiv preprint arXiv:2405.044 04532, 2024.
Enshu Liu, Xuefei Ning. Zinan Lin, Huazhong Yang. and Yu Wang. Oms-dpm: Optimizing the model schedule for diffusion probabilistic models, 2023a. URL 2306. 08860)
Enshu Liu, Xuefei Ning, Huazhong Yang, and Yu Wang. unified sampling framework for solver searching of diffusion probabilistic models, 2023b. URLettpp:://rr....orr//bbbb/
07243
Yaofang Liu, Xiaodong Cun, Xuebo Liu, Xintao Wang, Yong Zhang, Haoxin Chen, Yang Liu, Tieyong Zeng, Raymond Chan, and Ying Shan. Evalcrafter: Benchmarking and evaluating large video generation models. 2023c.
Yaofang Liu, Xiaodong Cun, Xuebo Liu, Xintao Wang, Yong Zhang, Haoxin Chen, Yang Liu, Tieyong Zeng, Raymond Chan, and Ying Shan. Evalcrafter: Benchmarking and evaluating large video generation models. arXiv preprint arXiv:2310. 14400 2023d.
Zechun Liu, Changsheng Zhao, Igor Fedorov, Bilge Soran, Dhruv Choudhary, Raghuraman Krish- namoorthi, Vikas Chandra, Yuandong Tian, and Tijmen Blankevoort. Spinquant-IIm quantization with learned rotations, arXiv preprini ar 2405 /6406, 2024.
Zhenhua Liu, Yunhe Wang, Kai Han, Wei Zhang, Siwei Ma, and Wen Gao. Post-training quanti- zation for vision transformer. Advances in Neural Information Processing Systems, 34:28092- 28103, 2021.
Xin Ma, Yaohui Wang, Gengyun Jia, Xinyuan Chen, Ziwei Liu, Yuan-Fang Li, Cunjian Chen, and Yu Qiao. Latte: Latent diffusion transformer for video generation. arXiv preprint arXiv:2401.0888 03048, 2024.
Xinyin Ma, Gongfan Fang, and Xinchao Wang. Deepcache: Accelerating diffusion models for free, 2023. URL htt.hhttss//// pttpps:///rxiiin wwtp:/wwwwwg/////////eeoee 00858
12

---

Published as a conference paper at ICLR 2025
Markus Nagel, Rana Ali Amjad, Mart Van Baalen, Christos Louizos, and Tijmen Blankevoort. Up or down? adaptive rounding for post-training quantization. In International Conference on Machine Learning. pp. 7197-7206. PMLR, 2020.
Markus Nagel, Marios Fournarakis, Rana Ali Amjad, Yelysei Bondarenko, Mart Van Baalen, and Tijmen Blankevoort. A white paper on neural network quantization. arXiv preprini wrXiv2210608829: 2021.
Zanlin Ni, Yulin Wang, Renping Zhou, Yizeng Han, Jiayi Guo, Zhiyuan Liu, Yuan Yao, and Gao Huang. Enat: Rethinking spatial-temporal interactions in token-based image synthesis, 2024.
URL httph:t/tts::ttt org/ abs http:/bb//2411.000 05959
NVIDIA, Nsight //css
html
OpenAl. Video generation models as world simulators.
2024.
William Peebles and Saining Xie. Scalable diffusion models with transformers, 2023.
Yifan Pu, Zhuofan Xia, Jiayi Guo, Dongchen Han, Qixiu Li, Duo Li, Yuhui Yuan, Ji Li, Yizeng Han, Shiji Song. Gao Huang, and Xiu Li. Efficient diffusion transformer with step-wise dynamic attention mediators, 2024. httpstts: /aaxivvvvvv.
PyTorch. PyTorch Memory Management, 2023.
URL https://pytorcch.or//o//
Sttablennoten cuda htm #memo ryatmoor-mmmaee wwaarrrrrmmmammmmmmmm
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from natural language supervision. In International conference machine learning, pp. 8748-8763. PMLR, 2021.
Weiming Ren, Harry Yang, Ge Zhang, Cong Wei, Xinrun Du Stephen Huang, and Wenhu Chen. Consisti2v: Enhancing visual consistency for image-to-video generation. arXiv preprint arXiv:2402.04324, 2024.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEF/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomed- ical image segmentation. In Medical image computing and computer-assisted intervention- MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceed- ings, part III 18, pp. 234-241. Springer, 2015.
Chitwan Saharia, Jonathan Ho, William Chan, Tim Salimans, David Fleet, and Mohammad Norouzi, Image super-resolution via iterative refinement, 2021.
abs/2104 07636
Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. NIPS' 16, pp. 22322224222222 2242, Red Hook, NY, USA, 2016. Curran Associates Inc. ISBN 108381988
Yuzhang Shang, Zhihang Yuan, Bin Xie, Bingzhe Wu, and Yan Yan. Post-training quantization on difffusinn models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1972-1981, 2023.
Uriel Singer, Adam Polyak, Thomas Hayes, Xiaoyue Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, Devi Parikh, Sonal Gupta, and Yaniv Taigman. Make-a- video: Text-to-video generation without text-video data. ArXiv, abs/2209. 14792, 2022.
Junhyuk So, Jungwon Lee, Dachyun Ahn, Hyungjun Kim, and Eunbyeok Park. Temporal dynamic quantization for diffusion models. Advances in Neural Information Processing Systems, 36, 2024.
13

---

Published as a conference paper at ICLR 2025
Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. Ucf101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint 12Xiv:1212.0402, 2012.
Yang Sui, Yanyu Li, Anil Kag, Yerlan Idelbayev, Junli Cao, Ju Hu, Dhritiman Sagar, Bo Yuan, Sergey Tulyakov, and Jian Ren. Bitsfusion: 1.99 bits weight quantization of diffusion model. arXiv preprint arXiv:2406.0433
2024.
Siao Tang, Xin Wang, Hong Chen, Chaoyu Guan, Zewen Wu, Yansong Tang, and Wenwu Zhu. Post-training quantization with progressive calibration and activation relaxing for text-to-image diffusion models. arXiv preprint wwxiv:2311.06322. 2023.
Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. In Computer Vision-ECCCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part II 16, pp. 402-419. Springer, 2020.
Thomas Unterthiner, Sjoerd van Steenkiste, Karol Kurach, Raphaël Marinier, Marcin Michalski, and Sylvain Gelly. Fvd: new metric for video generation. 2019.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Eukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural informa- tion processing systems, 30, 2017.
Chenfei Wu, Lun Huang. Qianxi Zhang, Binyang Li, Lei Ji, Fan Yang, Guillermo Sapiro, and Nan Duan. Godiva: Generating open-domain videos from natural descriptions. arXiv preprint arXiv:2104. 14806, 2021.
Haoning Wu, Erli Zhang, Liang Liao, Chaofeng Chen, Jingwen Hou, Annan Wang, Wenxiu Sun, Qiong Yan, and Weisi Lin. Exploring video quality assessment on user generated contents from aesthetic and technical perspectives. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 20144 201544, 2023a.
Junyi Wu, Haoxuan Wang, Yuzhang Shang, Mubarak Shah, and Yan Yan. Ptq4dit: Post-training quantization for diffusion transformers. arXiv preprint arXiv:2405.16000 2024.
Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text-to- 2023b.
image synthesis. arXiv preprint arXiv:2306.099::
Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. Smoothquant: Accurate and efficient post-training quantization for large language models, 2024.
Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagereward: Learning and evaluating human preferences for text-to-image generation, 2023.
Yuewei Yang, Xiaoliang Dai, Jialiang Wang, Peizhao Zhang, and Hongbo Zhang. Efficient quanti- zation strategies for latent diffusion models. arXiv preprint arXiv:2312.05431, 2023.
Zhuoyi Yang, Heyang Jiang, Wenyi Hong. Jiayan Teng, Wendi Zheng, Yuxiao Dong, Ming Ding, and Jie Tang. Inf-dit: Upsampling any-resolution image with memory-efficiett diffusion trans- former, 2024. URL /tttt:t org/ abs////////// /200/ 04312
Zhewei Yao, Reza Yazdani Aminabadi, Minjia Zhang, Xiaoxia Wu, Conglong Li, and Yuxiong He. Zeroquant: Efficient and affordable post-training quantization for large-scale transformers. Advances in Neural Information Processing Systems, 11-27168-27888 2022.
Zhihang Yuan, Yuzhang Shang, Hanling Zhang, Tongcheng Fang, Rui Xie, Bingxin Xu, Yan Yan, Shengen Yan, Guohao Dai, and Yu Wang. E-car: Efficient continuous autoregressive image gen- cration via multistage modeling, 2024a.
Zhihang Yuan, Hanling Zhang, Lu Pu, Xuefei Ning, Linfeng Zhang, Tianchen Zhao, Shengen Yan, Guohao Dai, and Yu Wang. DiTFastatt: Attention compression for diffusion transformer models. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024b. URL https::/openrest net wtw.:neeeoooommddiiiiooooore
14

---

Published as a conference paper at ICLR 2025
Evelyn Zhang, Bang Xiao, Jiayi Tang, Qianli Ma, Chang Zou, Xuefei Ning, Xuming Hu, and Lin- feng Zhang. Token pruning for caching better: times acceleration on stable diffusion for free, 2024a. https:// wttp/aaxivvv...///iv//.a./aa.iva org/ abs / 2000055111100 00375
Evelyn Zhang, Jiayi Tang, Xuefei Ning, and Linfeng Zhang. Training-free and serddwaarr-frindlly acceleration for diffusion models via similarity-bane -based token pruning. In Proceedings of the AAAI Conference on Artificial Intelligence, 2025a.
Jintao Zhang, Haofeng Huang, Pengle Zhang, Jia Wei, Jun Zhu, and Jianfei Chen. Sageattention2:
Efficient attention with thorough outlier smoothing and per-thread int4 quantization, 2024b. URL 21t.:/211......
tuttps://ariiv.
Jintao Zhang, Haofeng Huang, Pengle Zhang, Jun Zhu, Jianfei Chen, et al. Sageattention: Accurate 8-bit attention for plug-and-play inference acceleration. arXiv preprint arXiv:2410.02367, 20244
Jintao Zhang, Chendong Xiang, Haofeng Huang, Haocheng Xi, Jia Wei, Jun Zhu, and Jianfei Chen. Spargeattn: Accurate sparse attention accelerating any model inference, 2025b.
Tianchen Zhao, Xuefei Ning, Tongcheng Fang, Enshu Liu, Guyue Huang, Zinan Lin, Shengen Yan, Guohao Dai, and Yu Wang. Mixdq: Memory-efficient few-step text-to-image diffusion models with metric-decoupled mixed precision quantization. arXiv preprint 47X1v:2405.17873, 2024a.
Yilong Zhao, Chien- Yu Lin, Kan Zhu, Zihao Ye, Lequn Chen, Size Zheng, Luis Ceze, Arvind Krishnamurthy, Tianqi Chen, and Baris Kasikci. Atom: Low-bit quantization for efficient and accurate IIm serving. Proceedings of Machine Learning and Systems, 6: 1969-90, 2024b.
Chang Zou, Evelyn Zhang, Runlin Guo, Haohang Xu, Conghui He, Xuming Hu, and Linfeng Zhang. Accelerating diffusion transformers with dual feature caching, 2024. URL https: : // arrvv.. org/abs/2415 18911
15

---

Published as a conference paper at ICLR 2025
A ADDITIONAL EXPERIMENTAL DETAILS
A.1 MOTIVATION FOR QUANTIZING LINEAR LAYERS ONLY
In Sec. we mention that the linear layers accounts for the most of the computation. So we focus on quantizing the linear layers and leave the attention computation unquantized. We elaborate the reason for this focus here. In Fig. we visualize the detailed latency breakdown for an STDiT model block. The 'attention computation includes the matrix multiplication for query and key embedding to generate the attention map, and the multiplication of the attention map with the value embedding. The QKV linear mapping and the projection after attention aggregation are not included, as these are linear layers that can be quantized. As shown, when utiliziig FlashAttention, the latency cost of attention computation accounts for only 14.3% of the overall latency. Additionally, FlashAttention minimizes the activation memory usage for storing the attention map. Therefore, we focus on the primary cost: the linear layers. We quantize all linear layers except for the "t embedding". "y embedding" and "final layer". they appear the start or end of the model, and have smaller channel sizes. They account for only negligible amount of computation (< 1/1000 overall latency), therefore maintain them FP16.
CLDA ri DX00 40000 ATTERA
35.0% serdd
cmperu. d1 14.%% no
15.0%
FlashAttention: 14.3%
terel grops Videe
0.1% Memeey
302mm
5.50
WYTx
Figure 9: The latency comparison of linear layers and attention computation. When utilizing FlashAttention, the attention computation only takes up small portion (14.3%) of the latency.
A.2 IMPLEMENTION DETAILS FOR BASELINE METHODS
O-Diffuuion: We follow the official open-sourced code and collect timestep-wisg activation as cal- ibration data, and conduct optimization for the scaling factor "delta", and the AdaRound parameter "alpha".. PTQ4DiT: We reimplement the "p-guided" saliency correction. Following the original paper, we adopt static tensor-wise activation parameters. Q-DīT: Following the original paper, we adopt channel-wise quantization grouping for both the weight and activation. The group size is de- termined with the evolutionary algorithm search. SmoothQuant: We adopt the same formulation of channel scaling mask and leverage grid search to determine the optimal QuaRot: We adopt the same Hadamard matrix transformation as the original paper. Specifically, since we donot quantize the attention QK matrix multiplication, only apply rotation to linear layers.
B DETAILED DESCRIPTION OF EVALUATION METRICS
B. BENCHMARK SUITE
Following VBench Huang et (2023), our benchmark suite encompasses three key dimensions. (1) Frame-wise Quality assesses the quality of each individual frame without taking temporal qual- ity into concern.
Aesthetic Quality evaluates the artistic and beauty value perceived by humans towards each video frame.
Imaging Quality assesses distortion (e.g., over-exposure, noise) presented in the generated frames
(2) Temporal Quality assesses the cross-frame temporal consistency and dynamics.
16

---

Published as a conference paper at ICLR 2025
Subject Consistency assesses whether appearance of subjects in the video remain consis- tent throughout the whole video.
Background Consistency evaluates the temporal consistency of the background scenes.
Motion Smoothness evaluates whether the motion in the generated video is smooth and follows the physical law of the real world.
Dynamic Degree evaluates the degree of dynamics by calculating average optical flow on each video frame.
(3) Semantics evaluates the video's adherence to the text prompt given by the user. consistency.
Scene Consistency evaluates whether the video is consistent with the intended scene de- scribed by the text prompt.
Overall Consistency reflects both semantics and style consistency of the video.
We utilize three prompt sets provided by official github repository of VBench. We generate one video for each prompt for evaluation.
subject seaeeet_cosiisttee consistency.txt: include 72 prompts, used to evaluate subject consistency, dy- namic degree and motion smoothness.
overall consistency.txt: include 93 prompts, used to evaluate overall consistency, aesthetic quality and imaging quality.
scene.txt: include 86 prompts, used to evaluate scene and background consistency.
B.2 SELECTED METRICS
FVD and FVD-FP16: FVD measures the similarity between the distributions of features extracted from real and generated videos. We employ one randomly selected video per label from the UCF- 101 dataset (101 videos in total) as the reference ground-truth videos for FVD evaluation. We follow Blattmann et al. (023) to use pretrained I3D model to extract features from the videos. Lower FVD scores indicate higher quality and more realistic video generation. However, due to relatively smaller video size (e.g. 101 videos in our case), employing FVD to evaluate video generation models faces several limitations. Small sample size cannot adequately represent either the diversity of entire dataset or the complexity and nuances of video generation, leading to inaccurate and unstable results. To mitigate limitations above, we propose an enhanced metric, FVD-FP16, for assessing the semantic loss in videos generated by quantized models relative to those produced by pre-quantized models. Specifically, we utilize 101 videos generated by the FP16 model as ground-truth reference videos. The FVD-FP16 has significantly higher correlation with human perception.
CLIPSIM and CLIP-temp: The CLIPSIM and CLIP-temp metrics are computed using implemen- tation from EvalCrafter Liu al. 2023c). For CLIPSIM, We the CLIP-VIT-B/32 model Rad- ford et al. 2222) to compute the image-text CLIP similarity for all frames in the generated videos and report the averaged results. The metrie quantify the discrepancy between input text prompts and generated videos. For CLIP-temp, we use the same model to compute the CLIP similarity of each two consecutive frames of the generated videos and then get the averages on each frames. The metric indicates semantics consistency of generated videos.
We employ the Dover Wu al, (2023a) method to assess generated video quality in terms of aesthetics and technicality. The technical rating( VQA-T) measures common distortions like noise, blur and over-exposure. The aesthetic rating (AQA))) reflects aesthetic aspects such as the layout, the richness and harmony of colors, the photo-realism, naturalness, and artistic quality of the frames.
DOVER's VQA:
We employ flow score proposed by Liu ct al. (203cc measure the general motion information of the video. we use RAFT Teed Deng 2020). to extract the dense flows the video in every two frames. Then, calculate the average flow these frames obtain the average flow score of each generated video.
Flow Score:
Temporal Flickering:
We utilize the temporal flickering score provided by VBench Huang et al. (2023) to measure temporal consistency at local and high-frequency details of generated videos. We calculate the average MAE(mean absolute difference) value between each frame.
17

---

Published as a conference paper at ICLR 2025
C DETAILED ANALYSIS OF EXPERIMENTAL RESULTS
In this section, we present more detailed analysis of the experimental results Sec.
FP16
Motion Smooth.
Naive PTQ:WBAB
VIDIT-Q:WBAB
VIDIT-Q:W6A6
VIDIT-Q:W4A8
1-ADynamic Degree
Aesthetic Quality
1.0
0,8
0.6
0.2
BG. Consist.
Imaging Quality
Subject Consist.
Overall Consist.
Scene Consist.
Figure 10: The radar chart corresponding the data presented in Table] 5.2 ViDIT- Q has superior performance on VBench compared with the naive PTQ.
5.2
C.1 TEXT-TO- VIDEO PERFORMANCE ON VBENCH
VBench is comprehensive benchmark suite for video generation models, covering wide range of dimensions, such as motion smoothness and subject consistency. The metric values of ViDÍT- Q's performance on VBench is presented in Tab. [ Sec. 5 We visualize the Radar plot of the VBench performance in Fig. TO the metric values are normalized by the maximum value in each diemsnion. It's clearly illustrated that ViDiT-Q achieves similar performance with FP16 for all bit- widths (W8A8, W6A6 mixed precision, W4A8 mixed precision), outperforming the Naive PTQ W8A8. We further analyze the generated video's performance from three aspects as follows:
Dynamic degree indicates the range of motion in the video, higher dynamic degree denotes more dynamic movement in the video. Lower dynamic degree denotes that the video barely moves, resembling static image. Normally, higher dynamic degree is favored. However, in the quantization scenario, we discover that quantization often causes the generated videos to jitter and tremble. It is not favorable but results in notable dynamic degree value increase. In
Dynamic Degree:
18

---

Published as a conference paper at ICLR 2025
experimental setting, too high too low dynamic degree degradation. Therefore, in the radar plot, using FP16 generated videos as the ground -ruth reference, we use the (fo - {FP) to denote "relative dynamic degree changes from FP generated videos", and use 1- 1--ff--rr frr) as dynamic degree scoring in the radar plot. As illustrated Fig. TO dynamic degree dimension, Naive PTQ WSA8's scoring (< 0.8) is notably lower than ViDiT-Q results. The video examples in Fig. [1] supports this finding. In Fig. IIc the navive PTQ W8A8 generated buildings have jittering and glitches, and changes significantly across frames (ref the supplementary for the video). In contrast, both the FP16 and ViDiT-Q W8A8 generated buildings moves acutely.
The consistency denotes whether some object remains consistent (does not disappear, change significantly) across frames. Vbench evaluates consistency from the subject, scene, back- ground, and overall level. From the Radar plot, we witness ViDiT-Q also notably outperforms naive PTQ, especially in the "scene consistency" dimension (< 0.8). As seen in the aforementioned video example Fig. TTc. the buildings (act as the "scene") changes significantly across frames. vio- lates the scene consistency and lead to lower scoring, Also, as presented in Fig, [2cl the generated bear's ear does not exist in earlier frames, and suddenly appears. This also reflects the degradation of subject consistency.
Consistency:
Quality: VBench evaluates the quality from both the aesthetic (composition and color), and imag- ing quality (clarity, exposure) dimension. Fig. [3 shows the example of Naive PTQ W8A8's 's quality degradation. The color notably turns blue, and the mountain on the left is blurred. Similar color shifting degradation is also witnessed in Fig. 12c
19

---

Published as a conference paper at ICLR 2025
(a) FP16
(b) ViDiT-Q: W8AS
(c) Naive PTQ: W8A8
Figure 11: The qualitative results VBench about the ViDiT-Q's ability to maintain the dy-
namic degree.
20

---

Published as a conference paper at ICLR 2025
(a) FP16
ViDiT-Q: W8A8
(c) Naive PTQ: W8A8
Figure 12: The qualitative results on VBech about the ViDIT-Q's ability to maintain the con- sistency.
21

---

Published as a conference paper at ICLR 2025
(a) FP16
(b) ViDiT-Q: W8A8
(c) Naive PTQ: W8A8
Figure 13: The qualitative results VBench about the ViDiT-Q's ability to maintain the image quality.
22

---

Published as a conference paper at ICLR 2025
C.2 TEXT-TO-IMAGE GENERATION ON COCO
We present more qualitative results of generated images by baseline quantization and ViDiT-Q quan- tization in Fig. T4 As shown, the Naive PTQ's generated images are highly blurred. While the W8A8 images depict outlines of objects, the W4A8 images generate nearly pure noises. In con- trast, ViDiT-Q generates images nearly identical to the FPI6 ones, preserving both visual quality and text-image alignment.
FP16
ViDi--Q W4A8
Q-DIT W4A8
PTQ4DIT W4A8
A building wall and pair of doors that are open. along with vases of flowers on the outside of the building.
A baby elephant panting on a white canvas.
closeup of apple in the foreground with three oranges in the background.
Figure 14: Qualitative results of text-to-image generation
23

---

Published as a conference paper at ICLR 2025
C.3 VISUALIZATION OF ABLATION STUDIES
We present the generated videos for the ablation studies in Tab. 2 As seen in Fig. 15 video quality improves from blank images to similar to the FPI6 baseline. For the challenging W4A8 quan- tization, the baseline method generates blank images. After adding dynamic quantization, some meaningful background (deep ocean) appears, but the main object (turtle) is still missing. Channel balancing reduces color deviation (from dark blue to green-blue), but the main object remains un- recognizable and changes significantly across frames (please refer to the supplementary materials for the video). The static-dynamic channel balancing improves the consistency of the main object, notable degradation still observed compared to the FPI6 video. Finally, with mixed precision, a similar generation quality to the FP16 baseline is achieved."
Generated Videos Example of Ablation Studies: STDIT W4A8
"A avess underwater jesturing turtie right site the frame. The conalred! recnssy with ifc, is vtible the buzlgroudd Krik Abrair and coliffil bockshop the comte's jouneyy Several rnd for doning proudd the tumis, odd pense of assoomeet and dytomeee me scece. The adeo shor from slytty olested ongle, pronedly compoelennee siew of surtes surouadigg The curral ople- the wato is calre and proecfal copturing the boouly and burqualyy of the andereeeer norks
thesugn reee The turtle with greanteh- shel main
neitiiing gresdjaly tnwarts the
Baseline
Dynamic Quantization
Scaling Channel
Balancing
Channel Balancing
Static-Dynamic
Mixed Precision
FP16
Figure 15: Visualization of generated videos of ablation studies.
D ADDITIONAL EXPERIMENTAL RESULTS
D.1 PERFORMANCE oF VIDIT-Q ON UCF-101 DATASET
In this section, apply ViDiT-Q to both OpenSora HPC-AI 2024) Model and Latte Ma et al. (204) and evaluate them on UCF-101 [Soomro et al. 2012) dataset.
Experimental Settings: Similar to the settings mentioned in Sec. Sec. 5.1 We select multi-aspects metrics for more comprehensive evaluation. The commonly adopted FVD Unterthiner et al. (2019) also provided. Specifically, due the lack of ground-truth videos for prompt-only datasets, inspired by Tang et al. (2023). we also report FVD-FPIG which chooses the FP16 generated video as ground-truth. The above metrics are evaluated on 101 prompts for each class) for UCF- 101. We adopt the class-conditioned Latte model trained on UCF-101 and use the 20-steps DDIM solver with CFG scale of 7.0 for it.
Experimental Results: Similar to the results on VBench and OpenSORA prompt sets, for both the OpenSORA and Latte, the bascline quantization methods (Naive PTQ and Q-Diffusion) incur no- table performance degradation under W8A8, and fails under W4A8. While SmoothQuant channel balance technique could achieve good performance under W8A8, still witnesses notable degra- dation under W4A8. It is also worth noting that the FVD metrics are noisy when the number of videos are relatively small, and the "FVD-FPI6" metric could work as an effective alternative for measuring quantization's 's effect.
D.2 PERFORMANCE OF VIDIT-Q FOR SUPER RESOLUTION TASK.
ViDiT-Q addresses the core problem of reducing quantization error by analyzing the distribution of DiT', making it highly compatible and generalizable novel tasks that utilize DiTs. We have extended the application ViDiT-Q to the image super- resolutin task using the recent InfDiT model Yang et al. (2024). The statistics are presented in Tab. 4 and qualitative results in Fig. [6
As could seen, ViDiT-Q consistently maintains the performance Inf-DiT across all bitwidths. When quantizing the model under W4A8 with ViDiT-Q, the PSNR and SSIM scores of the quantized
24

---

Published as a conference paper at ICLR 2025
Table Performance of lext-to-video generation on UCF-101 Dataset.. The description of met- rics is provided in Sec. 5.1 unless specified with J, higher metric values denote better performance.
Bit-width (W/A)
VQA-
Flow
Temp
Model
Method
FVD())
EVD-FPI6L)
CLIPSIM
CLIP-T
Tochnical
(4)
Flick.
16/16
136.87
0.90
0.1996
0.9978
41.63
36.64
2.24
97.53
Naive PTQ
8/8
154,92
50.72
0.1993
0.9968
27.52
35,50
2.61
97.02
Q-Diffusion
8/8
144.77
74.97
0.1979
0.9964
32.88
44.42
2,50
96.71
SmoothQuant
878
109.24
48.78
0.1993
0.9971
39.19
52.64
2.53
97.21
STDiT
VIDIEQ
8/8
141.13
15.52
0. 1999
0.9978
43.59
55.36
2.32
97.45
Naive PTQ
4/8
544.34
637.02
0.1868
0.9982
0.16
0.13
1.61
99.90
SmoothQuant
4/8
122.51
96.25
0. 1960
0.9973
17.39
24.22
1.99
96.23
VIDIE-Q
4/8
136.54
77.43
0. 1.7978
0.9976
20:76
25.65
1.94
96.51
ViDITQ-MP
4/8
129.10
60.13
0.1995
0.9977
33.98
47.65
97.57
16/16
99.90
0.00
0.1970
0.9963
36,33
91.23
3.37
96.22
Naive PTQ
8/8
98.75
73.82
0.1981
0.9950
27.62
50.52
3.53
95.35
Latte
VIDITQ
8/8
110.96
20.83
0.1959
0.9962
30.26
80.32
3.14
95.95
Naive PTQ
4/8
183.52
239.08
0.1719
0.9929
5.62
0.41
66.06
65.14
VIDIT Q
4/8
95.04
79.11
0.1943
0.9971
21.76
32.17
2.84
95.57
FP16
ViDiT-Q (W8A8)
ViDiT-Q (W4A8)
Figure 16: Qualitative results of generated super resolution image for Inf-DiT with ViDIT-Q
quantization.
model still achieve good performance, similar to FP16. We have supplemented the implementation details as follows: We followed the settings used in the PixArt quantization experiments in our paper. For the super-resolution implementation and model evaluation, we adhered as closely possible to the original setup. We fixed the image degradation to bicubic interpolation with 4x downsampling and conducted the experiment on the DIV2K validation dataset. Additionally, unlike other super- resolution models, Inf-DiT performs non-overlapping patch division on the downsampled images during super-resolution. This means that the original image needs be divisible by the product of the super-resolution scale and the patch size. According to the original settings, the patch size set to 32. Therefore, we performed center cropping on the ground truth images to ensure image size is divisible by 128. Since the InfDiT official codebase Yang et al. (2024) did not provide detailed evaluation code, we implemented the SSIM and PSNR calculations based on the popular code repository from Saharia et al. (2021).
Table 4: Comparison of ViDIT-Q for Inf-DiT model for image super resolution.
SSIM InfDiT (ViDiT-Q W4A8)
Method
PSNR
InfDiT FP16
25.8015
0.7307
InfDiT (ViDiT-Q W8A8)
25.8628
0.7318
26.0139
0.7249
25

---

Published as a conference paper at ICLR 2025
D.3 EFFICIENCY IMPROVEMENT ON DIFFERENT HARDWARE DEVICES
We present hardware experiments on the RTX3090 and Jetson Orin Nano (a low-power embedded GPU with 7-10W power) in Fig. 17 The memory optimization for all platforms still achieves 2x reduction, and the latency speedup varies slightly. On the RTX3090, we achieve a 1.6x latency speedup, while on the Orin, we achieve approximately 1.82x speedup. This speedup could be further improved by tuning the tiling parameters in the CUDA code, as different platforms have diverse optimal tiling parameter setups due to varying hardware resources.
Normalized Latency of ViDiT-Q INT8
1.82x
Jessnn Nano
RTX3080
1.71x
A100
NAND
Figure 17: Comparison of ViDiT-Q's efficiency improvement on different devices.
D.4 PERFORMANCE OF VIDIT-Q UNDER LOWER BITWIDTH.
Analysis of Lower Bitwidth from Hardware Perspective: For GPUs, both the activation and weight need to be quantized into 4-bits to leverage efficient INT4 computation. In the current W4A8 implementation, the weights are quantized into 4-bits to save model size and memory cost, but they must be upcasted to 8-bit for computation with 8-bit activation. Therefore, pursuing lower bitwidths such as INT4 to fully utilize the potential of INT4 computation is definitely valid, and W2A8 could further reduce the model size and memory consumption. The W4A2 may require customized operator support, which is not currently supported by GPUs, and remains to be explored future direction.
We present W4A8 in the main paper as relatively "conservative" setting to ensure negligible performance degradation. ViDiT-Q remains capable of generating images of good quality with mixed-precision plan. We have supplemented the experiments on text-to-image generation for the Pixart-Sigma model. The statistical and qualitative examples are presented in Tab. and Fig. a8as as follows. The "ViDiT-Q W4A4-MP" plan employs mixed precision without careful tuning, assign- ing 66.7% of the linear layers as W4A4 and the remaining rest as W8A8, resulting in an average bitwidth of approximately W5A5. The "ViDiT-Q W2A8-MP" plan assigns around 50% of the linear layers as W2A8 and the rest as W8A8, achieving an average bitwidth of approximately W5A8. 2- weight quantization is particularly challenging and may require quantization -ware training to pre- serve performance. Despite these challenges, ViDiT-Q performs well under lower bitwidth settings (W4A4 & W2A8), achieving comparable or even higher metric values compared to Q-DiT W8A8. As illustrated in the Fig. [8] even under "aggressive compression" settings, the generated images closely resemble those produced by FP models.
D.5 COMPARISON WITH GENERAL QUANTIZATION METHODS
conducted experiments by adding the AdaRound Nagel al. (2020) and BRECQ al. (2021) techniques as basclines for the OpenSORA model, the implementation details of quantization tech- niques set the same O-Diffusion, and the other implementation details are kept the same with the main paper. The results are presented in Tab.
As can be seen from the table, both the AdaRound and Brecq methods experience moderate per- formance drop compared to FP16, while ViDiT-Q achieves comparable results with the FP16 base- line. For the challenging W4A8 scenario, due the significant channel-wise variation that
26

---

Published as a conference paper at ICLR 2025
Table 5: Comparison of performance under lower bitwidths (W4A4, W2A8) for Pixart-Sigma text-to-image generation. The "ViDiT-Q W4A4 MP" stands for utilizing the mixed precision for W4A4 quantization.
Method (Bitwidth)
FID (L)
CLIP())
ImageReward(1)
FPI6
73.34
0.258
0.901
Q-DiT W8A8
73.60
0.256
0.854
ViDiT-Q W8A8
75.61
0,259
0.917
ViDiT-Q W4A8
74.33
0.257
0.887
ViDiT-Q W4A4-MP
74.56
0.257
0.861
ViDiT-Q W2A8-MP
75.32
0.256
0.843
ViDiT-Q (W4A4-MP)
ViDiT-Q (W2A8-MP)
FP16
Figure 18: Qualitative results of ViDiT-Q generated images under lower bitwidth.
AdaRound and Breeq are not designed to handle, they fail to produce meaningful content, resulting in near-zero VQA scores. This underscores the importance of specialized techniques to address the channel imbalance problem effectively.
D.6 COMBINATION WITH ATTENTION QUANTIZAITON METHOD
Recent attention quantization method SageAttention Zhang et al. (024c) could reduce the cost of attention computation in DiT' through quantizing the QK into bits. VIDiT-Q and Attenntn focus on different aspects of quantization. ViDiT-Q quantizes the linear layers. while SageAttention accelerates the attention QK matrix multiplication. Therefore, these methods can be seamlessly combined to achieve better speedup. We applied Stgentttnntion on top of the OpenSORA model as presented Tab. 7 Since the linear layers constitute the majority of the computational cost for the model (more than 80%, presented in Fig.9), further introducing StteAttention will not cause notable performance degradation but could moderately improve latency. We present the algorithm performance and hardware efficiency as follows:
E ADDITIONAL ANALYSIS
27

---

Published as a conference paper at ICLR 2025
Table 6: Comparison with general quantization methods.
Method
W/A
CLIPSIM
CLIp-Temp
VQA-A
VQA-T
A Flow Score
FP16
0.1797
0.9988
63.40
50.46
Adaround
8/8
0.1796
0.9983
52.90
29.84
0.2934
Brecq
8/8
0.1791
0.9983
48.27
31.98
0.3978
ViDiT-Q
8/8
0.1950
0.9991
60.70
54.64
0.0890
Adaround
4/8
0.9971
0.1648
0.272
0.151
0,4210
Brecq
4/8
0.1669
0.9963
0.085
0.077
0.4303
ViDiT-Q
4/8
0.1809
0.9989
60.62
49.38
0.1530
Table 7: Comparison of efficiency when combining with SageAttention.
Method
Bit-width
Memory Opt.
Latency Opt.
16/16
ViDiT-Q
8/8
1.99x 1.99x
1.71x
ViDiT-Q SageAttn
8/8
1.72x
E.1 COMPARISON WITH BASELINE QUANTIZATION METHODOLOGY DESIGN
We present detailed comparison with existing quantization methods follows:
Static and coarse-grained quantization parameters: Previous diffusion based methods primarily focused on CNN-based model quantization (PTQ4DM (Shang et al. (023).. Q-Diffusion Li et al. (2033) adopt static and coarse-graines quantization parameters. The recent Di'T-targeted quantiza- tion method PTQ4DiT Wu et al. (2024) follows this scheme. Static and cearse-graines quantiza- tion parameter determination assigns the same set of shared quantization parameters for activations across different tokens, timesteps, and conditions. As illustrated in Fig. 2 the large data variation across these dimensions incurs sigmificant quantization errors, leading to substantial performance degradation. We also collect their performances and present them in the table below. demon- strates that these baselines incur notable performance degradation under W8A8, and fails under W4A8.
Channel group-wise and dynamic quantization parameters: Q-DiT Chen et al. 2024) adopts
dynamic and channel group-wise quantization parameters, where group (64 to 128) of channels shares the same set of quantization parameters. This approach can handle channel-wise imbalance to some extent, and the "dynamic quantization" addresses variation across timesteps. However, all tokens still share the same set of quantization parameters, which is problematic for video generation models where loken-wise variation is significant. This method still faces severe quality degradation. Additionally, having different quantization parameters for different channels introduces challenges for efficient CUDA implementation. As seen from the table below, the Q-DiT incurs notable perfor- mance degradation under W8A8, and fails under W4A8.
Timestep-wise static quantization parameters: Previous diffusion-based quantization methods that adopt static quantization parameters often determine timestep-wise quantization parameters through careful calibration (PTQ4DM (Shang et al. (2023), Q-Diffusion Li et al. (2023) and gradient-based optimization (TDQ So et (2024)). The existing methods for handling timestep- wise variation have the following disadvantager compared to simple dynamic quantization: (1) The process of determining timestep-wise quantization parameters could be costly: This process requires iterating the model multiple times for calibrating timestep-wise activation quantization, and parameter tuning like TDQ incurs additional training costs. In contrast, adopting dynamic quan- tization requires no overhead for determining parameters for each timestep.
(2) The determined timestep-wise quantization parameters may face challenges in generalizing across different timesteps and solvers: Static timestep-wise quantization parameters need to be calibrated and de- termined offline. However, in practical usage, the diffusion model can be inferred with different numbers of timesteps and different solvers. Whether the timestep-wise quantization parameters
28

---

Published as a conference paper at ICLR 2025
generalize unseen solvers numbers of sampling steps remains challenging. (3) Dynamic quantization acts as the algorithm performance upper bound for solving timestep-wise quan- tization: The primary goal of timestep- wise quantization parameters is to reduce the quantization emoor caused by sharing the same set of quantization parameters across different timesteps. However, sharing of quantization parameters is involved, and such error is minimized. (4) Comparison of hardware overhead: A potential downside of dynamic quantization is the overhead involved in the online calculation of quantization parameters compared to static schemes that determine these parameters offline. As seen in Table, we have implemented efficient CUDA kernels, demonstrating that such overhead acceptable (1.74x 1.71x), while significantly improving algorithm performance by introducing dynamic quantization.
when employing dynamic quantization,
E.2 HARDWARE IMPLEMENTATION OF QUANTIZED LINEAR LAYERS.
We present the process of quantized GEMM (General Matrix to Matrix Multiplication) in linear layers. It involves the following steps. Given a weights matrix of shape [Cins Cout] and an ac- tivation matrix of shape [Nioken: Can], the matrix multiplication process Figure Fig. 19 As can be seen, the elements for each row of the weight matrix and each column of the activation matrix need be summed together. These values should share the same quantiza- tion parameter for efficient processing (so that the process of "integer computation with summation Wint-Xint" " and "multiplying by the quantization parameters Sw5z" " can be conducted separately). Therefore, the weight matrix should have "output-chamnel-iis" quantization parameters, and the activation matrix should have "channel-wise" quantization parameters.
be described in the
Weight
Output
WX sw (Wine =z (Xint 2x)
Gu
Qaserbadoes Panemeter
Neaker
Sw Wint Sa -S Wins Sw
x.r
Gn
Activation
Neoken
Figure 19:
Hardware implementation of quantized linear layer computation. The yellow bar stands for the quantization parameters s, z.
E.3 ANALYSIS OF METRIC DECOUPLED ANALYSIS
E.3.1 MOTIVATION FOR METRIC DECOUPLE
We verify the motivation of metric decoupled analysis through analyzing the layer type's correlation with the metric values. We compare and present each layer's sensitivity with respect to the metric value of each aspect in Fig. 20 The values are calculated follows: Firstly, we calculate the relative metric value difference (Metricsp- - Metrico)/ M/eericrp. Then, we perform Z-score standardization for all values to ensure their values are within range of [0.1]. Next, we apply softmax to make each layer type's effect on different metrics sum to one. As can be seen, each layer type shows significant correlation with a certain metric, which corresponds to the model design. For instance, cross-attention, which is conducted between pixel and text embeddings, affects text-video alignment, and temporal attention, which models aggregation across frames, primarily affects the temporal consistency-relatee metric FlowScore. Despite some layer type (SeldfAttn) sensitive to multiple metrics simultaneously. all layer types have a major focus on some aspects. The layer type that best fits the "sensitive multiple metrics" category is the self-attention layers, which still predominantly
29

---

Published as a conference paper at ICLR 2025
affect visual quality (0.6232) compared to other aspects (0.2364 and 0.1404). This still supports the motivation of adopting the metric decoupling approach.
T
CLIP
0.9157
0.1008
0.2364
0.306
Alignment
007
Visuai Quality
(Ridelity)
VQA
0.03022
0.E093
0.0230
0.03107
144
1
Time Consistency
FlowScore
-53
CrossAttn
FFN
SpatialAttn TempAttn
Figure 20: Visualization of correlation between layer types and metric values.
E.3.2 DETAILED PROCESS OF METRIC DECOUPLED ANALYSIS
The metric-decoupled sensitivity analysis introduced is generalized framework that can be adapted to different tasks and models. It consists of three steps.
(I) Firstly, for specific task (especially generative tasks), need to identify common aspects that are typically evaluated and select corresponding metrics for them. These metrics can be chosen based on popular evaluation settings. For example, for video generation. we might consider visual quality (VQA), temporal consistency (FlowScore), and text-video alignment (CLIPSIM); for image generation, we might consider fidelity (FII) and text-image alignment (CorP-Score).
(2) Next, we need to analyze how each part of the network affects these metric values. Given the large number layers, we can group similar layers and conduct group- wise evaluation (for instance, grouping layers by operator types such self-attention, cross-attention, and temporal attention). We can plot a heatmap (as seen Appendix Section) to discover the correlation between layers and certain aspects. This correlation is intrinsically linked because of how the model is designed (for example, cross-attentic strongly correlates with text-video alignment, as it designed to model the correlation between text and image embeddings, and similarly for temporal attention). In addition to video generation, we observe similar phenomena in eet--to-mmage generation with Pixart, where cross-attention layers primarily correlate with image-text alignment, and for self-attention and FFN layers with quality, Such findings could help us design "how to decouple the metrics".
(3) Finally, we need "decouple" the effect on each layer type to obtain relative importance for more accurate sensitivity. With the guidance of the correlation from the previous step, we measure the relative importance of certain metries (for instance, comparing all CrossAttn layers with their relative effect on CLIPSIM as sensitivity). The advantage of the "decouple" is two-folder. (1) It re- duces the vast search space jointly searching for each layer (by comparing only within groups). (2) It resolves the issue of different metrics' absolute value changes not being directly comparable. By carefully selecting the mixed precision plan for each group, it helps preserve multi-aspect metrics, avoiding the search from over-emphasizing certain aspects and causing failures in others. As pre- sented in the results below and the qualitative results in Appendix, incorporating metric-decoupled analysis instead of joint search allows the generated quality to preserve multi- aspect metrics simul- tancously.
E.3.3 COMPARISON DIFFERENT MIXED PRECISION SEARCH METHIOD.
We compare the "asedd "Muitiple metries joint search based" mixed precision sensitivity analysis with metric decoupled analysis to demonstrate the effectiveness of metric decoupled mixed precision. As could be seen in Tab. g both the "MSE-based" and "Multiple metrics joint search based" achieves even worse results than uniform W4A8. We conclude the potential reasons for their failures as follows: For "MSE-based" analysis, as we discussed in Sec. 4.3 the MSE error could not accurately depict the changes in multiple aspects for video generation task. For "Multiple metrics joint search based", firstly, balancing the effect of different metrics a non non-trivial problem. Due to the diverse forms of various metries, the absolute values of their changes cannot be directly
30

---

Published as a conference paper at ICLR 2025
Method
CLIPSIM
CLIP-Temp
VQA-A
VQA-T
Flow Score
Without Mixed Precision
0.181
0.999
60.216
42,257
0.151
MSE-based Search
0.179
0,999
53.335
38.729
0.258
Multiple Metrics Joint Search
0.179
0.999
51.256
35.412
0.279
Metric Decoupled Search
0.199
0,999
60.616
49.383
0.334
Table Comparison of different mixed precision analysis schemes under W4A8.
compared. Therefore, we introduce metric decoupled analysis to ensure that each layer's sensitivity measured with comparing with layers that have similar effects. To certain extent, this approach demonstrates the principles of "controlled variable analysis".
31


Reviews:
[{"review_id": "oqszlQVvSm", "reviewer": "Reviewer_2N42", "summary": "This paper introduces ViDiT-Q, a novel quantization method designed to address the unique challenges faced by diffusion transformers (DiTs) in text-to-image and video generation tasks. Large model sizes and multi-frame processing in video generation pose significant computational and memory costs, making efficient deployment on edge devices challenging. \nThe authors propose ViDiT-Q, a tailored quantization method for DiTs. This scheme effectively manages quantization errors by addressing specific challenges such as data distribution variations and channel imbalance. ViDiT-Q uses channel balancing to reduce color deviations and dynamic quantization to handle temporal variations in video sequences.", "questions": "**Please note that since I am not an expert in model quantization and do not have any background in this field, the weaknesses I provide may not be sufficient to reveal the shortcomings of the work.**\n\n1. While ViDiT-Q performs well at W8A8 and W4A8 quantization levels, there is a noticeable performance drop at lower activation bit-widths (such as W4A4 or W4A2). This indicates that the current mixed precision design has room for improvement, especially in fully leveraging the acceleration potential of 4-bit weights.\n\n2.ViDiT-Q introduces multiple quantization parameters (such as different $\\alpha$ values) to handle data variations across different timesteps. This complex parameter management increases the model's complexity.\n\n3. I believe the model can further introduce 8-bit Attention (SageAttention) to improve model efficiency, which has already been integrated into some video diffusion model libraries. I wonder if 8-bit attention mechanisms or some linear attention acceleration mechanisms can further improve your solution?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "ViDiT-Q reduces the incoherence of data distribution, thereby lowering quantization error, by combining scaling and rotation-based channel balancing methods. Specifically, the scaling method addresses the \"static\" channel imbalance at the initial denoising stage, while the rotation method handles the \"dynamic\" distribution variations over time.\n\nViDiT-Q uses channel balancing to reduce color deviations and dynamic quantization to handle temporal variations in video sequences.\n\nViDiT-Q is validated on various text-to-image and video generation models, demonstrating minimal degradation in visual quality and metrics even at W8A8 and W4A8 quantization levels.\n\nQualitative results show that ViDiT-Q maintains high image quality and text-image alignment, while naive PTQ methods produce highly blurred or noisy images.", "weaknesses": "**Please note that since I am not an expert in model quantization and do not have any background in this field, the weaknesses I provide may not be sufficient to reveal the shortcomings of the work.**\n\n1. While ViDiT-Q performs well at W8A8 and W4A8 quantization levels, there is a noticeable performance drop at lower activation bit-widths (such as W4A4 or W4A2). This indicates that the current mixed precision design has room for improvement, especially in fully leveraging the acceleration potential of 4-bit weights.\n\n2.ViDiT-Q introduces multiple quantization parameters (such as different $\\alpha$ values) to handle data variations across different timesteps. This complex parameter management increases the model's complexity.\n\n3. I believe the model can further introduce 8-bit Attention (SageAttention) to improve model efficiency, which has already been integrated into some video diffusion model libraries. I wonder if 8-bit attention mechanisms or some linear attention acceleration mechanisms can further improve your solution?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MVUCRZaouQ", "reviewer": "Reviewer_9VJP", "summary": "This paper proposes a quantization method tailored for diffusion Transformer models used in image and video generation. The authors conduct a detailed analysis of data scale variations across different components of the model, leading to the design of fine-grained grouping, dynamic quantization, and static-dynamic channel balance. To address the issue that quantization errors do not accurately reflect the quality of generation, they also introduce a metric-decoupled mixed-precision design. Experimental results show that the proposed method effectively improves the quality of generated outputs after quantization.", "questions": "Some questions have already been pointed out in Weakness. \n\nHere, I have an additional question regarding the metric decoupled mixed-precision design. The paper emphasizes that different parts of the model affect generation quality in different ways. I would like to know to what extent these sensitivities are decoupled, and whether there are any modules that are sensitive to multiple metrics simultaneously. If such modules exist, should a joint sensitivity analysis involving multiple metrics be conducted?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "(1) The paper provides a detailed analysis of the sources of degradation in generation quality post-quantization and then proposes corresponding solutions. The article is easy to understand and is written with a clear logical flow.\n\n(2) Experimental results demonstrate that, compared to existing methods, the proposed approach shows a significant improvement in generation quality.", "weaknesses": "The article is written with a relatively clear logic and has a certain degree of innovation. However, some parts still require further elaboration. Both the fine-grained grouping and dynamic quantization strategies are closely linked to existing methods. Yet, the author only briefly describes the differences without providing detailed, intuitive, or quantitative explanations. For instance, the specific distinctions between \"channel-wise\" and \"output-channel-wise\" are not clearly articulated, nor is it explained why \"timestep-wise quantization parameters\" would be more costly compared to the method proposed in this paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OU25MsyvIM", "reviewer": "Reviewer_GGBr", "summary": "This paper introduces ViDiT-Q (Video & Image Diffusion Transformer Quantization), a quantization scheme designed for Diffusion Transformers (DiTs) to reduce memory and computational demands in visual generation tasks like text-to-image and video synthesis. To address the unique challenges of DiT quantization, such as large data variation and time-varying channel imbalance, ViDiT-Q introduces fine-grained dynamic quantization for timestep-specific adjustments, a static-dynamic channel balancing technique, and a metric-decoupled mixed precision approach that allocates bit-widths based on layer sensitivity to visual quality metrics. Experiments demonstrate that ViDiT-Q achieves substantial hardware efficiency gains—up to 2-2.5x in memory savings and 1.4-1.7x in latency reduction—while maintaining high visual quality, making it a viable solution for deploying DiTs on constrained devices.", "questions": "1. Could you explore or discuss the feasibility of further quantizing ViDiT-Q to configurations like W4A4 or W2A8? Understanding its performance at lower bit-widths would clarify its limitations and potential for more aggressive compression.\n\n2. Could you provide an ablation study showing the individual contributions of fine-grained dynamic quantization, static-dynamic channel balancing, and metric-decoupled mixed precision?\n\n3. Have you considered benchmarking ViDiT-Q against broader quantization methods, such as adaptive quantization or knowledge distillation for compression?\n\n4. In your metric-decoupled mixed precision approach, did you observe any limitations or challenges with determining sensitivity dynamically? Further details on how this sensitivity analysis adapts to different models or layers could provide practical guidance for real-world implementation.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper introduces a unique approach tailored for Diffusion Transformers, addressing specific challenges like time-varying channel imbalance and data variation, which are rarely explored in quantization research.\n\n2.ViDiT-Q’s methodology is grounded in thorough quantization error analysis, with each technique validated through extensive experiments on both text-to-image and text-to-video tasks, showing careful consideration of practical performance.\n\n3. By achieving substantial memory and latency reductions, ViDiT-Q makes deploying DiTs on constrained devices feasible, enabling real-world applications for visual generation in resource-limited environments.\n\n4.  The paper is well-structured and clearly explains each component of ViDiT-Q, supported by effective figures and diagrams that make the methodology and results accessible and easy to follow.", "weaknesses": "1. While the paper addresses W4A8 quantization, there is limited exploration of even lower bit-width configurations, such as W4A4 or W2A8, which are often critical for more aggressive compression on edge devices. A deeper analysis of these configurations would broaden the applicability of ViDiT-Q.\n\n2. Although ViDiT-Q integrates several techniques, the paper lacks ablation studies that isolate the impact of each component, such as fine-grained dynamic quantization and static-dynamic channel balancing. Detailed ablations would provide clarity on the individual benefits of these methods.\n\n3. The paper primarily compares ViDiT-Q to diffusion-specific quantization methods but lacks benchmarks against general quantization techniques (e.g., adaptive quantization). Including these comparisons would better contextualize ViDiT-Q’s performance.\n\n4. The evaluation is restricted to text-to-image and text-to-video tasks, but the applicability of ViDiT-Q to other DiT applications (e.g., super-resolution or other image manipulation tasks) remains unexplored. Testing ViDiT-Q on these tasks could expand its impact and highlight potential limitations.\n\n5. The hardware efficiency results are limited to a single hardware setup (NVIDIA A100). Testing on additional platforms, particularly lower-power devices, would provide more comprehensive insights into ViDiT-Q’s practicality for diverse deployment environments.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)