$\text{S}^2$Q-VDiT: Accurate Quantized Video Diffusion Transformer with Salient Data and Sparse Token Distillation
Authors: Weilun Feng
Haotong Qin
Chuanguang Yang
Xiangqi Li
Han Yang
Yuqi Li
Zhulin An
Libo Huang
Michele Magno
Yongjun Xu
Venue: NeurIPS 2025 poster ()
Keywords: Diffusion Transformer
Model Quantization
Primary area: applications

Abstract: Diffusion transformers have emerged as the mainstream paradigm for video generation models. However, the use of up to billions of parameters incurs significant computational costs. Quantization offers a promising solution by reducing memory usage and accelerating inference. Nonetheless, we observe that the joint modeling of spatial and temporal information in video diffusion models (V-DMs) leads to extremely long token sequences, which introduces high calibration variance and learning challenges. To address these issues, we propose **$S^2$Q-VDiT**, a post-training quantization framework for V-DMs that leverages **S**alient data and **S**parse token distillation. During the calibration phase, we identify that quantization performance is highly sensitive to the choice of calibration data. To mitigate this, we introduce *Hessian-aware Salient Data Selection*, which constructs high-quality calibration datasets by considering both diffusion and quantization characteristics unique to V-DMs. To tackle the learning challenges, we further analyze the sparse attention patterns inherent in V-DMs. Based on this observation, we propose *Attention-guided Sparse Token Distillation*, which exploits token-wise attention distributions to emphasize tokens that are more influential to the model's output. Under W4A6 quantization, $S^2$Q-VDiT achieves lossless performance while delivering $3.9\times$ model compression and $1.3\times$ inference acceleration. Code will be available at https://github.com/wlfeng0509/s2q-vdit.

Q- Accurate Quantized Video Diffusion Transformer with Salient Data and Sparse Token Distillation
Weilun F Haotong Qin"; Chuanguang Yang",, Xiangqi Li Han Yang', Yugi Li', Zhulin An' Libo Huang", Michele Magno", Yongjun Xu² Sttate Key Laboratory of AT Safety, Institute of Computing Technology, Chinese Academy of Sciences
University of Chinese Academy of Sciences 'ETH Zürich
(fengweilun24s haw.uu//snss.anuuaaannnuuu wwwgghunnguunggg...ooommm axianggi24s anzhul in xyw.xict..cc............. (haotong . qin, mi chele wwweee.mmeoeoee.ee....m.eee.eeee (aagno)@pbl. ethz. ch, (yug111010602, www www.:haammllbbommal...ommmmm
Baseline Hunyuan
Latency
1.3x
FP
Baseline
saa-vDI
Model Storage
Q'QDVD
3.9x
FP Baseline
Prompt: stylish womon walks down Tokyo street filled with worm glowing neon and animated city signage.
Figure We present S DiTT, post-training quantization method for video diffusion transformers. We quantize Hunyuan Video (24) to 4-bit weights and 6-bit activations without compromising visual quality. QDQ-VDiT can further achieve 3.9x model compression and 1.3x inference acceleration.
Abstract
Diffusion transformers have emerged the mainstream paradigm for video gener- ation models. However, the use of up to billions of parameters incurs significant computational costs. Quantization offers a promising solution by reducing memory usage and accelerating inference. Nonetheless, we observe that the joint model- ing of spatial and temporal information in video diffusion models (V-DMs) leads to extremely long token sequences, which introduces high calibration variance and learning challenges. To address these issues, propose S VD'T, post- training quantization framework for V-DMs that leverages Salient data and Sparse token distillation. During the calibration phase, we identify that quantization per- formance is highly sensitive to the choice of calibration data. To mitigate this, we introduce Hessian-aware Salient Data Selection, which constructs high-quality calibration datasets by considering both diffusion and quantization characteristics unique to V-DMs. To tackle the learning challenges, further analyze the sparse attention patterns inherent in V-DMs. Based on this observation, we propose
* "Equal contribution. Corresponding authors: Zhulin An, anzhulin ict.ac.en; Chuanguang Yang, yangchuanguang@iin. wwt.gguuuuuuuttt...omo @cc..ac.cn
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Attention- Attention-guided Sparse Token Distillation, which exploits token-wise attention distributions emphasize tokens that are more influential to the model's output. Under W4A6 quantization, Q achieves lossless performance while de- livering 3.9x model compression and x.3x inference acceleration. Code will be available http://gghhi.ccco////een////////
Introduction
In recent years, diffusion transformer 39 has emerged as a powerful generative paradigm, demon- strating remarkable performance across diverse domains such as image synthesis [6]
26
9 557, audio
generation [5] and increasingly, video generation [37 35 Among these, video diffusion models
(V-DMs)
[58 24 represent new frontier by extending the spatial generative capabilities of image
diffusion models (I-DMs) into the spatial-tempont domain, enabling high-quality video synthesis
from textual prompts.
However, the transition from image to video generation introduces substantial computational chal- lenges, primarily due the exponential growth in token count introduced by the temporal dimen- sion [B5 58 [24]. These memory and compute demands become particularly severe in large-scale
video generation models
33 58 24], which contain up to billions of parameters, where each input
consists of thousands or even tens of thousands of tokens. To enable efficient deployment of such models in connrre-constraineed environments, post-training quantization (PTQ) [2 52 20 5
has
become a widely adopted approach. PTQ compresses the pre-trained models into low-bit bit representa- tions without modifying the model weights, relying only on small dataset calibrate quantization
parameters with only hours on a single GPU [5] 28).
While PTQ has proven effective for I-DMs 30
45
54
directly applying it to V-DMs leads to 54
substantial performance degradation 2 62
62 have sought to improve V-DMs' quantization performance primarily from the perspective of quantizer design. In this paper, we delve deeper into the PTQ challenges specific to V-DMs, focusing on calibration data and optimization methods.
Prior works
2
We identify that the long token sequences characteristic of V-DMs significantly constrain the number of calibration samples (c.g., thousands for I-DMs vs. only dozens for V-DMs under equal compu- tational budgets). Under such limited budgets, quantization performance becomes highly sensitive to the selection of calibration samples. Existing methods [54, 2 62 typically employ random or uniform sampling strategies, which work reasonably well for I-DMs but fail to generalize well to only dozens of data for V-DMs. Moreover, observe that V-DMs exhibit sparse attention patterns across all tokens. Current PTQ optimization frameworks S: 50
treat all tokens equally during loss alignment between full-precision and quantized models. However, this uniform treatment is suboptimal for long token sequences, where only a small subset of tokens significantly impacts the final output. These observations highlight two fundamental challenges in PTQ for V-DMs: (1) the absence of a principled method for selecting calibration samples, and (2) the inefficiency of uniform token treatment during optimization, despite the varying importance of tokens.
To address these limitations, we propose S Q-QTT,, a post-training quantization framework tailored for V-DMs, built upon Salient data selection and Sparse token distillation. An overview of the proposed framework is illustrated in Fig. First, we introduce Hessian- Salient Data Selection, which constructs calibration datasets by jointly assessing diffusion informativeness and quantization sensitivity. We define unified metric to quantify sample's saliency to the denoising process and its sensitivity to quantization perturbations. Second, we present Attention Attention-guided Sparse Token Distillation, technique that leverages the inherent sparsity of Rather than treating all tokens equally during optimization, we reweight quantization losses based on token-wiss attention distribution, allowing the model to focus more the impactful representations.
attention in V-DMs.
Our main contribution can be summarized as follows:
We empirically identify that V-DMs suffer from high calibration data variance in quantization performance. We propose Hessian-aware Salieni Data Selection, which jointly considers diffusion informativeness and quantization sensitivity to construct effective calibration datasets.

---

Hessian-aware Salient Data Selection
Attention-guided Sparse Token Distillation
Diffusion Salience
Hession Solience
Transformer Block
Cair
||x,
x--ll"
Cquant x|*lx
0f
Forward
|
Samples Pool
Attn
FFN
Backward
JL
Tokens
Token-wise Distribution
(1)0 6
Sallence Evaluation
Attention Mops
Selection
Calibration Dataset
Q-Attn
Q-FFN
Normalized
89
Gsappee
Cair Quuat
Q- O-Tronsformer Block
Figure 2:
Overview V'DT. The framework includes Hessian-aware Salient Data Selection (SDS) for constructing calibration dataset and Attention block-wise optimization.
iitention-guided guided Sparse Token Distillation (STD) for
We introduce Attention-guided Sparse Token Distillation, method that leverages the inherent sparsity in spatial-temporal attention of V-DMs. We reweight the quantization loss of different tokens by measuring token-wise attention distribution. This enables the model to focus more on the impactful representations during optimization.
Extensive experiments on large-scale video diffusion transformers with 2B to 13B parameters demonstrate that our Q-T-VDiT consistently outperforms existing PTQ baselines, achieving ofte-of-thhe-nn performance under all quantization settings.
2 Related Works
Diffusion models 46 17
have demonstrated strong generative capabilities in video generation tasks.
However, up to billions of parameters [35
49
58
24 pose major challenges for deployment in
resource-constrained environments. Quantization has emerged as widely adopted solution for model
compression acceleration 40 14
25). growing body of work has explored post-training quantization (PTQ) for diffusion models, particularly focusing U-Net-based architectures [30
21
45
16
18.
29 63). For the Diffusion Transformer architecture specifically, recent methods [54]
1. have made further explorations from the perspective of data distribution and architecture characteristics on guantization behavior. To address performance degradation at ultra-low bits, several quantization aware training approaches have been proposed [64 these methods typically require extensive training time and sarge-scale datasets, making them less practical in many scenarios.
36 65 I1]. While effective,
Despite these advances, most existing quantization research remains focused on image diffusion models (I-DMs), with limited exploration of video diffusion models (V-DMs). ViDiT-Q 62 and Q-DiT 2 have made the first explorations on the quantization of V-DMs. Q-DiT 2 introduces automatic quantization granularity allocation for fine-grained quantizer selection. ViDiT-Q 62 pro- poses static-dynamic quantization strategy to enhance quantization accuracy. While these approaches improve performance from different perspectives, they primarily focus on quantization granularity and quantizer design. In this paper, we tackle V-DM quantization from a new angle callbbatiooo data quality and optimization strategy. Our method achieves lossless performance various large-scale video diffusion transformers from 2B to 13B.
Methods
3.1 Preliminary
Video Diffusion Transformer. Diffusion transformers 339 predict the target using the represen- tation of multiple tokens X E R where and d represent the number tokens and feature
dimension, respectively. For image diffusion models (I-DMs) 42 26], n accounts for spatial

---

tokens. But for video diffusion models (V-DMs) [88 24 37,, n x incorporates the temporal dimension t. This significandly increases the token count per sample (e.g.. t 49 for a 6-second video at 8 FPS), resulting in heightened memory consumption and greater optimization complexity.
Post-training Quantization. Quantization maps the model weights and activation low-bit integers for acceleration and memory saving, For a float vector x, the symmetry quantization process can be formulated as:
A 1 1), A
max (ax(((x)))) 2N-1-1
Fint = clamp(round
(I)
A
where N is the quantized bit, rowndd(- is the round operation, and clamp(() constrains the value
within integer range 1-2 2 1]. Among quantization methods, post-training quantization
(PTQ) is more efficient method that only calibrates quantization parameters using a small calibration
dataset Deali without altering model weights. According to common practice [] [5 62 the quantization loss is expressed as:
Lquan Ex-Da [110 (x) (x)))]
(2)
where of and 8 denote the full full-precision and quantized model parameters, respectively. Dealib Nxxxd where N denotes the sample number in Dealib- Due to the limitations in computing resources and long token sequences in V-DMs, the calibration sample size N is smaller than that in I-DMs, leading to higher variance in data representation. This variance is further exacerbated by the diverse text prompts and different denoising timesteps present in the diffusion models.
3.2 Hessian-aware Salient Data Selection
FP
ATOP
ATFP
RTFP
Ours
Figure 3: Visualization of different calibration data Cog VideoX-2B. We compare our proposed method with All Timesteps from One Prompt (ATOP), All Timesteps from Five Prompts (ATFP), and Random Timesteps from Five Prompts (RTFP). Our method has better generation quality.
Observation 1. Calibration sample selection methods result in high variance of the quantized model performance.
In line with we discussed in Sec. 3.1 we observed that under constrained calibrated data size, different samples have significant differences in the final model performance as shown in Fig. and Fig. 66 However, the sample selection method for V-DMs post-training quantization has not been thoroughly explored. Therefore, we hope to evaluate the importance of different data for V-DMs. To address this issue, we propose evaluating sample utility along two dimensions that naturally exist in the quantization of diffusion models: contribution to the diffusion process and sensitivity to quantization.
Prior work timestep distillation 43
44
and caching 33 22 indicates that skipping certain
consecutive timesteps has limited impact on output quality, suggesting varying information content measure the salient information of timestep for
across different timesteps. Based this insight, the whole denoising diffusion process by calculating the contribution of two consecutive timesteps latent representation. Given all candidate data among all the diffusion timesteps [x] 22; ... where T' is the total denoising timesteps defined in the pretrained models. We define the diffusion salience as:
.T]]
1004
Jz- 11-1112
Cann
||
where 24 stands for the denoised feature of timestep t. A higher Can value denotes more informative denoising steps, while a lower Cair value indicates that the contained information largely overlaps with the previous timestep. Can naturally measures the saliency of different timesteps during the diffusion denoising process. By focusing on the salient data, we can better approximate the distribution of the entire diffusion process and achieve better performance.

---

We then consider the quantization of weight W and its quantized version W W + A, the quantization loss that jointly considers the input X can be be approximated using Taylor expansion:
B[j|xw - XW [ XW X(W + +1
Ag*
AH
where gx the gradient and H is the Hessian matrix. Using g for a well-trained model
provided in [32 59] and H |E|2X X] provided [3
Eq.
be further simplified
XWT T XW [19] R E
where Hessian matrix X X is given by matrix represents the inherent perturbation ability of sample X to the quantization process, which leads us to define quantization salience as:
approximation 12 38 The Hessian
quaatt = ||, z1l12.
where larger Cquam denotes that 2, is more sensitive to the quantization process due to the property of the Hessian matrix [13 12 59]. By focusing the quantization-sensine samples, can further relieve the bridge between the original data distribution and quantization operations, making the quantized model more robust and perform better.
To jointly emphasize diffusion informativeness and quantization sensitivity, we apply min-max normalization over the candidate calibration pool Dealib:
Omin
Cquant (z:) Cmin
Cam((x))
Cquant(az))
Cmmix
Cquax
mmin
quant
where Cmmin Cmaxx Opnin and quani denote the mininum value and maxminum value of all Cair(-)
and Cquam respectively. The unified salience score is then defined as the product:
Csample (xt) Catr( Equant S (Can((a))) Cyuam
(Ize) Cyuam((ze)
(8)
by the Aritheeti--Gemmetrii Mean inequality [67 which ensuring the score is maximized only when both normalized metrics are high. This mutual-salience product metric inherently penalizes samples that are only strong one dimension, aligns with mutual-information objectives, and yields more strong, robust calibration set.
3.3
Attention-guided Sparse Token Distillation
Sum Attention Weights
10%
200
50% 90%
40
Bottom 50%
Query
400
600
800
1000
250
500 Key
750
1000
0.000
25000
5000
10000
12500
15000
Token Index
(a) Attention heatmaps.
(b) Token-wise attention distribution.
Figure 4: Visualization of sparse attention patterns in Cog VideoX-2B block-10. In 4a). fewer columns have significantly higher weights. In (4b), only 10% of tokens have larger attention weights.
The fully spatial-temporal attention V-DMs exhibits certain sparse patterns, suggesting that only subsets of tokens notably impact the model output.
Observation 2.
5

---

Table I: Performance of 4-bit weight and 6-bit activation quantization text-to-video generation under VBench evaluation benchmark suite. We evaluate on Imaging Quality (QQ), Aesthetic Quality (AQ), Motion Smoothness (MS), Dynamic Degree (DD), Background Consistency (BC), Subject Consistency (SuC), Scene Consistency (ScC), and Overall Consistency (OC). Higher (t) metrics represent better performance. Bold: the best result.
Model
Method
IQ
AQ
MS
DD
BC
SuC
ScC
OC
FP
58.69
55.25
97.95
50.00
96.40
94.30
33.79
25.91
Q-DIT 48.63
47.63
98.08
19.44
95.30
92.15
23.84
24.00
PTQ4DiT
42.91
45.49
98.48
5.56
95.65
92.85
17.88
21.15
Cog VidcoX-22
SmoothQuant
44.60
44.33
98.22
9.72
95.62
92.04
18.60
21.20
Quarot
51.89
48.48
97.49
31.94
95.61
93.01
22.97
23.57
ViDIT-Q
51.94
48.06
97.47
33.33
95.54
92.87
22.17
23.69
Q-VVDIT
55.49
53.74
98.10
40.28
96.05
94.16
32.70
25.19
FP
61.80
58.88
97.61
72.22
95.56
94.63
45.28
26.46
Q-DiT
49.94
50.18
97.03
43.06
95.52
91.58
29.65
24.49
PTQ4DiT
43.54
42.70
97.77
4.17
96.70
93.32
10.93
21.75
Cog VideoX-BB
SmoothQuant
39.50
36.92
97.88
6.94
96.39
92.28
23.11
18.19
Quarot
43.95
44.81
97.33
31.94
96.58
92.27
20.93
22.34
ViDiT-Q
48.87
50.51
97.66
37.50
96.25
93.60
27.76
23.57
Q-VVDiT
60.75
56.90
97.46
58.33
96.76
94.24
46.66
26.30
FP
62.30
62.49
99.00
56.94
98.08
95.30
33.36
26.85
Q-DiT
50.23
48.40
98.95
40.28
97.14
94.03
18.46
14.41
PTQ4DiT
48.31
50.13
98.26
19.44
97.95
94.37
20.19
19.85
Hunyuan Video
SmoothQuant
47.55
56.03
98.77
27.78
97.33
94.57
23.69
25.47
Quarot
52.31
58.50
99.13
37.50
97.98
95.31
25.51
26.01
ViDiT-Q
52.21
58.38
99.12
41.67
98.02
95.20
23.69
26.15
Q-Q-DDIT
58.83
59.62
99.20
48.61
98.15
95.57
33.65
26.91
Let w E [Rd be the token embeddings, we can express Eq. 22 in the summation form as follows:
Lquant
l (xx,) 8 (z;,))1
where æj; refers the jia token in the video diffusion transformer. This loss function assumes that each token contributes equally to the overall error between the quantized and full-precision models. However, due the spatial-temporal modeling objectives, V-DMs typically require large-scale pretraining to achieve full convergence (37 58 24 56).
In the post-training quantization (PTQ) stage, only small dataset is used calibrate the quantization parameters, which naturally limits the model's ability to optimize from all tokens. Nevertheless, attention maps derived from V-DMs reveal that only subsets of tokens significantly influence the final output (see Fig. g. and Appendix Sec. H). This observation aligns with prior studies on attention in V-DMs (60] 61 (6), which have shown that pruning irrelevant tokens has negligible impact on generation quality. These findings motivate a strategy that focuses learning more intensely on salient tokens while relaxing constraints less impactful ones. Thereby enabling better convergence and improved performance even with limited calibration data.
To improve alignment between quantized and full-precision outputs, reweight each token's contribution the loss function based on its influence on the block output. Formally, we modify Eq. (9 to:
n Equant Σ X 21110 (I(,)) (1,,)))
(10)
j=1
where Xy denotes the weighting factor corresponding to token Ty..- Leveraging the attention mech- anism within each transformer block of V-DMs, can obtain the complete multi-heal attention

---

Table 2: Performance of both 4-bit weight and activation quantization text-to-video generation under VBench evaluation benchmark suite.
Model
Method
IQ
AQ
MS
DD
BC
SuC
ScC
OC
58.69
55.25
97.95
50.00
96.40
94.30
33.79
25.91
Q-DiT
26.26
27.66
99.14
98.09
96.52
1.16
8.43
PTQ4DiT
20.66
28.50
99.30
97.61
95.33
2.11
11.11
Cog VideoX-2B
SmoothQuant
29.76
28.31
98.95
97.62
94.65
5.31
9.74
QuaRot
43.22
39.59
97.54
13.89
96.18
92.35
12.21
19.57
ViDiT-Q
45.56
42.03
97.57
12.5
96.08
92.43
11.91
19.61
Q-Q-DDIT
53.71
52.31
98.09
36.11
96.15
93.99
34.23
24.90
FP
61,80
58.88
97.61
72.22
95.56
94.63
45,28
26.46
Q-DIT
40.80
33.00
95.71
36.
98.26
96.99
0.22
1.91
PTQ4DiT
41.48
28.63
96.38
97.29
95.09
7.37
Cog VideoX-5B
SuoothQuant
40.30
29.99
95.76
1.39
96.54
96.02
0.44
6.51
QuaRot
29.41
35.36
97.77
15.28
97.23
92.71
8.36
15.31
ViDiT-Q 31.95
36.71
97.09
15.28
96.37
93.01
10.85
16.91
Q-VVDIT 58.76
55.35
97.18
47.22
96.25
93.69
36.56
26.02
map A E xxnxn where H is the number of attention heads, naturally represents the importance matrix of different tokens within each block, and As.i. denotes the attention weight jth token receives from the ieh token in hes attention head. We use the attention map A to compute Xy using:
S min(S) max mS) (Amax Amii) + Amin+
S = Σ Ana. Ay h
(11)
where min (S) and max(S) denote the minimum and maximum values in all respectively. The hyperparameters. Amin and Amax define the normalization range for token importance. Ultimately, A quantifies each token's 's salience and helps guide the optimization process to prioritize alignment for tokens that exert greater influence.
Experiments
4.1 Experimental and Evaluation Settings
Quantization Scheme. We employ uniform per-channel weight quantization and dynamic per-token activation quantization with channel-wing scale and rotation matrix same as prior works [2] [ 62] We use symmetry quantization for both weight and activation for better hardware acceleration and memory saving. We follow the block-wise post-training strategy used in prior works (30 54 2
More implementation details and model settings can be seen in Appendix Sec. A
Evaluation Settings. We conduct text-to-video experiment on different scale SOTA models Cog VogVi CogVideoX-SB (58) and Hunyuan Video-133 (24 for better evaluation. We evaluate the performance of the quantized model using the VBench benchmark [19], which provides a com- select 8 major evaluation dimensions from VBench to ensure a thorough assessment. We also present more
prehensive evaluation video generation performance. Same the prior works
2
62
experiments on EvalCrafter (34 benchmark Appendix Sec. D As current works 2 62 have achieved almost lossless performance at high bits (c.g., 6-8 bits), we evaluated the performance at more challenging and unexplored low-bit W4A6 and W4A4 settings.
compare S*Q-VDiT with current PTQ baseline methods. For diffusion baseline, we compare with Q-DiT 2], PTQ4DiT 54 and ViDiT- 622). We further compare with strong LLM baseline, SmoothQuant (55) and QuaRot [I]
Compared Methods. Consist with prior works 2 62

---

BF16
Quarot
VIDIT-O
Q-DII
Prompt: panda standing surfboord in the ocean in sunset.
(a) Cog V
BF16
Quarot
VIDiT-Q
S Q-VDIT
Prompt: A robot DJ playing the turntable, in heavy raining futuristic tokyo rooftop cyberpunk night, sci-fi, fontasy.
(b) HunyuanVideo-133 Video-13BB
Figure 5: Visual comparison on different models under W4A6 quantization setting.
4.2 Quantitative Comparison
We present text-to-video experiment under VBench evaluation benchmark suite in Tab. 1 and Tab. 2 W4A6 Quantization. In Tab. [ focus relatively higher bit quantization setting of W4A6 (4-bit weight and 6-bit activation). three different scale current V-DMs Cog VideoXX2,,, Cog VideoX-55.. and Hunyuan Video-1
method outperforms all current quantization methods by a notable margin. Our Q VDIT achieves almost lossless performance across all eight selected dimensions. For Cog VideoXX-5BB S'QTVDiT even outperforms FP model with 46.66 scene consistency while other methods achieved the highest score 29.65.
we further explored the quantization performance of V-DMs under extremely low bit W4A4 settings. It is worth noting that this currently the first exploration under 4-bit activation quantization. In this extremely low bit setting, S*Q-VDiT can still maintain 95% of the model's performance while other methods showed significant performance degradation. Although some methods are particularly high in metrics such as SuC and BC, this is due their almost collapsed generation quality. ViDiT-Q (62) pointed out that these metrics are particularly high on extremely collapsed methods, and maintaining performance closer FP better. For Cog VideoX-2B, our method achieves even lossless scene consistency of 34.23 while other methods achieved the highest score of 12.21 with almost a three times improvement.
W4A4 Quantization. In Tab. []
4.3 Visual comparison
We present visual comparisons on different models under W4A6 in Fig. Compared with the current SOTA methods QuaRot [ and ViDiT-Q 62]. S VDiT has significant improvements in image quality and dynamic degree, and is lossless compared to FP models. For CogVideo QuaRot can hardly generate clear images; ViDiT-Q lacks the ability in color richness and image details; Q'QTTDiT is significantly better in color, detail, and video dynamics. For Hunyuan Vidco, although all methods have not significantly reduced image clarity, the semantic information of QuaRot has severely declined; the generated characters and background details of ViDiT-Q are also insufficient. S²Q-VDiT maintains high quality in the details and colors of both the background ensures the dynamic level of the video at different frame. The consistent and significant improvement on three different scales V-DMs also demonstrates the generalization and effectiveness of our method. We provide more visual comparison in Appendix Sec.
characters, and
8

---

Imaging Quality
Imaging Qualety
Overall Consees
Ansthelic Quality
Overall Consist.
Aesthetic Qualty
Scene Constet,
Notion Smooth.
Subject Consiss.
Dynamic Degree
Subject Consiat.
Dynamic Degree
FP15
ATOP
86. Consist
wi STD
Asan 0.1
(Durr))))
Auoo 0.7
(a) Ablation study on SDS.
(b) Ablation study on STD.
Figure 6: Ablation study of proposed methods on W4A4 Cog CogVideX-2BB
Table 3: Ablation study on calibration data size.
Calbration Time (Hour)
Imaging Quality
Aesthetic Quality
Overall
Method
Data Size
Consistency
FP
58.61
55.25
25.91
Q-DT
20
1.64
53.56
53.07
24.69
VDiT
40
2.88
55.49
53.74
25.19
QDITT
80
5.56
55.52
53.64
25.21
4.4
Ablation Study
In Fig. we present ablation studies on Hessian-aware Salient Data Selection (SDS) and Attention-guided Sparse Token Distillation (STD). To verify the effectiveness of these techniques, we conducted integration experiments with existing PTQ methods in Appendix Sec.
Ablation on SDS. We study different calibration data selection methods with our proposed SDS and shown the results in Fig. 6a We compare three different straightforward methods, including All Timesteps from One Prompt (ATOP), All Timesteps from Five Prompts (ATFP), and Random Timesteps from Five Prompts (RTFP). We selected 40 samples for all methods for fair comparison. We also present the visual comparison in Fig. Our proposed SDS outperforms all other methods in terms of both visual and metric effects while other methods can not maintain high generation quality. We conducted more ablation experiments used in SDS. We present the experimental results in Appendix Sec. B We further conduct an ablation study calibration data size in Cog VideoX-22 under W4A6 setting and present the results in Tab. 3 It can be seen that the calibration time increases almost linearly with the increase of data size. The performance of 40 data is significantly better than that of 20 data, but the performance improvement of 80 data is minor. Therefore, in the trade-off of performance and calibration time, we choose use 40 data as the unified experimental settings.
the random seeds and decoupled the two saliences
Ablation on STD. In Fig. 6b, we compare our proposed STD with no sparse distillation (w/o STD). It can be seen that compared with no STD, all distillation strategies can improve model performance. We also compare different hyperparameters used in Eq (11) We set Xmaz 1 as default and investigate different Xmin selections which control the relaxation degree on less impactful tokens. It can be seen that all different Amin can improve quantization performance which proves the robustness STD. We select Amnnn 0.5 in the main experiments for the most balanced per- formance improvement. We provide more visualization of the sparse patterns in Appendix Sec. H
4.5 Efficiency Study
We study the deployment efficiency of different-scale video diffusion transformers Tab. 5 We used the CUDA implementation provided in 662 47] for deployment and conducted all experiments

---

on single NVIDIA A800 GPU. For Inference Memory and Latency, we use a batch size of 1 in Tab Compared with baseline method PTQ4DiT [54]] our method brings significant performance improvement with almost no extra inference burden. Compared with FP model, our method can bring 3.94x model memory saving, 1.56x inference memory saving, and 1.28x inference acceleration on Cog Voggeeoe In Appendix Sec. F we conducted more experiments on deployment efficiency.
4.6 Calibration Resource Cost
Table 4: Calibration cost on W4A4 CogVideoX-2B.
Method
GPU Memory (GB)
GPU Time (hour)
Imaging Quality
Aesthetic Quality
FP
58.61
55.25
Q-DiT 29.85
2.69
26.26
27.66
PTO4DiT 33.30
2.25
20.66
28.50
S'Q-VDiT 35.68
2.88
53.71
52.31
We reported on the calibration resource consumption of our S'Q-VDiT compared with existing baseline methods Q-DiT [2] and PTQ4DiT [54] in Tab. Compared with existing methods, S²Q- VDiT only increases 2GB memory consumption and 0.2h calibration time, but improves Imaging Quality from 26.26 53.71, significantly enhancing the quantization performance. This proves the high efficiency and performance of Q-QTTD.. We further reported more detailed calibration resource consumption of each proposed component in Appendix Sec. G
Table 5: Efficiency study on different W4A6 models.
Model
Inference
Imaging Quality
Aesthetic
Model
Method
Latency (s)
Storage (GB)
Memory (GB)
Quality
FP
10.375
15.801
259.2
61,80
58.88
Cog CogVideoX-SB
PTQ4DiT
2.633
10.139
203.1
43.54
42.70
S Q-DIT
2.633
10.145
203.2
60.75
56.90
FP
23.881
29.260
191.3
62.30
62.49
Hunyuan Video
PTO4DIT
6.494
13.703
175.1
48.31
50.13
Q-VDIT
6.494
13.713
175.2
58.83
59.62
Conclusion
In this paper, we propose SQQ-DDIT, a post-training quantization framework for V-DMs using Salient data and Sparse token distillation. To address the sensitivity calibration data, propose Hessian-aware Salient Data Selection to construct high-quality datasets from the perspectives of diffusion and quantization. To address the learning challenge brought by long token sequences, propose Attention-guiis guided Sparse Token Distillation, which utilizes the natural sparse attention in V-DMs to allocate more loss weights to important tokens. Extensive experiments have shown that S*Q-VDiT outperforms all existing methods different scales of V-DMs.
Acknowledgements
This work partially supported by the National Natural Science Foundation of China under Grant Number 62476264 and 62406312, the Postdoctoral Fellowship Program and China Postdoctoral Science Foundation under Grant Number BX20240385 (China National Postdoctoral Program for Innovative Talents), the Beijing Natural Science Foundation under Grant Number 4244098, the Science Foundation the Chinese Academy Sciences, and Swiss National Science Foundation (SNSF) project 20002 EE 219943 Neuromorphic Attention Models for Event Data (NAMED).
10

---

References
[1] Salch Ashkboos, Amirkeivan Mohtashami, Maximilian L Croci, Bo Li, Pashmina Cameron, Martin Jaggi, Dan Alistarh, Torsten Hoefler, and James Hensman. Quarot: Outlier-free 4-bit inference in rotated Ilms. arXiv preprint arXiv:2404.0044 2024.
[2] Lei Chen, Yuan Meng. Chen Tang, Xinzhu Ma, Jingyan Jiang. Xin Wang, Zhi Wang, and Wenwu Zhu. Q-dit: Accurate post-training quantization for diffusion transformers. arXiv preprint arXiv:2406. 17343, 2024.
[3] Krishna Teja Chitty-Venkata Venkata, Sparsh Mittal, Murali Emani, Venkatram Vishwanath, and Arun Somani. A survey of techniques for optimizing transformer inference. Journal of Systems Architecture, page 102990, 2023.
[4] Hangliang Ding, Dacheng Li, Runlong Su, Peiyuan Zhang, Zhijie Deng, Ion Stoica, and Hao Zhang. Efficient-vdit: Efficient video diffusion transformers with attention tile. arXiv preprint arXiv:2502.061555 2025.
[5] Yifu Ding, Weilun Feng, Chuyan Chen, Jinyang Guo, and Xianglong Liu. Reg-piq: Regression- specialized post-training quantization for fully quantized object detector. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16174-16188 84, 2024.
[6] Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow trans- formers for nigh-resolution image synthesis. In Forty-first international conference on machine learning, 2024.
[7] Yuming Fang, Hanwei Zhu, Yan Zeng, Kede Ma, and Zhou Wang. Perceptual quality assessment of smartphone photography. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pages 3677-3686, 2020.
[8] Weilun Feng, Haotong Qin, Chuanguang Yang, Zhulin An, Libo Huang, Boyu Diao, Fei Wang, Renshuai Tao, Yongjun Xu, and Michele Magno. Mpq-dm: Mixed precision quantization for extremely low bit diffusion models. In Proceedings of the AAAI Conference Artificial Intelligence, volume 39, pages 16595-16603, 2025.
[9] Weilun Feng, Chuanguang Yang, Zhulin An, Libo Huang, Boyu Diao, Fei Wang. and Yongjun Xu. Relational diffusion distillation for efficient image generation. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 205-213, 2024.
[10] Weilun Feng, Chuanguang Yang. Haotong Qin, Xiangqi Li, Yu Wang. Zhulin An, Libo Huang, Boyu Diao, Zixiang Zhao, Yongjun Xu, al. Q-vdit: Towards accurate quantization and distillation of video-generation diffusion transformers. arXiv preprint a5X5v: 22 267, 2025.
[11] Weilun Feng, Chuanguang Yang, Haotong Qin, Yuqi Li, Xiangqi Li, Zhulin An, Libo Huang, Boyu Diao, Fuzhen Zhuang, Michele Magno, al. Mpq-dmv2: Flexible residual mixed precision quantization for low-bit diffusion models with temporal distillation. arXiv preprint arXiv:2507.04290. 2025.
[12] Elias Frantar and Dan Alistarh. Optimal brain compression: A framework for accurate post- training quantization and pruning. Advances in Neural Infommaionn Processing Systems, 35:4475- 4488, 2022.
[13] Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. Gptq: quantization for generative pre-trained transformers. arXiv preprint 1rXiv:2210.17322 2022.
Accurate post-training
[14] Amir Gholami, Sehoon Kim, Zhen Dong, Zhewei Yao, Michael W Mahoney, and Kurt Keutzer. survey of quantization methods for efficient neural network inference. In Low-Power Com- puter Vision, pages 291-326. Chapman and Hal//CRC, 2022.
[15] Jiarui Hai, Yong Xu, Hao Zhang, Chenxing Li, Helin Wang, Mounya Elhilali, and Dong Yu. Ezaudio: Enhancing text-to-audio generation with efficient diffusion transformer. arXiv preprint 24X9v:2409.108/ 10819, 2024.

---

[16] Yefei He, Luping Liu, Jing Liu, Weijia Wu, Hong Zhou, and Bohan Zhuang. Ptqd: Accurate post-training quantization for diffusion models. Advances Neural Information Processing Systems, 36, 2024.
[17] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
[18] Yushi Huang, Ruihao Gong, Jing Liu, Tianlong Chen, and Xianglong Liu. Tfmq-dm: Temporal feature maintenance quantization for diffusion models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition. pages 7362-7371, 7371, 2024.
[19] Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive benchmark suite for video generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21807-21818, 2024.
[20] Itay Hubara, Yury Nahshan, Yair Hanani, Ron Banner, and Daniel Soudry. Improving post training neural quantization: Layer-wise calibration and integer programming. arXiv preprint arXiv:2006. 10518, 2020.
[21] Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko. Quantization and training of neural networks for efficient anteger-aritmmetic-on inference. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages
2018.
[22] Kumara Kahatapitiya, Haozhe Liu, Sen He, Ding Liu, Menglin Jia, Chenyang Zhang. Michael Ryoo, and Tian Xie. Adaptive caching for faster video generation with diffusion transformers. arXiv preprint arXiv:2411.00 1..09977, 2024.
[23] Junjie Ke, Qifei Wang, Yilin Wang, Peyman Milanfar, and Feng Yang. Musiq: Multi-scale image quality transformer. In Proceedings of the IEEE/CVF international conference on computer vision, pages 5148-5157, 2021.
[24] Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03600. 2024.
[25] Raghuraman Krishnamoorthi. Quantizing deep convolutional networks for efficient inference:
A whitepaper. arxiv 2018. arXiv preprint arXi::1806.08834 1806.
[26] Black Forest Labs. Flux. httpp:://gittubb.cmmmbaoodoo-ooa waw.orresstllsssflluu 2024.
[27] LAION-AL Aesthetic-predictor, 2022. Accessed: 2022-04-16.
[28) Jiedong Lang, Zhehao Guo, and Shuyu Huang. A comprehensive study on quantization techniques for large language models. In 2024 4th International Conference on Artificial Intelligence, Robotics, and Communication (ICAIRC), pages 224-231. IEEE, 2024.
[29] Muyang Li, Yujun Lin, Zhekai Zhang, Tianle Cai, Xiuyu Li, Junxian Guo, Enze Xie, Chenlin Meng, Jun-Yan Zhu, and Song Han. Svdqunat: Absorbing outliers by low-rank components for 4-bit diffusion models. arXiv preprint arXiv:2411.05007, 2024.
[30] Xiuyu Li. Yijiang Liu, Long Lian, Huanrui Yang. Zhen Dong, Daniel Kang, Shanghang Zhang and Kurt Keutzer. Q-diffusion: Quantizing diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 17535-17545, 2023.
[31] Yanjing Li, Sheng Xu, Xianbin Cao, Xiao Sun, and Baochang Zhang. Q-dm: An efficient low-bit quantized diffusion model. Advances in Neural Information Processing Systems, 36, 2024.
[32] Yuhang Li, Ruihao Gong, Xu Tan, Yang Yang, Peng Hu, Qi Zhang, Fengwei Yu, Wei Wang, and Shi Gu. Brecq: Pushing the limit of post-training quantization by block reconstruction. arXiv preprint 210.022 2021.
12

---

[33] Feng Liu, Shiwei Zhang, Xiaofeng Wang, Yujie Wei, Haonan Qiu, Yuzhong Zhao, Yingya Zhang, Qixiang Ye, and Fang Wan. Timestep embedding tells: It's time cache for video diffusion model. arXiv preprint arXiv:2411 1t.::2411.1908,, 2024.
[34] Yaofang Liu, Xiaodong Cun, Xuebo Liu, Xintao Wang, Yong Zhang, Haoxin Chen, Yang Liu, Tieyong Zeng, Raymond Chan, and Ying Shan. Evalcrafter: Benchmarking and evaluating large video generation models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages
2024.
[35] Yixin Liu, Kai Zhang. Yuan Li, Zhiling Yan, Chujie Gao, Ruoxi Chen, Zhengqing Yuan, Yue Huang, Hanchi Sun, Jianfeng Gao, et al. Sora: A review background, technology, limitations, and opportunities of large vision models. arXiv preprint axXiv:2402.17177. 2024.
[36] Xudong Lu, Aojun Zhou, Ziyi Lin, Qi Liu, Yuhui Xu, Renrui Zhang, Yafei Wen, Shuai Ren, Peng Gao, Junchi Yan, et al. Terdit: Ternary diffusion models with transformers. arXiv preprint arXiv:2405. 14854, 2024.
[37] Xin Ma. Yaohui Wang, Gengyun Jia, Xinyuan Chen, Ziwei Liu, Yuan-Fang Li, Cunjian Chen, and Yu Qiao. Latte: Latent diffusion transformer for video generation. arXiv preprint MwXiv:2401.0304 2024.
[38] Donald W Marquardt. An algorithm for least-squares estimation of nonlinear parameters. Journal of the society for Industrial and Applied Mathematics, 11(2):431-44 4411, 1963.
[39] William Peebles and Saining Xie. Scalable diffusion models with transformers. Proceedings of the IEEF/CVF International Conference on Computer Vision, pages 4195-4205, 2023.
[40] Ratko Pilipović, Patricio Bulić, and Vladimir Risojević. Compression of convolutional neural networks: A short survey. In 2018 17th International Symposium INFOTEH-JAHORA (INFOTEH), pages 1-6. IEEE, 2018.
[41] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748-8763. PMLR, 2021.
[42] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684-10695, 2022.
[43] Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. arXiv preprint arXiv:2202.00511 2022.
[44] Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach. Adversarial diffusion distillation. In European Conference on Computer Vision, pages 87-103. Springer, 2024.
[45] Yuzhang Shang, Zhihang Yuan, Bin Xie, Bingzhe Wu, and Yan Yan. Post-training quantization on diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattem recognition. pages 1972-1981, 2023.
[46] Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint 2011:22010 2020.
[47] Yuxuan Sun, Ruikang Liu, Haoli Bai, Han Bao, Kang Zhao, Yucning Li, Jiaxin Hu, Xianzhi Yu, Lu Hou, Chun Yuan, et al. Flatquant: Flatness matters for I1m quantization. arXiv preprint 1.Xiv:2410009426 2024.
[48] Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. In Computer EcionnEECV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part I1 16, pages 402-41 419. Springer, 2020.
[49] Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang. et al. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv arXi:2500..23144 2025.
13

---

[50] Yi Wang, Yinan He. Yizhuo Li, Kunchang Li, Jiashuo Yu, Xin Ma, Xinhao Li. Guo Chen, Xinyuan Chen, Yaohui Wang, et al. Internvid: large-scale video-text dataset for multimodal understanding and generation. arXiv preprint arXiv:2307.06942, 2023.
[51] Lu Wei, Zhong Ma, Chaojie Yang, and Qin Yao. Advances in the neural network quantization: A comprchensive review. Applied Sciences, 14(17):7445, 2024.
[52] Xiuying Wei, Ruihao Gong, Yuhang Li, Xianglong Liu, and Fengwei Yu. Qdrop: Ran- domly dropping quantization for extremely low-bit post-training quantization. arXiv preprint a2X3v:2203,05740, 2022.
[53] Haoning Wu, Erli Zhang, Liang Liao, Chaofeng Chen, Jingwen Hou, Annan Wang. Wenxiu Sun, Qiong Yan, and Weisi Lin. Exploring video quality assessment user generated contents from aesthetic and technical perspectives. Proceedings of the IEEE/CVF International Conference on Computer Vision, pages
2023.
[54] Junyi Wu, Haoxuan Wang, Yuzhang Shang, Mubarak Shah, and Yan Yan. Ptq4dit: Post-training quantization for diffusion transformers. arXiv preprint MXiv:2405.16008 2024.
[55] Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. Smoothquant: Accurate and efficient post-training quantization for large language models. In International Conference on Machine Learning, pages 38087-38099, PMLR, 2023.
[56] Chuanguang Yang, Helong Zhou, Zhulin An, Xue Jiang, Yongjun Xu, and Qian Zhang. Cross- image relational knowledge distillation for semantic segmentation. In Proceedings of the IEFE/CV Conference on Computer Vision and Pattern Recognition, pages 12319-12328, 2022.
[57] Han Yang, Chuanguang Yang, Qiuli Wang, Zhulin An, Weilun Feng, Libo Huang, and Yongjun Xu. Multi-party collaborative attention control for image customization. Proceedings of the Computer Vision and Pattern Recognition Conference, pages 7942-7951, 2025.
[58] Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to-vides diffusion models with an expert transformer. arXiv preprint arXiv:2408.0600 2024.
[59] Zhihang Yuan, Chenhao Xue, Yiqi Chen, Qiang Wu, and Guangyu Sun. Ptq4vit: Post-training quantization for vision transformers with twin uniform quantization. In European conference on computer vision, pages 191-207. Springer, 2022.
[60] Jintao Zhang, Chendong Xiang, Haofeng Huang, Jia Wei, Haocheng Xi, Jun Zhu, and Jianfei Chen. Spargeattn: Accurate sparse attention accelerating any model inference. arXiv preprint arX1v:2502.18137, 2025.
[61] Peiyuan Zhang, Yongqi Chen, Runlong Su, Hangliang Ding, lon Stoica, Zhenghong Liu, and Hao Zhang. Fast video generation with sliding tile attention. arXiv preprint arXiv:2502.04507, 2025.
[62] Tianchen Zhao, Tongcheng Fang. Enshu Liu, Wan Rui, Widyadewi Soedarmadji, Shiyao Li, Zinan Lin, Guohao Dai, Shengen Yan, Huazhong Yang, et al. Vidit-q: Efficient and accurate quantization of diffusion transformers for image and video generation. arXiv preprint arXiv:2406.025400 2024.
[63] Tianchen Zhao, Xuefei Ning, Tongcheng Fang, Enshu Liu, Guyue Huang, Zinan Lin, Shengen Yan, Guohao Dai, and Yu Wang. Mixdq: Memory-efficient few-step text-to-image diffusion models with metric-decouple mixed precision quantization. In European Conference on Computer Vision. pages 285-302. Springer, 2025.
[64] Xingyu Zheng, Xianglong Liu, Yichen Bian, Xudong Ma, Yulun Zhang Jiakai Wang, Jinyang Guo, and Haotong Qin. Bidm: Pushing the limit of quantization for diffusion models. arXiv preprint arXiv:2412.05926, 2024.
14

---

[65] Xingyu Zheng, Haotong Qin, Xudong Ma. Mingyuan Zhang, Haojie Hao, Jiakai Wang, Zixiang Zhao, Jinyang Guo, and Xianglong Liu. Binarydm: Towards accurate binarization of diffusion model. arXiv preprinl arXiv:2404.0500
2024.
[66] Chang Zou, Xuyang Liu, Ting Liu, Siteng Huang, and Linfeng Zhang. Accelerating diffusion transformers with token-wiss wise feature caching. The Thirteenth International Conference on Learning Representations, 2025.
[67] Limin Zou and Youyi Jiang. Improved writhmetic-eoomette mean inequality and its application.
J. Math. Inequal, 91):1077-111 11, 2015.
15

---

A Implementation Details
In the main experiment, We finally selected 40 samples for post-training quantization for all methods, For our method, we use channel-wise scale used in [55 62 54] and rotation matrix used in [47] for linear quantization. We further use a learnable threshold for clipping the weight and activation min-max value as prior We also use GPTQ weight quantizer [13] for our experiment, following prior
10 random prompts for generating the candidate calibration samples.
work 30 18
work
We conduct all the experiments on single NVIDIA A800 GPU.
For optimization, we train the diag-balancing scale, rotation- rotation-based matrix. and learnable clipping threshold following the layer-wise post-training quantization framework as prior works (30 [54]. We use 30 samples and train 15 epochs for each layer. We use AdamW optimizer and cosine learning rate scheduler. For the diag-balancing scale and rotation- matrix, we use a learning rate of 5e-3.
For the learnable clipping threshold, we use learning rate of 5e-2.
For deployment, we absorb all weight quantization parameters as prior works [44 53 62 which brings no extra burden. For activation quantization, we apply online dynamic quantization follow- ing 62
B More Ablation on Hessian-aware Salient Data Selection
Table 6: Performance of both 4-bit weight and activation quantization on Cog VideoX-222 under three
random seeds.
Imaging
Aeshetic
Motion
Dynamic
BG
Subject
Scome
Overil
Quality
Quality
Smooth.
Degres
Consist.
Consist
Consist.
Consist.
58.69
55.25
97.95
50.00
96-41)
94.30
33.79
25.91
ATOS
(1 .76)
(0 59)
96 00.00: 16)
29. (7:1:3
05 35)
93, (0.24:2:000) 19)
29. 94 (1,35)
24.31:1:0.37)
(0.63:)-0088)
(0.25)
98.05 (0.11)
29.63 (22662)
95.94:5(0.16)
(3.16:1:0441
30.95 (2.14)
24.11:1:0.27)
DTTDS
50.66:
50.33g00.119
(0.13
(4.58))(4.55)
96.01
(0.07
(1.47::(.777)
24.75:(00225)
DS
(2.73:2:0:58)
50622:0881)
98. (0.119 19)
31,75::(2.73)
96 96.06:2 0.1))
(0.15
31,38:200.55)
24.78:1:0:22)
(2.342((085)
(0.17=(022))
98.1 (0.111)))
(2.01:222:97)
96. (0.11)
(1.572(011)
31 (1.862200000
24.79::00223)
SDS:Oursi
$2.95:1(0066)
(0.18:2(011))
98.16:1:0.09)
32,87 (2.36)
96.13:1(0.11)
93,89 (6.17)
10.75:110.77)
24.241((0266)
In this section, we investigate the random seed influence on the quantization performance of different calibration datasets mentioned in Sec. 3.2 and Sec. Sec.44.4 We compare our proposed Hessian-aware Salient Data Selection (SDS) with All Timesteps from One Prompt (ATOP), All Timesteps from Five Prompts (ATFP). and Random Timesteps from Five Prompts (RTFP) using three different random seeds. We further decoupled SDS into Diffusion Salience (DS) in Eq. B and Quantization Salience (QS) in Eq. (6) and reported the performance. We present the average results and variance in Tab.
Other straightforward sampling methods have lower average performance and larger variances, prov- ing the influence of random seeds in these random sampling methods. Using our proposed diffusion salience (DS) quantization salience (QS) can all improve the performance and reduce the impact of random seeds. Only using DS and QS can improve Scene Consistency to over 31 with variances less than 1, while other random sampling methods cannot achieve. By jointly considering two saliences, aassian-awwre Salient Data Selection (SDS) can achieve the best quantization performance with minimal impact from randomness. SDS achieved average Imaging Quality of 52.95 with only 0.69 variance, while the random sampling only achieved the best average of 51.65 with 1.67 variance.
C Detailed Description of Selected Evaluation Metrics
C.1 VBecch Benchmark
For VBench 19 benchmark, we follow the previous work ViDiT-Q 62 which selects dimensions from three key aspects in video-generation task.
Frame-wise Quality: this aspect, we assess the quality of each individual frame without taking temporal quality into concern.
16

---

Imaging Quality assesses distortion (e.g., over-exposure, noise) presented in the generated
frames using the MUSIQ 23 image quality predictor trained on the SPAQ 7 dataset.
Aesthetic Quality evaluates the artistic and beauty value perceived by humans towards each video frame using the LAION aesthetic predictor [27].
Temporal Quality: In this aspect, we assess the cross-frame temporal consistency and dynamics.
Dynamic Degree evaluates the degree of dynamics (i.e., whether it contains large motions) generated by each model.
Motion Smoothness evaluates whether the motion in the generated video smooth, and follows the physical law of the real world.
Subject Consistency assesses whether the subject's appearance remains consistent through- out the whole video.
Background Consistency evaluate the temporal consistency of the background scenes by calculating CLIP 41 feature similarity across frames.
Semantics: In this aspect, we evaluate the video's adherence to the text prompt given by the user. consistency.
Scene evaluates whether the synthesized video is consistent with the intended scene de- scribed by the text prompt.
Overall Consistency further overall video-text consisteney computed by VICLIP (50) on general text prompts as an aiding metric to reflect both semantics and style consistency.
We use three different prompt sets provided by the official github repository of VBench 19 to generate videos. We generate video for each prompt for evaluation same ViDiT-Q 62
overall consistency.txt: includes 93 prompts, used to evaluate overall consistency, aesthetic quality, and imaging quality.
subject consistency.txt: includes 72 prompts, used to evaluate subject consistency, dynamic degree, and motion smoothness.
scene.txt: includes 86 prompts, used evaluate scene and background consistency.
C.2 EvalCrafter Benchmark
For EvalCrafter 34] benchmark, consistent with prior work ViDiT-Q 62 we select 5 low-level
metrics to evaluate the generation performance.
CLIPSIM and CLIP-Temp: CLIPSIM computes the image-text CLIP similarity for all frames in the generated videos, and we report the averaged results. This quantifies the similarity between input text prompts and generated videos. CLIP-Temp computes the CLIP similarity of each two consecutive frames of the generated videos and then gets the averages for each two frames. This quantifies the semantic consistency of generated videos. We use the CLIP-VIT-B/32 (50] model to compute CLIPSIM and CLIP-Temp. We use the implementation from EvalCrafter to compute these two metrics.
DOVER's VQA: Vee-Tccchical measures common distortions like noise, blur, and over-exposure. reflects aesthetic aspects such as the layout, the richness and harmony of colors, the photo-realism, naturalness, and artistic quality of the frames. We the Dover (53) method to compute these two metrics.
Flow score was proposed in [34 measure the general motion information of the video. We use RAFT [88] to extract the dense flows of the video in every two frames, and we calculate the average flow on these frames obtain the average flow score of each generated video.
FLOW Score:
We use the prompt sets provided by the official github repository of ViDiT-Q 62 to generate 10 videos for evaluation. We also attached the prompt sets in the supplementary material.
17

---

D Experiments on more metrics
Following prior work we evaluate different methods on EvalCrafter 34 benchmark for multi-
aspects metrics evaluation. We select CLIPSIM, CLIP-Temp, DOVER [3] video quality assessment (VQA) metrics to evaluate the generation quality, and Flow-score to evaluate the temporal consistency, We conduct experiments on Cog VideoX-2B, Cog VigVieooX5B,, and Hunyuan Video- 13B under W4A6
quantization setting. We present the evaluation results in Tab.
Table Performance of 4-bit weight and 6-bit activation quantization on eex--tovvieo generation under EvalCrafter benchmark. Higher (f) metrics represent better performance.
VQA-
VQA- Technical
FLOW
Model
Method
CLIPSIM
CLIP-Temp
Aesthetic
Score.
FP
0.1844
0.9978
76.64
85.02
3.452
Q-DIT
0.1787
0.9978
63.15
67.37
2.331
PTQ4DIT
0.1772
0.9985
58.76
52.60
1.837
CogViceo-22B
SmoothQuant
0.1762
0,9981
55.18
53.87
1.378
Quarot
0.1808
0.9975
51.83
56.79
2.867
ViDiT-Q
0.1812
0.9976
53.09
59.84
3.040
Q-QTT
0.1838
0.9979
70.50
73.31
3.122
FP
0.1814
0.9982
78.87
73.17
4.536
Q-DiT
0.1835
0.9976
47.96
46.72
2.967
PTQ4DiT
0.1789
0.9984
22.93
44.07
2.230
Cog VideoX-5B
SmoothQuant
0.1742
0.9976
3.05
14.13
1.026
Quarot
0.1805
0.9983
33.10
43.67
3,040
ViDiT-Q
0.1795
0.9980
42.01
48.59
1.850
QQQVDIIT
0.1819
0.9987
73.45
74.41
3.688
FP
0.1910
0.9985
80.66
63.51
1.674
Q-DiT
0.1871
0.9987
56.45
43.17
1.482
PTQ4DIT
0.1786
0.9973
42.17
33.69
1.089
Hunyuan Video
SmoothQuant
0.1782
0.9978
7.24
0.42
0.111
Quarot
0.1873
0.9977
66.49
52.81
0.899
ViDiT-Q 0.1895
0.9978
66.23
51.35
0.897
Q-
0.1902
0.9985
77.80
66.38
1.562
can be seen that under the EvalCrafter benchmark, our S*Q-VDiT still achieved almost lossless performance and showed significant performance improvement compared to all comparison methods. Especially in terms of VQA-TTcchncal metrics, our S²Q-VDiT even outperforms the full precision model on Cog Videoo -B and Hunyuan VineuanVideo, while other methods show notable performance degradation. For Cog VideoX-5B, S*Q-VDiT achieves 74.41 in VQA-Technical which outperforms the full precision model of 73.17, while current methods achieve the best of 48.59.
E
Integration with Existing PTQ Methods
The techniques that we proposed Hessian-aware Salient Data Selection (SDS) and Attention-guie guideed Sparse Token Distillation (STD) can also be applied to existing block-wise optimization- based post-training quantization methods. To verify the generality of these two techniques, we combined them with the existing baseline method PTQ4DiT [54) and reported the performance improvement of these techniques on W4A6 Cog VideoxX022 under VBench [19] benchmark in Tab. g By using the calibration constructed by SDS, we further improved the performance of PTQ4DiT and increased Aesthetic Quality by 1.4. This demonstrates the improvement of SDS-constructed datasets under different optimization frameworks. From optimization perspective, we further improved the Aesthetic Quality to 47.27 by using sparse distillation STD. This also demonstrates the effectiveness and generalization of our attention-baun optimization method.
18

---

Table 8: Performance of 4-bit weight and 6-bit activation quantization Cog Vodeoo00-22 under
VBench evaluation benchmark suite
Imaging Quality
Aesthetic Quality
Motion
Dynamic
BG
Subject Consist.
Scene Consist.
Overall
Method
Smooth.
Degree
Consist.
Consist.
FP 58.69
55.25
97.95
50.00
96.40
94.30
33.79
25.91
PTQ4DiT
42.91
45.49
98.48
5.56
95.65
92.85
17.88
21.15
+SDS
43.06
46.89
98.64
11.11
95.79
93.33
18.10
22.27
+STD
43.08
47.27
98.78
9.72
95.97
93.68
19.04
22.09
F
More Experiments Deployment Efficiency
PTO-OIT Sie VDIT
242
PTO4DIT
SPQ VDIT
000
5Pe VDIT
1T24DIT
cco
2353 944
188-
50
100
Bonth
Batmn
(a) CogVideoX-20
(b) Cog CogVideoX-5B
Hunyuan Video- 13B.
Figure 7: Deployment latency comparison under different batch size.
We further expanded the experiments provided in Sec. 4.5 We compared the deployment efficiency of different models under different batch sizes in Fig. Our Q-QVDIIT can bring consistent inference acceleration to different models under different batch sizes. Under the 50-step inference setting of VogVideoX-DD 5B with a batch size of 4, our S'Q-VDiT can reduce the inference latency from 945.4, 4s to 782.5s, achieving significant acceleration of 163 seconds and outperforming the baseline method PTQ4DiT [54].
Table 9: Calibration cost about each component.
Hessian Approximation
Attention Computation
Construct Time (mins)
Imaging Quality
Calibration Time (hours)
Imaging Quality
Method
Method
CogVideoX-22
FP
58.69
FP
58.69
w/o Hessian
7.708
53.16
w/o Attention
2.82
52.16
w Hessian
7.717
55.49
W Attention
2.84
55.49
CogVideoX-5B
FP
61.80
FP
61.80
w/o Hessian
20.719
58.91
w/o Attention
3.97
58.23
w Hessian
20.734
60.75
W Attention
4.00
60.75
Hunyuan Video-13B
FP
62.30
FP
62.30
w/o Hessian
19.505
57.25
w/o Attention
5.70
56.94
w Hessian
19.508
58,83
w Attention
5.73
58.83
19

---

G More Detailed Calibration Resource Cost
We reported the time increase caused by using the Hessian approximation when constructing the calibration dataset and the attention scores calculation across different scale video generation models in Tab. 9
It can be seen that the computational burden of using Hessian approximation is minor, but it can bring significant performance improvement. We use the Levenberg- -aaruuurr-arruurrd approximation [3] to calculate the Hessian approximation, which requires only one step matrix multiplication to obtain the approximate result, and very efficient.
Also, during the calibration process, we only need to use the Full-Precision model to conduct single forward calculation of attention scores for all data advance. When optimizing the quantization model, we can directly get the pre-computed attention scores by the data index, which brings minimal burden.
H
More Visualization about Sparse Attention Pattern
28
4.00
200
240
0000
200
-009
100
1204
S1M
623-
0004
256
1000
200
2000
200
0000
350
(a) Block-5.
(b) Block 12.
(c) Block- 13.
(d) Block- 14.
1200
AME
40%
000
600-
009
2/5
200
10:00
209
200
2000
400
203
1000
420
525
1/0
2003
(c) Block-15.
(f) Block--17 17.
(g) Block-18
(h) Block- 19.
Figure 8: Visualization of attention heatmaps in Cog VideoX-2B.
20C 150
10.0
1200
1952
500
1007
Sale Files
(a) Block-5.
(b) Block- 12.
(c) 130ck-13.
(d) Block-
(e) B5ock-15.
(I) Block-17.
(g) Block-
(h) Block-19.
Figure Visualization of noken-wise attention distribution Cog VideoX-2B.
We demonstrate the sparse attention patterns existing in V-DMs that we mentioned Sec 3.3 We present more visualization results of different blocks of Cog VideoXX2B in Fig. and Fig. 9 There a considerable degree of sparse attention patterns the most layers the model, and almost all 90% tokens have significantly lower attention weights than the top 10% tokens. This indicates that
20

---

FP
SmoothQuant
Q-DiT
ViDiT-Q
Q'D-VDiT
Figure 10: Hunyuan Video 13B results. Prompt: A cat wearing sunglasses on a beach.
sparse attention is commonly present in V-DMs, and almost every layer only has small portion of tokens that play an important role in the final output. This proves the universality of our observations Sec. 3.3 and the effectiveness of our Attention-guicl guidded Sparse Token Distillation.
I More Visualization Results
We present more visual comparison results on Hunyuan Video- 24
CogVideoX-5BB and
Cog VideoX-2B (58] under W4A6 quantization in the following figures. Compared with current meth- ods SmoothQuant (55), Q-DiT ViDiT-Q [62], our Q-VDD made notable visual improvement different scale video diffusion models. This proves that our S-Q-VDiT not only surpasses existing methods in terms of evaluation metrics but also shows significant improvement in visual effects, demonstrating the effectiveness of our S'Q-VDiT.
Limitations
Although our QDQTT outperforms existing methods, it cannot achieve completely lossless per- formance under the most difficult fully 4-bit quantization. We hope to optimize the quantization performance under low bit settings in the future.
K
Broader Impacts
Our quantized model may be used by people to generate false content, and we will require users to apply our model in legitimate and reasonable scenarios and label Al-generated.
21

---

FP
Smooth
Q-DIT
ViDIT-Q
S'Q S'Q-VDIT
Figure 11: Hunyuan Video- 13B results. Prompt: A boat sailing leisurely along the Seine River with the Eiffel Tower in background.
FP
SmoothQuant
Q-DIT
ViDIT-Q
Q'Q-DDiT
Figure 12: Hunyuan Video-13B results. Prompt: A panda cooking in the kitchen.
22

---

FP
SmoothQuant
Q-DiT
ViDiT-Q
Q-D-VD
Figure 13: Cog VideoX-5B results. Prompt: A beautiful coastal beach in spring, waves lapping sand by Hokusai, in the style of Ukiyo.
FP
SmoothQuant
Q-DiT
ViDiT-Q
Q2Q-VDiT
Figure 14: Cog VideoX-BB results. Prompt: A modern art museum, with colorful paintings.
23

---

FP
SmoothQuant
Q-DiT
ViDiT-Q
Q-Q-VD
Figure 15: CogVidoX results. Prompt: Yoda playing guitar the stage.
FP
SmoothQuant
Q-DIT
ViDiT-Q
S2Q-VDIT
Figure 16: CogVideXX-28 results. Prompt: Macro slo-mo. Slow motion eropped closeup of roasted coffee beans falling into an empty bowl.
24

---

FP
SmoothQuant
Q-DIT
ViDiT-Q
Q'DIVDiT
Figure 17: Cog CogVideoX22B -B results. Prompt: A boat sailing leisurely along the Seine River with the Eiffel Tower in background by Vincent van Gogh.
NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The claims reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of the work in Appendix Sec. J
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
25

---

The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proof's
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [NA]
Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced
statement of any theorems. The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification:
We provide all the details in Sec. and Appendix Sec. A
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
26

---

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g.. registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
We provide the code in supplemental material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cr cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
7/
gwwss CCodeSoodiieeeeee ssi mollicyy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
27

---

Answer: [Yes]
Justification:
We provide the details in Appendix Sec. A
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification:
We report experiment about statistical significance in Appendix Sec. B
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of exccution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide the compute resources in Sec. 4 Sec A and Appendix See G
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
28

---

Question:
Does the research conducted in the paper conform, in every respect. with the
NeurIPS Code of Ethics
Answer: [Yes]
Justification:
We conform with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss the broader impacts in Appendix Sec.
K
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation. generating fake profiles, surveillance), fairness considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
29

---

We recognize that providing effective safeguards is challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: The models used in the paper are properly credited.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website). the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswi thcode. cow datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We did release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
30

---

According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based on the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to L.MM policy http:p/s//earrpspscc///offeeeeea for what should or should not be described.
31


Reviews:
[{"review_id": "QHe4zcHV54", "reviewer": "Reviewer_QBEu", "summary": "This paper proposes a post training quantization method for video diffusion transformer. The paper addresses the issues in PTQ from two perspectives regarding the long sequence characteristics of video DiTs. For limited calibration data caused by long sequences, the paper proposes Hessian-aware Salient Data Selection to select key calibration data. Aiming at the naturally occurring sparse attention phenomenon in V-DiTs,  Attention-guided Sparse Token Distillation is proposed to focus more on key tokens to improve the performance during calibration process. Massive experiments are conducted on multiple popular V-DiTs and achieved SOTA performance.", "questions": "1.Providing more low-level metrics for video evaluation like PSNR and SSIM would further strength the superiority of the proposed methods.\n\n2.Providing more acceleration results under different resolution of HunyuanVideo would be better for verifying the acceleration effect in different scenarios.", "limitations": "yes", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths:\n1. The paper focuses on the problems caused by long token sequences in video diffusion models during PTQ process, which is a novel and valuable perspective.  And the paper writing is clear and easy to understand, with good definitions and solutions to the problems from both calibration data and optimization process perspective.\n2. The method proposed in the paper has a good theoretical basis and a large number of model analyses and references, ensuring the effectiveness of the methods.\n3. Compared with existing methods, the paper has significant improvements in metrics and visual effects, proving the effectiveness of the method. And random seeds experiments are provided in the appendix to demonstrate the robustness of the method.\n4. The paper conducted extensive quantitative and ablation experiments on multiple large-scale models, demonstrating the generality and effectiveness of the method.\n\n##Weaknesses:\n1. The paper provides evaluation results under VBench and Evalcraft benchmark, but providing more low-level metrics such as PSNR and SSIM will more fully demonstrate the effectiveness of the method.\n\n2. Large video diffusion models such as HunyuanVideo support video generation at multiple resolutions. The paper provides acceleration effects under union experiment setting. But providing different resolutions results, especially at higher resolutions, will further illustrates the potential of the method for acceleration.", "was_revised": true, "final_justification": "My concerns are resolved. I remain positive on this work.", "rebuttal": "Thank you very much for your high recognition of our work and the valuable suggestions you provided. Our response is as follows:\n\n> **Q1:** Providing more low-level metrics for video evaluation like PSNR and SSIM would further strength the superiority of the proposed methods.\n\n**A1:** We provide more low-level metrics (PSNR, SSIM, and LPIPS) comparison results of CogVideoX-5B and HunyuanVideo under W4A6 setting. **Compared with the existing state-of-the-art methods Quarot [1] and ViDiT-Q [2], our method achieves the best performance in all metrics**.\n\nWe provide the results in the following table. The experimental results show that our method not only has better results on the VBench (Tab.1 and Tab.2) and EvalCrafter (Tab. 5) provided in the main paper, but also has advantages on the low-level metrics. It further proves that our method can generally improve the model performance.\n\n|Method|PSNR$\\uparrow$|SSIM$\\uparrow$|LPIPS$\\downarrow$|\n|-|-|-|-|\n|HunyuanVideo||||\n|Quarot|19.13|0.6168|0.3864|\n|ViDiT-Q|19.22|0.6171|0.3837\n|**Ours**|**19.64**|**0.6359**|**0.3068**|\n|CogVideoX-5B|\n|Quarot|16.17|0.4963|0.4780|\n|ViDiT-Q|17.26|0.5389|0.4620|\n|**Ours**|**17.58**|**0.5865**|**0.3322**|\n\nReference:\n\n[1]. Ashkboos, et al. QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs. NeurIPS 2024.\n\n[2]. Zhao, et al. Vidit-q: Efficient and accurate quantization of diffusion transformers for image and video generation. ICLR 2025.\n\n> **Q2:** Providing more acceleration results under different resolutions of HunyuanVideo would be better for verifying the acceleration effect in different scenarios.\n\n**A2:** We provide multi-resolution efficiency study on HunyuanVideo under W4A4 setting. Compared with FP model, our method achieves obvious model compression and inference acceleration under various resolutions. \n\nWe provide the detailed results in the following table:\n\n|Method|Resolution|Model Storage (GB)$\\downarrow$|Inference Memory (GB)$\\downarrow$|Latency (s)$\\downarrow$|\n|-|-|-|-|-|\n|FP|512×768|23.9|29.3|191.3|\n|**Ours**|512×768|6.5 (**3.7×**)|13.7 (**2.1×**)|175.2 (**1.09×**)|\n|FP|720×1280|23.9|35.8|807.5|\n|**Ours**|720×1280|6.5 (**3.7×**)|22.7 (**1.6×**)|684.0 (**1.18×**)|\n\n**We find that the model acceleration effect is more obvious when the resolution is increased.** We believe that when the resolution is improved and the number of tokens to be processed by the model is increased, the greater computational burden can benefit more from the quantization technique. This fully reflects the great potential of quantization for generating high-resolution video."}, {"review_id": "xD5BcCp692", "reviewer": "Reviewer_uJYk", "summary": "This paper introduces S2Q-VDiT, a post-training quantization (PTQ) framework designed to address the performance degradation of video diffusion models (V-DMs) under low-bit quantization. Due to the long token sequences and limited calibration budget inherent in V-DMs, existing PTQ approaches often struggle with unstable performance. To tackle this, S2Q-VDiT introduces two main components. The first is Hessian-aware Salient Data Selection, which scores and selects calibration samples based on their contribution to the denoising process and their sensitivity to quantization, using an efficient approximation of the Hessian. The second is Attention-guided Sparse Token Distillation, which leverages the sparsity of attention maps to prioritize important tokens during quantization loss computation, assigning higher weights to those with stronger semantic relevance.\nThe method is evaluated on large-scale video diffusion models, including CogVideoX and HunyuanVideo, under W4A6 and W4A4 quantization settings. Results show that S2Q-VDiT maintains high generation quality while achieving up to 3.9× model compression and 1.3× inference speedup. Additionally, the proposed techniques can be integrated into other PTQ frameworks and require only a small number of calibration samples, with minimal one-time overhead during deployment.", "questions": "Please refer to the weakness part.", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength:\n* The paper is easy to follow, with a clear structure that helps the reader understand the challenges being addressed and the proposed solutions.\n* The authors introduce a Hessian-aware salient data selection method, offering a novel and principled approach for selecting calibration data, which improves the post-training quantization process compared to traditional random sampling.\n* The method effectively leverages the sparse attention patterns inherent in video diffusion models (VDMs), emphasizing salient tokens to enhance PTQ performance.\n* Extensive experiments are conducted across multiple models and quantization settings to validate the robustness of the proposed approach. The method is compared with various PTQ baselines using multiple evaluation metrics, showing strong performance in generation quality. Furthermore, it outperforms other calibration data selection strategies across all reported metrics.\n\n Weaknesses:\n* It seems that the calculation of the Hessian approximation introduces non-negligible computational overhead, which can become significant as the number of calibration samples or timesteps increases.\n* The paper lacks a discussion or ablation study on the effect of calibration data size. It does not investigate how performance varies with different numbers of calibration samples, nor does it explore the trade-off between calibration data size and quantization quality.", "was_revised": true, "final_justification": "Thank you to the authors for responding to my questions and concerns. After reading the author's response and other reviewers' comments, I will raise my score to 5.", "rebuttal": "We are deeply grateful for your support of our work, and we provide detailed responses to your comments as follows:\n\n>**Q1:** It seems that the calculation of the Hessian approximation introduces non-negligible computational overhead, which can become significant as the number of calibration samples or timesteps increases.\n\n**A1:** We reported the time increase caused by using the Hessian approximation when constructing the calibration dataset in different models. **Compared with without using Hessian, using Hessian approximation will only bring less than 0.1% additional time, which is very efficient.**\n\nWe provide the detailed comparison in the following table:\n|Method|Construct Time (mins)$\\downarrow$|Imaging Quality$\\uparrow$|\n|-|-|-|\nCogVideoX-2B (50steps)\n|FP|-|58.69|\n|Ours (w/o Hessian)|7.708|53.16|\n|Ours (w Hessian)|7.717 (**+0.1％**)|**55.49**|\nCogVideoX-5B (50steps)\n|FP|-|61.80|\n|Ours (w/o Hessian)|20.719|58.91|\n|Ours (w Hessian)|20.734 (**+0.07％**)|**60.75**|\nHunyuanVideo-13B (30steps)\n|FP|-|62.30|\n|Ours (w/o Hessian)|19.505|57.25|\n|Ours (w Hessian)|19.508 (**+0.01％**)|**58.83**|\n\nIt can be seen that the computational burden of using Hessian approximation is minor, but it can bring significant performance improvement. **We use the Levenberg-Marquardt approximation [1][2] to calculate the Hessian approximation, which requires only one step matrix multiplication (See Eq. 6) to obtain the approximate result, and is very efficient.**\n\nReference:\n\n[1]. Frantar, et al. Optimal brain compression: A framework for accurate post-training quantization and pruning. NeurIPS 2022.\n\n[2]. Frantar, et al. Gptq: Accurate post-training quantization for generative pre-trained transformers. ICLR 2023.\n\n>**Q2:** The paper lacks a discussion or ablation study on the effect of calibration data size. It does not investigate how performance varies with different numbers of calibration samples, nor does it explore the trade-off between calibration data size and quantization quality.\n\n**A2:** We conduct ablation study on calibration data size in CogVideoX-2B under W4A6 setting. **Increasing data size will bring additional calibration burden, but it does not always bring significant performance improvement. We finally chose a compromise solution of 40 data size.**\n\nWe provide the detailed results in the following table:\n|Method|Data Size|Calbration Time (h)|Imaging Quality$\\uparrow$|Aesthetic Quality$\\uparrow$|Overall Consistency$\\uparrow$|\n|-|-|-|-|-|-|\n|FP|-|-|58.69|55.25|25.91|\n|Ours|20|1.64|53.26|53.07|24.69|\n|Ours|40|2.88|55.49|**53.74**|25.19|\n|Ours|80|5.56|**55.52**|53.64|**25.21**|\n\nIt can be seen that the calibration time increases almost linearly with the increase of data size. The performance of 40 data is significantly better than that of 20 data, but the performance improvement of 80 data is minor. Therefore, in the trade-off of performance and calibration time, we choose to use 40 data as the unified experimental settings, as we described in Appendix Sec. A (Line 441)."}, {"review_id": "0J2Vlnww78", "reviewer": "Reviewer_ZQ33", "summary": "This paper introduces S²Q-VDiT, a post-training quantization (PTQ) framework designed specifically for video diffusion transformers (V-DMs). The core contribution addresses two major challenges in quantizing V-DMs: the high variance in quantization performance due to limited calibration data, and the inefficiency of uniform token treatment during optimization in the presence of long token sequences and sparse attention patterns. To tackle these, the authors propose two key components: Hessian-aware Salient Data Selection (SDS) and Attention-guided Sparse Token Distillation (STD).", "questions": "please refer to the weaknesses part.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. The paper identifies and addresses crucial challenges specific to post-training quantization of Video Diffusion Models (V-DMs), particularly the high variance from limited calibration data and inefficient token treatment in long sequences.\n\n2. The proposed Hessian-aware Salient Data Selection (SDS) and Attention-guided Sparse Token Distillation (STD) are well-motivated and technically sound. SDS combines diffusion informativeness and quantization sensitivity effectively. STD leverages the empirically observed sparse attention patterns in V-DMs for more efficient optimization.\n\n3. The methods are thoroughly evaluated on large-scale V-DMs ranging from 2B to 13B parameters, demonstrating consistent state-of-the-art performance against various baselines. The evaluation uses the VBench benchmark suite, which covers multiple quality dimensions including Imaging Quality, Aesthetic Quality, Motion Smoothness, Dynamic Degree, and various consistencies.\n\nWeaknesses:\n\n1. While the paper mentions that QAT requires extensive training time and large datasets, a more direct comparison or discussion on scenarios where QAT might still be preferred could strengthen the argument for PTQ.\n\n2. Although the paper highlights a 1.3x inference acceleration , the detailed latency numbers in Table 3 show a more modest improvement, especially for the Hunyuan Video model (191.3s for FP vs. 175.2s for S²c-VDIT). This translates to approximately a 1.09x acceleration for Hunyuan Video and 1.28x for Cog VideoX-5B (259.2s for FP vs. 203.2s for S²-VDIT). Given the significant model compression (3.9x), the observed inference acceleration is not as substantial as might be expected, which could make the quantization less convincing for applications prioritizing runtime speed above all else.", "was_revised": true, "final_justification": "Thank you to the authors for their detailed rebuttal. I have reviewed both their responses and the comments from the other reviewers. The rebuttal satisfactorily addressed most of my concerns. Therefore, I will maintain my original score.", "rebuttal": "Thank you for reviewing our manuscript and providing valuable suggestions. Here are our responses to some of the concerns you raised:\n\n>**Q1:** While the paper mentions that QAT requires extensive training time and large datasets, a more direct comparison or discussion on scenarios where QAT might still be preferred could strengthen the argument for PTQ.\n\n**A1:** QAT conducts full training on the model weight and quantization parameters. which can guarantee relatively good performance under some extreme bits (1-2 bits). However, compared with the PTQ paradigm, QAT usually requires more training resources. For example, **we compare our PTQ method with the existing QAT work for diffusion models [1][2] and report the required training resources:**\n\n|Paradigm|Model Size|Training Time|Training Data|\n|-|-|-|-|\n|QAT|LDM (**~1GB**) [3]|~30h (**10×**)|300k (**7500×**)|\n|PTQ|CogVideoX (**~4GB**)|~3h|40|\n\n**Compared with the QAT paradigm, the PTQ paradigm we use can greatly reduce computing resources and calibrate larger models, which fully reflects its efficiency.** It is still worth mentioning that under extreme bit (1-2 bit) quantization, QAT is still a reliable method that can guarantee the maximum performance of the model. However, in this paper, we mainly explore 4-bit quantization, and only an efficient PTQ process can ensure the nearly lossless model performance.\n\nReference:\n\n[1]. Zheng, et al. Bidm: Pushing the limit of quantization for diffusion models. NeurIPS 2024.\n\n[2]. Zheng, et al. Binarydm: Accurate weight binarization for efficient diffusion models. ICLR 2025.\n\n[3]. Rombach, et al. High-resolution image synthesis with latent diffusion models. CVPR 2022.\n\n>**Q2:** Although the paper highlights a 1.3x inference acceleration, the detailed latency numbers in Table 3 show a more modest improvement, especially for the Hunyuan Video model (191.3s for FP vs. 175.2s for S²c-VDIT). This translates to approximately a 1.09x acceleration for Hunyuan Video and 1.28x for Cog VideoX-5B (259.2s for FP vs. 203.2s for S²-VDIT). Given the significant model compression (3.9x), the observed inference acceleration is not as substantial as might be expected, which could make the quantization less convincing for applications prioritizing runtime speed above all else\n\n**A2:** Our empirical observation shows that HunyuanVideo does have a relatively low acceleration ratio. However, our quantization method has significant memory compression and inference acceleration across a variety of popular video generation models, and the acceleration effect is more obvious in high-resolution generation.\n\nFor the deployed quantization model, compared with CogVideo-X, we do observe that HunyuanVideo has a relatively lower inference acceleration rate. We attribute this to the dual-stream DiT block used in HunyuanVideo. The dual-stream DiT block introduces much more high-precision normalization, scale, and shift operations that cannot fully benefit from quantization. **We also observed that for models that support multi-resolution generation like HunyuanVideo and Wan2.1 [1], higher resolutions that are closer to the business scenarios can bring better acceleration effects.** We present the comprehensive evaluation results here:\n\n|Method|Resolution|Model Storage (GB)$\\downarrow$|Inference Memory (GB)$\\downarrow$|Latency (s)$\\downarrow$|\n|-|-|-|-|-|\nCogVideoX-2B\n|FP|480×720|3.2|6.7|166.5|\n|Ours|480×720|0.8 (**4.0×**)|5.5 (**1.2×**)|97.5 (**1.71×**)|\nCogVideoX-5B\n|FP|480×720|10.4|15.8|259.2|\n|Ours|480×720|2.6 (**3.9×**)|10.1 (**1.6×**)|203.2 (**1.28×**)|\nHunyuanVideo-13B\n|FP|512×768|23.9|29.3|191.3|\n|Ours|512×768|6.5 (**3.7×**)|13.7 (**2.1×**)|175.2 (**1.09×**)|\n|FP|720×1280|23.9|35.8|807.5|\n|Ours|720×1280|6.5 (**3.7×**)|22.7 (**1.6×**)|684.0 (**1.18×**)|\nWan2.1-14B\n|FP|480×832|26.6|33.9|1202.0|\n|Ours|480×832|7.00 (**3.8×**)|16.1 (**2.1×**)|971.0 (**1.24×**)|\n|FP|720×1280|26.6|42.5|4031.0|\n|Ours|720×1280|7.00 (**3.8×**)|26.0 (**1.6×**)|3160.0 (**1.28×**)|\n\nWe can see that latency results are consistent with our observations. Also on the four popular open-source video generation models, quantization can bring continuous inference acceleration. **The benefits of quantization are not only reflected in latency.** The compression of model storage can greatly reduce the model storage occupation. The significant reduction of inference memory can make it more widely used on devices with limited computing resources, which has high research value for the wide application of video generation models.\n\nReference:\n\n[1]. Wan, et al. Wan: Open and advanced large-scale video generative models. Arxiv 2025."}, {"review_id": "75aSz6ptv6", "reviewer": "Reviewer_gBkv", "summary": "This paper proposes S$^2$Q-VDiT, a post-training quantization framework for video diffusion models, which leverages salient data selection and sparse token distillation to improve quantization performance. Specifically, the authors introduce a Hessian-aware salient data selection method to construct a high-quality calibration set, and propose attention-guided sparse token distillation to retain tokens that are more influential to the model’s output. Comprehensive experiments on the VBench benchmark demonstrate the effectiveness of the proposed techniques.", "questions": "1. In the W4A6 setting, why are activations quantized to 6 bits while weights use only 4 bits? Some rationale or empirical justification would help clarify this design choice.\n\n2. In the attention modules, do the authors distinguish between cross-attention and self-attention mechanisms?", "limitations": "While the proposed methods show strong empirical performance, they may introduce notable computational overhead. In particular, computing the Hessian matrix for salient data selection and attention scores for sparse token distillation could be costly, especially in large-scale or resource-constrained settings.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. The paper is generally well written and easy to follow.\n\n2. Both the Hessian-aware salient data selection and attention-guided sparse token distillation components are well motivated. The authors provide insightful observations and analyses that support the design of their proposed methods.\n\n3. The experimental evaluation on VBench is thorough and shows promising performance improvements over existing baselines.\n\nWeaknesses:\n1. In Figure 5, it would be helpful to include frame indices for better interpretation of temporal consistency.\n\n2. The abbreviation “FP” likely refers to “full precision,” but this should be explicitly defined in the text for clarity.\n\n3. The mathematical derivation from Equation (4) to Equation (6) is somewhat condensed and would benefit from additional explanation or intermediate steps to improve readability and reproducibility.\n\n4. The computation of the Hessian matrix for dataset samples and attention scores for each token may introduce significant computational overhead, especially for large-scale video diffusion models. A discussion of the method’s scalability and computational cost would be valuable.", "was_revised": true, "final_justification": "I am satisfied with the authors’ response and have adjusted my score accordingly.", "rebuttal": "We thank you for your positive feedback and comments. We respond to the concerns below:\n\n>**Q1:** Writing improvement about Fig.5, \"FP\", and Eq.(4) to Eq.(6).\n\n**A1:** Thank you for your valuable suggestions. We will modify our statements according to your requirements in the revised version to make them clearer and more readable.\n\n>**Q2:** Computational overhead about the Hessian approximation.\n\n**A2:** We report the computational burden of Hessian approximation across different scale video generation models. **In fact, the Hessian approximation is very efficient, and will only bring minor computational burden ($\\le$ 0.1%).**\n\nWe reported the time increase caused by using the Hessian approximation when constructing the calibration dataset in the following table:\n\n|Method|Construct Time (mins)$\\downarrow$|Imaging Quality$\\uparrow$|\n|-|-|-|\nCogVideoX-2B\n|FP|-|58.69|\n|Ours (w/o Hessian)|7.708|53.16|\n|Ours (w Hessian)|7.717 (**+0.1％**)|**55.49**|\nCogVideoX-5B\n|FP|-|61.80|\n|Ours (w/o Hessian)|20.719|58.91|\n|Ours (w Hessian)|20.734 (**+0.07％**)|**60.75**|\nHunyuanVideo-13B\n|FP|-|62.30|\n|Ours (w/o Hessian)|19.505|57.25|\n|Ours (w Hessian)|19.508 (**+0.01％**)|**58.83**|\n\nIt can be seen that the computational burden of using Hessian approximation is minor, but it can bring significant performance improvement. **We use the Levenberg-Marquardt approximation [1][2] to calculate the Hessian approximation, which requires only one step matrix multiplication to obtain the approximate result, and is very efficient.**\n\nReference:\n\n[1]. Frantar, et al. Optimal brain compression: A framework for accurate post-training quantization and pruning. NeurIPS 2022.\n\n[2]. Frantar, et al. Gptq: Accurate post-training quantization for generative pre-trained transformers. ICLR 2023.\n\n\n>Q3: Computational overhead about the attention scores calculation.\n\n**A3:** We report the computational burden of the attention scores calculation across different scale video generation models. **The attention scores calculation is very efficient, and will only bring minor computational burden ($\\le$ 0.7%).**\n\nWe reported the time increase caused by calculating the attention scores during calibration process in the following table:\n\n|Method|Calibration Time (h)$\\downarrow$|Imaging Quality$\\uparrow$|\n|-|-|-|\nCogVideoX-2B\n|FP|-|58.69|\n|Ours (w/o Attention)|2.82|52.16|\n|Ours (w Attention)|2.84 (**+0.7％**)|**55.49**|\nCogVideoX-5B\n|FP|-|61.80|\n|Ours (w/o Attention)|3.97|58.23|\n|Ours (w Attention)|4.00 (**+0.7％**)|**60.75**|\nHunyuanVideo-13B\n|FP|-|62.30|\n|Ours (w/o Attention)|5.70|56.94|\n|Ours (w Attention)|5.73 (**+0.5％**)|**58.83**|\n\nDuring the calibration process, **we only need to use the Full-Precision model to conduct a single forward calculation of attention scores for all data in advance.** When optimizing the quantization model, we can directly get the pre-computed attention scores by the data index, which brings minimal burden.\n\n>**Q4:** In the W4A6 setting, why are activations quantized to 6 bits while weights use only 4 bits? Some rationale or empirical justification would help clarify this design choice.\n\n**A4:** Since our main baseline method ViDiT-Q [1] has explored the high bit quantization of the video generation model (e.g., W8A8 and W6A6) and almost achieved the lossless performance, **continuing to experiment under high bit cannot fully reflect the differentiation.** Also, lower bits can bring greater model compression effect and fewer bit operations, which is more significant for the model compression and acceleration. Therefore, **we target the 4-bit weighted quantization that can bring higher compression ratio but is still heavily underexplored.**\n\nWe first explored W4A4, a full 4-bit quantization setting. This is a very radical setting for Post-training Quantization, but our method still made great improvement compared with existing methods (See Tab. 2, Line 182) with only minimal performance drop (~1%).\n\nWe then hope to guarantee the lossless model performance under 4-bit weight quantization so that it can be widely used in practice. We found that under W4A6 setting, the existing quantization methods still show significant performance degradation, but our method can guarantee almost lossless performance. Therefore, we further conduct W4A6 quantization experiments in Tab.1 (See Line 164). **We further reduce the quantization bit of the lossless video generation model to W4A6 and set a new benchmark for video model quantization.**\n\nReference:\n\n[1]. Zhao, et al. Vidit-q: Efficient and accurate quantization of diffusion transformers for image and video generation. ICLR 2025.\n\n\n>**Q5:** Difference between cross-attention and self-attention mechanisms.\n\n**A5:** The experiment of CogvideoX and HunyuanVideo in this paper use a single 3D-attention on the architecture, so there is no need to distinguish attention mechanisms. For models that distinguish attention mechanisms, such as Wan2.1 [1], we only use the results of self-attention. We give the detailed explanation in the following paragraph. And **we will add the explanation of this part in the revised version.**\n\nAt present, the mainstream open-source video generation models are mainly divided into two types of architectures: MMDiT (e.g., CogVideo-X [2] and HunyuanVideo [3], which are used in the paper's experiments) and DiT (e.g., Wan2.1 [1]) architectures. \n\n**MMDiT architecture uses only a single 3D-Full Attention, which directly applies global attention to visual and text tokens.** Therefore, we directly use the results of 3D attention to calculate the attention score in the proposed *Attention-guided Sparse Token Distillation* (introduced in Sec. 3.3). This is also the implementation in the paper's experimental part.\n\nThe DiT architecture applies two different attention mechanisms, self-attention and cross-attention. However, in the current mainstream architecture (e.g., Wan2.1 [1]), text tokens are only injected as fixed conditions, meaning that the output token of the attention block only contains visual tokens. **The purpose of our proposed *Attention-guided Sparse Token Distillation* is to reweight the output token's contribution in the alignment to the Full-Precision model. Therefore, we only use the attention score of self-attention mechanism to calculate token-wise distribution mentioned in Eq. 11 (Line 178).**\n\nReference:\n\n[1]. Wan, et al. Wan: Open and advanced large-scale video generative models. Arxiv 2025.\n\n[2]. Yang, et al. Cogvideox: Text-to-video diffusion models with an expert transformer. Arxiv 2024.\n\n[3]. Kong, et al. Hunyuanvideo: A systematic framework for large video generative models. Arxiv 2024."}]

Author rebuttal:

Meta-review: The paper considers the problem of post-training quantization in video diffusion transformers (V-DMs) where availability of calibration data would be much less than in image-diffusion models. To this end, the paper leverages two ideas, a Hessian-based salient data selection that ranks diffusion time steps based on the generated content, and an attention-guided sparse token distillation leveraging the observation that only a small number of tokens are actually attended in the diffusion steps. Experiments are presented on text-to-video methods using VBench and EvalCrafter, showing clear improvements in image quality and dynamic degree. 

The paper received four reviews, all reviewers very positive in their recommendation. There were concerns raised on a few aspects. 
1) Missing experiments, e.g., with different resolutions, evaluation metrics, calibration data size, etc. (QBEu, uJYk) -- authors provided the numbers during rebuttal.
2) Computational overhead from the Hessian (uJYk, ZQ33, gBkv) -- clarifications were provided during the rebuttal. 

AC had an independent reading of the paper and agrees with the reviewers that the paper studies an important problem in video diffusion and makes a practical contribution towards enabling video diffusion models for low-resource settings. Thus, recommends acceptance. Authors should incorporate the additional results into the revised paper.
Decision: Accept (poster)