QVD: Post-training Quantization for Video Diffusion Models

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-05 12:35:05

Abstract: Recently, video diffusion models (VDMs) have garnered significant attention due to their notable advancements in generating coherent and realistic video content. However, processing multiple frame features concurrently, coupled with the considerable model size, results in high latency and extensive memory consumption, hindering their broader application. Post-training quantization (PTQ) is an effective technique to reduce memory footprint and improve computational efficiency. Unlike image diffusion, we observe that the temporal features, which are integrated into all frame features, exhibit pronounced skewness. Furthermore, we investigate significant inter-channel disparities and asymmetries in the activation of video diffusion models, resulting in low coverage of quantization levels by individual channels and increasing the challenge of quantization. To address these issues, we introduce the first PTQ strategy tailored for video diffusion models, dubbed QVD. Specifically, we propose the High Temporal Discriminability Quantization (HTDQ) method, designed for temporal features, which retains the high discriminability of quantized features, providing precise temporal guidance for all video frames. In addition, we present the Scattered Channel Range Integration (SCRI) method which aims to improve the coverage of quantization levels across individual channels. Experimental validations across various models, datasets, and bit-width settings demonstrate the effectiveness of our QVD in terms of diverse metrics. In particular, we achieve near-lossless performance degradation on W8A8, outperforming the current methods by 205.12 in FVD.

Main paper: 
 related work 2.1 video diffusion: 
 Recently, Diffusion Probabilistic Models [22, 50] have overtaken Generative Adversarial Networks (GANs) [13] as the leading approach in generative modeling, establishing a new benchmark for the field. Following the success of image diffusion techniques, video diffusion has also received widespread interest. VDM [23] becomes the pioneer in video generation and adopts 3D U-Net [12] structure. Some text-to-video (T2V) methods, such as MagicVideo [64] ,LVDM [21] utilize the Latent Diffusion Model (LDM) [46] and plug temporal modeling technique to it. Subsequent T2V schemes [5, 55, 55, 62] extend the single-stage to multi-stages. Image-to-video (I2V) methods, as another promising scheme, generate the video from a conditional image. Initially, LaMD [26] focuses on training an autoencoder to isolate motion information contained in videos. Stable Video Diffusion leverages text-to-image pretraining, video pretraining, and high-quality video finetuning to produce high-resolution videos. AnimateDiff [18] integrates the LoRA [24] and avoids the time-consuming retraining. Other conditions, such as pose [29, 38] , motion [7, 61] , sound [31, 36] are also proposed. Substantial efficient solutions, including retraining-free sampler [2, 35, 37] and retraining-based methods [51, 63] . The first aims to decrease the number of sampling steps and the second is time-consuming. However, there is a gap in the research on video diffusion, and our work is the first to undertake a study in this area.Quantization has achieved substantial advancements in the domain of neural network acceleration, as corroborated by numerous scholarly investigations [6, 10, 28, 34, 40, 42, 56, 58] . Mainstream quantization schemes can be briefly classified into two categories: quantizationaware training (QAT) [9, 15] and post-training quantization (PTQ) [6, 10, 40] . QAT aims to retrain the network on the whole dataset, while PTQ only requires a small amount of unlabeled datasets for calibration. Several classical quantization methods, such as MinMax [28] , Percentile [58] , LSQ [15] , PACT [9] are proposed successively for the convolutional neural networks. In recent years, the quantization of diffusion models [19, 27, 33, 47] has garnered widespread attention within the academic community. PTQ4DM [47] first discovers the difficulty of multiple-step activation distribution and generates the calibration data from a kew-normal distribution. Q-diffusion [33] introduces the uniform sampling calibration and split shortcut quantization for the bimodal activation distribution of the shortcut layers. PTQD [19] decomposes the quantization noise into interrelated and residual parts. TFMQ-DM [27] addresses the temporal feature disturbance and optimizes them separately. However, these existing methods mainly focus on image diffusion. In comparison, video diffusion necessitates significantly greater computational resources and storage. To the best of our knowledge, our work is the first to conduct the quantization for video diffusion models. 
preliminaries 3.1 diffusion models: 
 Diffusion models employ a sophisticated approach to image generation, relying on the application of Gaussian noise through a Markov chain in a forward process and a learned reverse process to generate high-quality images. Beginning with an initial data sample x 0 ∼ 𝑞(x) from a real distribution 𝑞(x), the forward diffusion process incrementally adds Gaussian noise over 𝑇 steps:EQUATIONwhere 𝑡 is an arbitrary timestep and {𝛽 𝑡 } is the variance schedule. The reverse process, in contrast, aims to denoise the Gaussian noise 𝑥 𝑇 ∼ N (0, I) into the target distribution by estimating 𝑞 (x 𝑡 -1 |x 𝑡 ). In every step of the reverse process, marked by 𝑡, the model estimates the conditional probability distribution using a network 𝜖 𝜃 (x 𝑡 , 𝑡), which incorporates both the timestep 𝑡 and the prior output x 𝑡 as its inputs:EQUATIONwhere 𝛼 𝑡 = 1 -𝛽 𝑡 and ᾱ𝑡 = 𝑇 𝑖=1 𝛼 𝑖 . When extending the image diffusion to video diffusion, the latent noise adds a new dimension 𝐾 which denotes the length of the video frames.Model quantization represents a technique for model compression, vital in optimizing neural networks for resource-constrained environments. Quantization transforms the network's weights and activations from a floating-point to a low-bit representation, thereby reducing memory footprint and computational intensity. This transformation is quantitatively described as follows:EQUATIONEQUATIONwhere 𝑤 and ŵ denote the original and de-quantized weights or activations, 𝑤 𝑞 is the quantized integer representation, 𝑠 represents the scaling factor, 𝑧 is the zero point, and 𝑏 is the bit precision corresponding to 2 𝑏 quantization levels. The uniform quantization here has equal intervals between each level. Quantization introduces an approximation (quantization) and a subsequent reconstruction (de-quantization) of network parameters.Expanding upon uniform quantization, our research incorporates logarithmic quantizer that also aligns well with the hardwareoriented aspects. For instance log2 quantization, used primarily on positive activation values, is succinctly represented as:EQUATIONEQUATIONThis method, like uniform quantization, involves the scaling factor 𝑠 but introduces a logarithmic approach to the quantization process. It offers rapid bit-shifting operations, making it a strategic choice for efficiently implementing models on hardware platforms. The integration of log2 quantization into our model framework further exemplifies our commitment to enhancing computational efficiency while maintaining fidelity in the intricate process of diffusion-based video generation.In the video diffusion model, the time step 𝑡 is encoded by the function ℎ (•) into a temporal encoding, which is then mapped to temporal features by the embedding function 𝑓 (•). These temporal features are channel-adjusted by the function 𝑔 (•) in every Resnet-block3D of the noise estimation network and fused with all frame features. Formally, for the 𝑖-th Resnetblock3D, this process can be described using the following equation:EQUATIONwhere F 𝑡 represents the frame feature fused with the projected temporal feature. We denote the temporal feature at time-step t as T 𝑡 𝑒𝑚𝑏 :EQUATION 
modifications 4.1 high temporal discriminability quantization: 
 As previously discussed, video diffusion models introduce a frame dimension, which enables the model to predict the noise for N frame features in each inference, a concept illustrated in Figure 2 .In contrast, in image diffusion models, the frame count remains fixed at one. Structurally, temporal features are integrated into the features of each frame, and the quantization noise consequently spreads across all frames. Features from different frames are further fused within the temporal attention blocks, which results in the effects of quantization being compounded. As indicated in Table 4 , omitting the quantization of temporal features leads to a significant reduction in the FVD by 160.82 compared to the uniform baseline. These findings highlight the critical importance of precise temporal features for video diffusion models.To further explore this, we investigate the temporal features to explain why uniform quantizers fail to function effectively. The distribution of temporal features demonstrates a pronounced skewness, with a majority of the values aggregating near zero, and outliers are several orders of magnitude greater than the typical values observed as depicted in Figure 4(a) . Even with a 10-bit uniform quantizer, dense intervals utilize only one of the 1024 quantization levels, as depicted in Figure 4(b) . This causes most values in the temporal features to collapse to a single value, as shown in Figure 3 (b), severely impairing their distinguishability. The log2 quantizer, as shown in Figure 4 (c), allocates more quantization levels to dense intervals, preserving the distribution of small values in the temporal features and thus their discriminability. As indicated in Table 4 , the log quantizer reduces FVD by 131.63 compared to the linear quantizer, demonstrating its effectiveness. However, we note that despite the improved FVD with the log quantizer, it incurs a greater L2 quantization loss compared to the uniform quantizer, as shown in Figure 5 . We hypothesize that L2 loss prioritizes the impact of larger values while neglecting smaller values, which is inappropriate given the unique distribution.We conduct comparative experiments to further validate the contribution of minor values to the discriminability of temporal features. Specifically, in setting 1, we zero the interval [-0.5𝑚𝑖𝑛, 0.5𝑚𝑎𝑥], and in setting 2, we apply a noise mask ranging from [-1.5, 1.5] to scale values within the [0.9𝑚𝑎𝑥, 𝑚𝑎𝑥] interval, where 𝑚𝑎𝑥 and 𝑚𝑖𝑛 denote the maximum and the minimum for the temporal feature, respectively. As detailed in Table 4 , the model exhibited robustness to disturbances in large values, while homogenization of small values lead to a collapse in model performance, underscoring the critical importance of minor values in preserving the discriminability of temporal features.This analysis indicates the necessity of designing a quantizer specifically for the unique distribution of temporal features and 4.1.1 Temporal Discriminability Score. The TDScore evaluates the similarity between the current temporal feature and several adjacent temporal features. We denote the discriminability score of the 𝑡-th temporal feature as TDScore 𝑡 . We initially apply a logarithmic function to T 𝑡 𝑒𝑚𝑏 to enhance focus on minor values within the temporal feature as defined in Equation 9:EQUATIONSubsequently, we compute the mean cosine similarity between the feature and its 𝑛 contiguous time steps following Equation 10:EQUATIONA lower TDScore indicates higher discriminability of the temporal feature. We can employ the TDScore to assess the efficacy of quantization precisely.Quantizer for Temporal Feature.As aforementioned, the quantization of temporal features not only necessitates minimizing quantization loss but also preserving distinctions between time steps. We evaluate quantizers calibrated using min-max or mean squared error (MSE) calibration strategy, as illustrated in Figure 5 . These quantizers minimize quantization loss but concurrently diminish the distinguishability of temporal features (TDScores are near 1). We introduce the high discriminability quantizer considering the unique distribution of temporal features. This quantizer is based on a logarithmic quantizer which is non-uniform.It allocates more quantization levels to values concentrated around zero, unlike uniform quantizers. Conversely, sparse distributions of large values are allocated fewer quantization levels. However, the vanilla logarithmic quantizer maps both the positive part and the negative part of temporal features to the same positive interval. This causes the data concentration regions of positive and negative intervals to overlap, exacerbating data concentration. Consequently, this reduces the utilization of available quantization levels. To address this issue, a relaxation coefficient 𝛽 is introduced, as follows: The quantization process is initiated by a 𝛽 shift in the temporal features, thereby reducing the clustering of absolute values. T 𝑧 𝑒𝑚𝑏 represents the quantized temporal features. The scaling factor 𝑠 is typically set to be greater than the maximum absolute value of T 𝑒𝑚𝑏 to ensure that the scaled activation values fall within the range [0, 1]. Additionally, 𝑠 can be adjusted to modify the data concentration, where increasing the value of 𝑠 narrows the range of activation values and makes it more concentrated.EQUATION) (a) (b) (c)As illustrated in Figure 5 , although the logarithmic quantizer ensures foundational temporal discrimination (low TDScore), it also incurs a considerable L2 loss (MSE loss). To balance the precision of quantization and temporal discrimination, we utilize a composite metric 𝐾 as defined in Equation 12, which incorporates both TDScore and L2 loss, to search the optimal 𝑠 and 𝛽:EQUATION)The TDScore allows 𝐾 to prioritize smaller values, while the L2 component ensures attention to losses in larger values. To mitigate truncation errors, We constrain the value of 𝑠 to the intervalEQUATION)This typically spans a very large range, to enhance search efficiency, we employ exponential step sizes where 𝑠 𝑖 = 𝑚𝑎𝑥 |T 𝑒𝑚𝑏 | × 2 0.05×𝑖 .Compared to the image diffusion models, we observe significant inter-channel variations in the video diffusion model. As depicted in Figure 1 of individual channels leads to minimal overlap in the activation ranges across channels, resulting in low coverage of quantization levels per-channel, as illustrated in Figure 6 , this diminishes the performance of the quantized model. Unlike the issues previously identified in large language models (LLMs), the problem of inter-channel variations is particularly pronounced in video diffusion models. This issue differs significantly from the outliers extensively studied in LLMs. In LLMs, as indicated by OS+ [57] , outliers manifest as extreme shifts in specific channels. These shifts are consistent across samples, which allows for the straightforward identification and adjustment of an accurate shift amount to align the channel midpoints. Such a method is effectively utilized in LLMs, where techniques like OS+ [57] align and scale channel midpoints in the activation of the Layer-Norm layer through a channel-wise shift to address these outliers. However, in video diffusion models, the challenge of inter-channel variations requires distinct approaches due to its more complex and variable nature compared to the channel-specific shifts observed in LLMs. Specifically, in video diffusion models, most channels exhibit varying degrees of shift that change with each sample, making it challenging to calculate an accurate shift for aligning the channel midpoints.To address this issue, we propose the method of Scattered Channel Range Integration (SCRI). Formally, SCRI is a straightforward approach, as illustrated in Equation 14:EQUATION)where X denotes the activations with 𝐶 channels, while s, a vector of length 𝐶 similar to that used in OS+ [57] , is employed to adjust the range of activation values. X 𝑐𝑚𝑎𝑥 represents a vector composed of the maximum activation values within each channel, and 𝑡 serves as an adaptive parameter.The SCRI, through meticulous design, can amplify the range of activation values for each channel, reducing discreteness and increasing overlap. However, an excessively large activation range is not conducive to improving the coverage of quantization levels for individual channels. To strike a balance between the activation range and the coverage of quantization levels, identifying the optimal value for 𝑠 is essential. Specifically, through a forward pass, we use a calibration set to determine the maximum activations values X 𝑐𝑚𝑎𝑥 . Concurrently, we determine the optimal 𝑡 using a grid search approach, confining our search within [𝑚𝑖𝑛(X 𝑐𝑚𝑎𝑥 ), 𝑚𝑎𝑥 (X 𝑐𝑚𝑎𝑥 )]. The optimization criterion during this search is the minimization of the MSE loss, comparing the output of the quantized model block against that of the full-precision model, as follows:EQUATIONwhere F represents the mapping function for the layers following LayerNorm in temporal attention modules and self-/cross-attention modules in the video diffusion model and F 𝑄 denotes the mapping function corresponding quantized module. Specifically, we identify several layers to apply SCRI: the Feed Forward Network (FFN), the projection layer, typically a linear layer or a convolution layer, and the attention layer subsequent to the LayerNorm. Similar to OS+ [57] , we implement SCRI by making equivalent transformations to LayerNorm and subsequent layers, which incurs no additional overhead during inference. As depicted in Figure 6 , SCRI significantly enhances the coverage of quantization levels by individual channels. 
experiments 5.1 implementation details: 
 Datasets and Quantization Settings. Video synthesis experiments are conducted on two cutting-edge models, MagicAnimate [60] and AnimateDiff [17] , utilizing the TED-talks [48] , FS-COCO [11] and COCO Captions [8] datasets. Both the input and output layers are consistently set at an 8-bit representation, whereas all remaining convolutional and linear layers are quantized in accordance with the predetermined target bit-width. To calibrate the models accurately, samples from all time steps of application (25 in this work) are collected to form a calibration set, corresponding to one video consisting of 16 consecutive frames. This process ensures that the models are finely tuned for generating high-quality video sequences, establishing a benchmark in the domain of video synthesis. Evaluation Metrics. In our experimental framework, we report the spatiotemporal fidelity of video synthesis using the FID-VID [1] and FVD [52] metrics, combining image quality assessment through Fréchet Inception Distance with temporal coherence evaluation using Fréchet Video Distance. Following AnimateDiff [18] , semantic integrity is evaluated using the CLIP metric [45] , which leverages a sophisticated language-image pretraining model to measure the alignment between generated animations and reference imagery. This involves calculating the cosine similarity between CLIP image embeddings of each animation frame and reference images, thereby evaluating domain similarity (Domain). Furthermore, our assessment is enhanced by examining the similarity between the prompt embeddings and individual frames to assess text alignment (Text) and by analyzing the similarity between consecutive frames to evaluate motion smoothness (Smooth). For computational performance, we calculate Bit Operations (BOPs) [3, 19] per video diffusion model during each forward pass, balancing efficiency with processing demands.Motion Guided. In this section, we apply our quantization method to the task of Human Image Animation, utilizing the MagicAnimate [60] framework on the TED-talks dataset [48] . MagicAnimate uses a motion sequence and a reference image to generate a video clip where the sequence directs the person's movements. We use the plain round-to-nearest Linear Quantization [43] , PTQ4DM [47] and Q-Diffusion [33] as our baselines. As shown in table 1, our QVD consistently outperforms other methods by a large margin. Actually, these methods specifically designed for image diffusion, namely PTQ4DM and Q-Diffusion, do not achieve significant performance improvements in video diffusion quantization. On the contrary, at W8A8, our QVD gains a FID-VID reduction of 25.78 ( 75.16→49.38 compared with Q-Diffusion) and a FVD reduction of 205.12 (590.89→385.77 compared with PTQ4DM) on TED-Talks, which is almost lossless. When the bit-width decreases (i.e., W6A8 and W6A6), other methods drop drastically while our QVD maintains the performance to a great extent, merely introducing a 1.56 upswing in FID-VID and a 0.7 upswing in FVD. In our ablation studies, we meticulously dissect the impact of each component in our QVD. These experiments are performed using the TED-talks dataset with W8A8 quantization, aiming to elucidate the individual contributions of different components to the overall effectiveness of the model. Linear quantization served as the baseline, adopting the MinMax calibration strategy for weight quantization and the MSE calibration strategy for activation quantization. Our analysis involves examining the effects of SCRI and HTDQ, as detailed in the following.Overall Effect of HTDQ and SCRI. Table 3 outlines the collective impact of these components. The baseline model with linear quantization scores an FID-VID of 80.70 and an FVD of 618.33. Incorporating HTDQ remarkably improves these metrics to 61.22 and 483.99, respectively. Similarly, adding SCRI yields an improvement, achieving scores of 67.57 (FID-VID) and 529.70 (FVD). The Detailed Ablation of HTDQ. To dissect the influence of HTDQ, we consider two stages in its integration (Table 4 ): the log quantization and our enhanced HTDQ. The experimental data presented in Table 4 clearly indicates the superiority of our enhanced HTDQ over the log quantization. When comparing the SCRI+Log2 Quant method and that with full SCRI+HTDQ, there is a noticeable improvement in both FID-VID and FVD scores, dropping from 50.43 to 49.38 for FID-VID, and from 398.07 to 385.77 for FVD. This demonstrates that our HTDQ, with its additional refinements, effectively enhances the quality of the quantized animations, offering a more faithful representation compared to the baseline.Exploring the Impact of SCRI. In our ablation study detailed in Table 5 , we compare the impact of different scaling methods on quantization quality. When the shift operation is removed from the OS+ [57] method, we see a modest improvement in both FID-VID (reduced from 51.21 to 50.48) and FVD (lowered from 407.93 to 405.16). Further refining the scale computation in our SCRI method yields even better results, with a notable decrease in FID-VID to 49.38 and FVD to 385.77, demonstrating the effectiveness of our adjustments in scale calculation for quantization.Investigating Calibration Efficiency. To enhance the efficiency of the search process of SCRI, we investigate the impact of varying the number of frames and time steps on the calibration outcomes.As shown in Table 6 , a calibration setup using 16 frame features and 1 time-step achieves a balance between performance and time efficiency. All the experiments are conducted based on this setup.Figure 7 shows the W8A8 qualitative results on FS-COCO and TED-talks datasets. When the reference is sketch (top), Linear Quantization and Q-Diffusion exhibit grid-like artifacts in their prediction results, while QVD successfully eliminates this issue and more closely approximates the performance of a full-precision model. When the reference is the motion video (bottom), Q-Diffusion generates noise and disordered backgrounds in its outputs. Moreover, QVD demonstrates superior detail retention for moving objects, exemplified by the hand of the speaker in the lower right corner. 
conclusion: 
 In this work, we explore the application of post-training quantization in video diffusion models. We identify the significance of distinctive features for high-quality video generation and introduce the HTDQ method to preserve the discriminability of temporal features after quantization. Additionally, we observed a severe inter-channel variation issue in video diffusion models. To address this, we proposed the SCRI method to integrate activations across channels, thereby enhancing the performance of the quantized model. Our proposed QVD quantization framework is the first to quantize video diffusion models to 8-bit without significant performance degradation.  
: 
 Despite the remarkable effects of diffusion models, their relatively slow inference speed and substantial memory usage have hindered their broader application, particularly in video tasks. The core reasons for these limitations include: 1) The denoising procedure encompasses several hundred iterations, and 2) The extension of frame dimensions results in a notable escalation in memory utilization compared with image diffusion models. There are primarily two strategies to overcome such bottlenecks: minimizing the number of iteration steps, including [37, 51] , and optimizing the efficiency of individual denoising operations through techniques like pruning [16, 32] , distillation [30, 39] , or quantization [33, 47] . The former only focuses on the first issue while ignoring the significant memory consumption. In this work, we mainly study the quantization of video diffusion models.Model quantization is a widely adopted and practical approach for reducing model footprint and accelerating inference by mapping floating-point values into low-bit integers. Among various quantization methods, post-training quantization (PTQ) requires no retraining or fine-tuning of the model and incurs minimal overhead, making it more practical in deployment. PTQ methods have been extensively studied in image diffusion models [19, 27, 33, 47, 49] . However, these methods exhibit significant performance degradation when directly applied to video diffusion models. We discover that the rationale lies in two aspects, i.e., the introduction of the frame dimensions and temporal attention modules in video diffusion models. Specifically, based on the following two observations, we explore the difficulty of quantization for VDMs:Observation 1: Highly reliant on discriminable temporal features. As illustrated in Figure 1 (a), in a single denoising iteration, the features of all frames rely on the same temporal feature, which implies that disturbances arising from the quantization of temporal features will impact the generative quality of all frames. Furthermore, we observe that the uniform quantizer leads to the homogenization of temporal features, demonstrating a significant performance gap compared to those utilizing full-precision temporal features. Upon analyzing the distribution of temporal features, we observe a pronounced skewness near zero, where outliers often exceed thousands of times the magnitude of regular values, making conventional uniform quantizers unsuitable for temporal features.Observation 2: Inter-channel variations reduce the coverage of quantization levels. As depicted in Figure 1 (b), in the image diffusion model, activation across different channels tends to be concentrated, with the range of activation for individual channels closely approximating the range of overall activation. Each channel covers nearly all quantization levels, indicating low quantization difficulty. In contrast to the image diffusion model, the activation values of the temporal attention module in video diffusion models are discrete and asymmetric across channels. The range of individual channels is significantly narrower than the overall activation. As a result, each channel accesses only a tiny fraction of quantization levels, posing a new challenge for PTQ.To tackle these obstacles, we propose QVD, the first post-training quantization scheme for video diffusion models. Figure 2 shows the overall pipeline of the QVD. To mitigate the problem in observation 1, we introduce the High Temporal Discriminability Quantization (HTDQ), which contains a High Discriminability Temporal Quantizer (HiDi-TQ) to prioritize numerous near-zero values and retains high identifiability of time, and the Temporal Discriminability score (TDScore) to evaluate the similarity of adjacent temporal features. To settle the issue in observation 2, we introduce a Scattered Channel Range Integration (SCRI) method, which employs a per-channel integration operation to enhance the coverage of quantization levels by individual channels, therefore handle the discrete range of activations. We conduct comprehensive experiments to validate the superiority and versatility of our QVD.In summary, our contributions are as follows:• We propose QVD quantization framework, which, to our knowledge, is the first PTQ method tailored explicitly for video diffusion models. • We identify the critical inter-channel variations issue in video diffusion models and highlight the significance of accurate and discriminable temporal features for video generation. • We introduce SCRI to improve the coverage of quantization levels across individual channels, TDScore to quantify temporal similarity, and the HiDi-TQ quantizer to keep high discriminability of temporal features. • Extensive experiments on various models and datasets demonstrate the superiority of QVD, which results in a 257.9 decrease in the FVD for the W6A8 PTQ of video diffusion models compared to existing methods in the image domain. 
 
 Appendix:  


Decision: This work presents an approach to post-training quantization for video diffusion models, with nearly no degradation under the w8a8 setting. The paper is well-written, and the approach is innovative. The rebuttal has addressed the concerns of all the reviewers. After the rebuttal, the reviewers are positive about the work. Thus the AC recommendation acceptance of this work.

Reviews: review: paper_summary: paper_summary: This paper presents QVD, a pioneering post-training quantization approach tailored for video diffusion models. It introduces two key methods: High Temporal Discriminability Quantization (HTDQ) for maintaining temporal features and Scattered Channel Range Integration (SCRI) to enhance quantization level coverage across channels.
strength_weakness: strength_weakness: 1. **Innovativeness**: The paper proposes QVD, the first quantization framework specifically tailored for video diffusion models, afilling a notable gap in current research.
2. **Clear Structure**: The paper is well-structured and clearly written, making it easy to follow and understand the proposed methods and results.
limitations: limitations: **(Major)** As a work related to video generation, I did not see any video materials in the supplementary content.

**(Minor)** Given the complexity of the QVD, it is unclear whether it can serve as a convenient, plug-and-play plugin for existing video diffusion models, potentially limiting its ease of adoption and practical utility.

**Recommendations**:

Include Video Materials: To better demonstrate the effectiveness and practical utility of the proposed methods, it is essential to include relevant video materials. I recommend providing comparisons of the quantized and original models across different video diffusion methods, as well as comparisons with previous approaches. This is necessary because static images can be cherry-picked, making it difficult to assess the true quality of the video.

Conduct Portability and Generalizability Experiments: Consider conducting experiments that demonstrate the portability and generalizability of QVD across more video diffusion models and datasets. This would help in understanding the adaptability of QVD in diverse settings. Given the workload, it is acceptable not to provide comparisons with previous methods, but it is still necessary to show comparisons between the quantized and original models.
paper_summary: paper_summary: The authors propose a novel method for post-training quantization (PTQ) of video diffusion models. They first explore failures of existing PTQ and identify two main concerns, namely quantization of temporal feature activations into mostly one bin, and significant inter-channel variation of activations which leads to many bins having no quantized values. The authors then propose methods to resolve these issues, namely a novel quantizing strategy HiDi-TQ that spreads activations into more bins, which is calibrated according to a novel TDScore (which measures temporal discriminability), and SCRI, which scales activations to use a wider range of quantization bins between channels. Their proposed method is extensively evaluated and they show highest performance in generation compared to other PTQ methods.
strength_weakness: strength_weakness: * Insightful experiments to understand the drawbacks of PTQ on video diffusion models
* Creative and novel solutions to these issues, via the TDScore, HDTQ, and SCRI
* Extensive evaluations
* Good performance
limitations: limitations: * I still have a bit of confusion on the two drawbacks. It seems both lead to only a few quantization bins having values, and I find that the motivations/reasonings for the proposed method (specifically SCRI) are only explained at a high level. I would appreciate a deeper explanation, although I am not so familiar with LLM quantization which may be the reason for my confusion.
* Are HiDi-TQ, HTDQ, and high discriminability quantizer for temporal feature all the same thing? It should be given one name or explained more clearly if they are different.
* Fig. 1 is confusing, it would be better with labels and I think it could be laid out more clearly.
paper_summary: paper_summary: This paper proposes QVD, the first post-training quantization scheme for video diffusion models. The paper introduces High Temporal Discriminability Quantization (HTDQ), which includes a High Discriminability Temporal Quantizer (HiDi-TQ) that prioritizes numerous near-zero values and maintains high temporal identifiability, as well as a Temporal Discriminability score (TDScore) to evaluate the similarity of adjacent temporal features. Additionally, the paper introduces the Scattered Channel Range Integration (SCRI) method, which enhances the coverage of quantization levels by individual channels through per-channel integration operations, thereby addressing the discrete range of activations.
strength_weakness: strength_weakness: (1) This paper proposes the QVD quantization framework, which is the first PTQ method tailored explicitly for video diffusion models. 

(2) This paper identifies the critical inter-channel variations issue in video diffusion models and highlights the significance of accurate and discriminable temporal features for video generation. 

(3) It introduces SCRI to improve the coverage of quantization levels across individual channels, TDScore to quantify temporal similarity, and the HiDi-TQ quantizer to maintain high discriminability of temporal features.
limitations: limitations: **Limited Generalizability**: The paper focuses mainly on video diffusion models, leaving its applicability to other types of video models or neural networks unverified.

**Implementation Complexity**: The QVD framework introduces multiple components, such as SCRI, TDScore, and HiDi-TQ, which could complicate implementation and integration into existing systems.
paper_summary: paper_summary: This paper investigates post-training quantization for video diffusion model. They design the first video-level quantization method, which consider high reliant on discriminable temporal features and inter-channel variations. Specifically, they introduce SCRI to improve the coverage of quantization levels across individual channels, TDScore to quantify temporal similarity, and the HiDi-TQ quantizer to keep high
discriminability of temporal features. Extensive experiments on various video diffusion models and datasets are conducted to demonstrate the effectiveness of the proposed method to improve the efficiency of VDMs while maintaining decent generation performance.
strength_weakness: strength_weakness: The motivation is greatly described. It is an interesting work to investigate pos-training quantization for video diffusion models.
This work is the first to conduct the quantization for VDM. It can be a plug-and-play process to accelerate any on-the-shelf VDMs.
They introduce TDScore to quantify temporal similarity and HiDi-TQ to keep high discriminability of temporal features, which are reasonable.
limitations: limitations: 1. In Fig. 6, the relation between the three groups should be better explained
2. More visualization results from the Magic-Animate are appreciated because the improvement on the first case seems slight. Also, The abbreviation should be described in Fig. 7, e.g., 'FP'.
3. The ablation study on log2 quantization seems missing.