FasterCache: Training-Free Video Diffusion Model Acceleration with High Quality
Authors: Zhengyao Lv
Chenyang Si
Junhao Song
Zhenyu Yang
Yu Qiao
Ziwei Liu
Kwan-Yee K. Wong
Venue: ICLR 2025 Poster (2025)
Keywords: Efficient Video Synthesis
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: In this paper, we present \textbf{\textit{FasterCache}}, a novel training-free strategy designed to accelerate the inference of video diffusion models with high-quality generation. By analyzing existing cache-based methods, we observe that \textit{directly reusing adjacent-step features degrades video quality due to the loss of subtle variations}. We further perform a pioneering investigation of the acceleration potential of classifier-free guidance (CFG) and reveal significant redundancy between conditional and unconditional features within the same timestep. Capitalizing on these observations, we introduce FasterCache to substantially accelerate diffusion-based video generation. Our key contributions include a dynamic feature reuse strategy that preserves both feature distinction and temporal continuity, and CFG-Cache which optimizes the reuse of conditional and unconditional outputs to further enhance inference speed without compromising video quality. We empirically evaluate FasterCache on recent video diffusion models. Experimental results show that FasterCache can significantly accelerate video generation (\eg 1.67$\times$ speedup on Vchitect-2.0) while keeping video quality comparable to the baseline, and consistently outperform existing methods in both inference speed and video quality. \textit{Our code will be made public upon publication.}

Published as a conference paper at ICLR 2025
FASTERCACHE: TRAINING-FREE VIDEO DIFFUSION MODEL ACCELERATION WITH HIGH QUALITY
Zhengyao L
Chenyang Si
Junhao Song
Zhenyu Yang*
Yu Qiao Ziwei L
Kwan-Yee K. Wong
The University of Hong Kong 2S-Lab, Nanyang Technological University
Shanghai Artificial Intelligence Lahoratory
Code: https: hypps:///ithbbb com htt////hhctt/ttteer//eco
ABSTRACT
In this paper, we present FasterCache, novel training- free strategy designed accelerate the inference of video diffusion models with high-quality genera- tion. By analyzing existing mache-based methods, we observe that directly reusing adjacent-step features degrades video quality due to the loss of subtle variations. We further perform pioncering investigation of the acceleration potential of classifier-free guidance (CFG) and reveal significant redundancy between condi- tional and unconditional features within the same timestep. Capitalizing these observations, we introduce FasterCache to substantially accelerate diffusion uiffusion-based video generation. Our key contributions include dynamic feature that preserves both feature distinction and temporal continuity, and CFG-Cache which optimizes the reuse of conditional and unconditional outputs to further hance inference speed without compromising video quality. We empirically evalu- ate FasterCache on recent video diffusion models. Experimental results show that FasterCache can significantly accelerate video generation (e.g., 1.67 x speedup on Vchitect-2.00 2.0) while keeping video quality comparable the baseline, and consis- tently outperform existing methods in both inference speed and video quality. Original
strategy
AITI
PAB
Ours
Open-Sora
Lab 192.07
Lab 1601 LMISS 0332
Lat: 156.7% 73 LPIPS 0152
Lat: TR44 LPISS 0MD
Open-Soraa Plan
LPIPSS
LPIPS
LPIPS
Latte
Lat: 23222
23,0, LPIPS 01246
22.8%, LPIPS 01330
18.90, LPIPS 0.022
latency. messured single Video confgurratin:
480P Open-Sura, frames at 512x Open-Som-Plaan Som--lan, and 16 frames at 512x 512 Laile)
Figure 1: Comparison of visual quality and inference speed with competing methods.
INTRODUCTION
Diffusion transformers (DiT) (Peebles & Xie 2023) have achieved notable success in image Chen et al. 2023 2024b Esser et al.) 2024) and video generation (Ma et al. 2024al Zheng et al.) 2024 PKU Yuan Lab and Tuzhan Al etc. 2024). attracting significant attention for their potential. Al-
though iterative denoising, classifier-free guidance (CFG) (Ho & Salimans 2022), and transformer
Corresponding authors. Project leader. *The work was done during an internship Shanghai AI Lah.

---

Published as a conference paper at ICLR 2025
Iteration denolisin
Ieeraioo denoising
Itration dencising
X
X1-1
Feature
Atsercion
TOUSO
Attertion
Anssntion
IFN
PFN
FFN
Figure 2: Vanilla cache-based acceleration method typically reuses features cached from previous timesteps directly for the current timestep.
attention mechanisms have significantly improved the generative capabilities of diffusion models, they also lead to substantial computational costs and increased memory requirements for inference, especially for video generation which typically takes 2-5 minutes synthesize 6-second 480P video, limiting their practical use. This calls for the development of new techniques that require less computational cost for diffusion models (Salimans & Ho 2022 Ma al., 2024b) Chen al., 2024c Zhao et al. 2024c).
Among the recently proposed solutions, cache-based acceleration has emerged one of the most widely adopted approaches. This approach speeds up the sampling process by reusing intermediate features across timesteps, thereby reducing redundant computations and significantly improving computational efficiency. Besides, it requires no additional training costs for inference acceleration and offers straightforward generalization to other video diffusion models. Examples include cache- based methods for U-Net based diffusion models (Ma et al. (002bb Li et al. (023b), residual caching A-DiT (Chen et al., 2024c) for transformer based diffusion models, and hierarchical attention caching of PAB (Zhao al., 2024c) for video generation. Despite their proven effectiveness, there exist two critical concerns: 1) Whether directly reusing intermediate features aligns with the iterative denoising mechanism, considering the inherent feature variations between timesteps. 2) Current cache- based methods focus primarily on the attention features within the transformer networks, with limited exploration of accelerating different parts of the pipeline. In this work, aim address these two concerns.
To thoroughly investigate the acceleration potential of DiT inference for video generation, we delve into the feature reuse process of existing cache-based methods. As shown in Fig. 2 these meth- ods typically assume high degree of feature similarity between adjacent timesteps in the itera- tive denoising process, and achieve accelerated inference by sharing features across consecutive timesteps. However, our investigation reveals that while features in the same attention module (e.g., spatial attention) appear to be nearly identical between adjacent timesteps, there exist some subtle yet discernible differences. As a result, a naive feature caching and reuse strategy often leads to degradation of details in generated videos, shown in Fig. (a).
Following this analysis, we further extend the scope of our investigation to explore potential redundancy within the classifier- frree guidance (CFG). As shown Fig. (bg.).)) compared to internal network modules (e.g., spatial atten- tion and temporal attention). CFG almost doubles the in- ference time due to the additional computation required for unconditional outputs. Our experiments reveal a no- table difference from our earlier conclusion regarding attention modules. In CFG, the conditional and un- conditional outputs at the same timestep exhibit very high degree of similarity, suggesting significant infor- mation redundancy. In contrast, the similarity of un- conditional features between adjacent timesteps is rela- tively weak. We further discover that the differences be- tween the conditional and unconditional outputs are pre- dominantly concentrated in low- mid-frequency fea- tures during the mid-sampling phase, shifting to high- frequency features in the samplin phase, with these differences evolving gradually.
Original
Cache
conditional
-unsondiiinal
unsonditooal
66
50k
(b)
Figure 3: (a) Vanilla cache-basee meth- ods typically lead to detail loss. (b) Time overhead proportions of different components in video models.
Based on the above insights, we propose a novel strategy, termed FasterCache, to accelerate the inference of video diffusion models while ensuring high-quality generation and remaining training- free. Specifically, we first introduce a dynamic feature reuse strategy for attention modules which dynamically adjusts the reused features across different timesteps, ensuring both distinction and
2

---

Published as a conference paper at ICLR 2025
continuity of features between adjacent timesteps maintained. This strategy preserves the subtle variations essential for the iterative denoising process while ensuring temporal consistency, resulting in accelerated inference with minimal loss of details in the generated videos. Furthermore, introduce CFG-Cache, an innovative technique that stores the residuals between conditional and unconditional outputs, dynamically enhancing their high-frequency and now-frequency components before reuse. This significantly accelerates inference while preserving details in generated videos.
We evaluate our FasterCache on various video diffusion models, including Open-Sora 1.2 (Zheng et al. 2024), Open-Sora-Plan (PKU- Yuan Lab and Tuzhan Al etc. 2024). Latte (Ma et al. 2024a Cog VideoX X Yang et al. 2024). and Vchitect-2.0 (Fan et al. 2025). As shown in Fig expen- mental results demonstrate that FasterCache can sigmificantly accelerate inference while preserving high-quality video generation across all tested models. Specifically, on Vchitect-2.0, FasterCache achieves 1.67 x speedup, with performance comparable to the baseline (VBench: baseline 80.80% - FasterCache 80.84%) Furthermore, our method outperforms existing approaches both infer- ence speed and video generation quality, highlighting its effectiveness and efficiency in real-world applications.
Overall, the contributions of this work are follows:
We analyze the feature reuse process in tache-based methods and discover that while adjacent-step features in attention modules appear to be similar, their subtle differences can degrade output quality if ignored.
We conduct pioneering investigation of CFG's potential for acceleration, finding high redundancy within the same timestep but weaker similarity across adjacent timesteps, re- vealing new acceleration opportunities.
We propose FasterCache, a training-free strategy that dynamically adjusts feature reuse, preserving both feature distinction and continuity. It also introduces CFG-Cache to accel- erate inference while preserving details in generated videos.
We empirically evaluate our approach various video diffusion models, demonstrating sigmificant improvement in inference speed while maintaining high video quality.
METHODOLOGY
2.1 PRELIMINARY
Diffusion model is a generative model consisting of a forward process and reverse process. Specif- ically, its forward diffusion process progressively adds noise to the data 26 ~ Patata (mr), eventually destroying the signal. This can be formulated as:
q(>, NN(xi Varzo, vI-
where {ar)T- controls the noise schedules and T represents the total number of diffusion timesteps. The reverse process is typically parameterized as a UNet or transformer architecture eg which is trained predict the noise with the following loss function:
CDM = En Lol..ll...,,,llle - eg (x t)
(2)
A clean signal 2p can be recovered through iterative inference steps which predict æ;-1 from using co. This can formulated as:
p(z,-1 (x,) = 1: t), Eg (2,, t)),
where Ae and Eg are the mean and variance parameterized with learnable 0.
Video diffusion models recently employ diffusion transformers as the backbone for noise predic- tion. This work explores vidco synthesis acceleration based on Open-Sora 1.2 (Zheng et al. [2024). This model is composed of 56 stacked transformer layers, with alternating spatial and temporal lay- Each layer contains not only spatial or temporal attention module but also cross-ateention and a feed-forward network. Latte (Ma et al. 2024a) and Open-Sora- Soann--or--Phhae Plaa (PKU- Yuan Lab and Tuzhan etc. 2024) also adopt similar architecture their noise prediction networks.
Classifier-Free Guidance (CFG) has proven to be a powerful technique for enhancing the quality of synthesized images/videos in diffusion models. During the sampling process, CFG computes two
3

---

Published as a conference paper at ICLR 2025
outputs, namely ea (a c) for the conditional input and ca(2,, 0) for the unconditional input (often an empty or negative prompt). The final output is given by:
Éo De, c) = +
91+ () c) - geg (z,, 9),
(4)
where g is the guidance scale. As shown in Fig. B (b), while CFG significantly enhances visual
quality, it also increases computational cost and inference latency due to the additional computation required for unconditional outputs.
2.2 RETHINKING ATTENTION FEATURE REUSE
Attention feature reuse has become a primary focus for cache-based acceleration methods in video generation (e.g., pyramid attention reuse of PAB). In video diffusion models, features of attention modules (e.g., spatial attention and tem- poral attention) exhibit high similarity between adjacent timesteps, as illustrated in Fig. 4 Hence, existing methods completely bypass the attention computations in subsequent timesteps by reusing the cached attention features, thereby significantly reducing computational costs.
Cross Altencion Feature
temporal Atsention Teature
Snartsl Athert on Feature
FFN Feature
Featurg MSE
To gain a better understanding of the implications of atten- tion feature reuse in video generation, we first visualize the videos generated with the same random seed and observe that existing feature reuse methods result a noticeable loss of details in the output. For example, illustrated the original video generated with- out feature reuse, the video generated with vanilla feature reuse exhibits a smoother sky, with a lack visible stars, indicating noticeable degradation in fine details.
Sampling Progress (86)
Figure Comparison of the mean squared error (MSE) of attention fea- tures between the current and previ- ous diffusion steps. Smaller values indicate higher similarity.
Fig. 5] compared
Original
Varilla Fenture Rouse
Skep21
Skyp22 Ste221
Sleg22
Slep23 Step22
Step23
Figure 5: Visual quality degradation caused by Vanilla Feature Reuse (left) and feature differences between adjacent timesteps (right).
To investigate the underlying causes of this phenomenon, subsequently visualize the attention features between adjacent timesteps and analyze their differences. The results indicate that while the attention features between adjacent timesteps highly similar, there exist noticeable differences between them. These subtle variations between timesteps are essential for preserving fine details in video generation. Therefore, directly reusing features without accounting for these differences leads to the loss of important visual information, resulting in smoother and less detailed outputs. This highlights the need for a more refined approach feature reuse, i.e., one that can retain computational efficiency while preserving key inter-step variations.
2.3 FEATURE REDUNDANCY IN CFG
Following the observation of feature redundancy in attention modules across adjacent timesteps. we further extend our investigation into other critical components of the diffusion models. Through this broader analysis of the entire denoising process, find that classifier-fres guidance (CFG) signifi- cantly increases inference time, as it requires the computation both conditional and unconditional outputs at every timestep. While CFG has been widely adopted for enhancing visual quality, there little exploration to reduce its computational burden, leaving this aspect largely uncharted.
To explore potential redundancy within CFG, first conduct quantitative analysis of the similar- ity between conditional and unconditional outputs at the same timestep as well as across adjacent timesteps based mean squared error (MSE). As shown in Fig. 6 (a), the results reveal that, in

---

Published as a conference paper at ICLR 2025
MSEEconditl, unconditi) Milll uncondity unconddt-
Rosurr MSE
Sampling Progress (56) (a) Festure MSE Curve between CRG Outputs
Odiginl
Reuse unonditionl ouputs duectly
(b) Direct masse of uncoddiiional outputs degrades visual quality
Figure 6: (a) The MSE between conditional and unconditional outputs the same timestep as well as across adjacent timesteps. (b) Directly reusing unconditional outputs from previous timesteps will lead significantly degraded visual quality.
the mid to later stages of sampling, the similarity between conditional and unconditional outputs at the same timestep is remarkably high, significantly surpassing that of adjacent steps. Hence, as illustrated in Fig. (b), directly reusing unconditional outputs from adjacent timesteps, suggested in existing methods, leads to significant error accumulation, resulting
decline video quality. These results indicate substantial redundancy the CFG process and highlight the necessity for new strategy to accelerate CFG without compromising the quality of the generated outputs.
Original
Conditioaal Output
CFG-Cache
(a) Visualization results of different methods avoid CFG redundancy
(b) MSE different Erequency features bias
Figure 7: (a) Simply reusing the conditional output from the same time step results in the poor gen- eration of intricate details. (b) Trend curves high and low-frequency biases between conditional and unconditional outputs change sampling progresses.
2.4 FASTERCACHE FOR VIDEO DIFFUSION MODEL
Capitalizing on the above discoveries, we introduce innovative approach, FasterCache, which accelerates inference for video diffusion models while preserving high-quality generation. This accomplished through Dynamic Feature Reuse Strategy that maintains feature distinction and temporal continuity. Furthermore, we introduce CFG-Cache to optimize the reuse conditional and unconditional outputs, further enhancing inference speed without compromising visual quality,
Dynamic Feature Reuse Strategy As discussed in Section 2.2 vanilla attention feature reuse strategy neglects the feature differences between adjacent timesteps which leads visual quality degradation. Hence, instead of directly reusing previously cached features at the current timestep, propose Dynamic Feature Reuse Strategy that can more effectively capture and preserve critical details in the generated videos. Specifically, for the attention modules in diffusion models, we compute the attention module outputs every alternate timestep. For example, we calculate the attention outputs for each layer at + 2 and t timesteps, denoted as Fi+2 and Fú, and store them in the feature cache as Ft-2 and Ft Ff To dynamically adjust feature reuse, we compute the difference between the adjacent cached features. This serves bias for approximating the feature variation trend and enables the reused features to more accurately capture the evolving details across timestep, its features can be computed as:
timesteps. For the intermediate
F Flache -F
cacha! * w(t).
(5)
where w(t) is a weighting function that modulates the contribution of the feature difference to ac- count for variation between adjacent timesteps, ensuring both efficiency and the preservation fine details in the generated videos. In our experiments, w(t) gradually increases as the sampling pro- cess progresses, allowing the model to place greater emphasis on the feature differences at later stages of generation. Further discussions the design of feature bias term and the selection of w(t) in Eq. (5) can be found in Appendix A. Consequently, our approach significantly accelerates inference while preserving the visual quality of the synthesized videos.
5

---

Published as a conference paper at ICLR 2025
step
step
n step
Xt
X₁
x
xt-
DiT
DiT
DiT
DiT
60% oe 1,0)
40 of 1.0)
f0 xn n.))
Fol%
Agr
Figure 8: Overview of the CFG-Cache. CFG-Cache accelerates the computation of the uncondi- tional output (in the dashed orange box) by caching the high- and low-frequency biases between the conditional and unconditional outputs, and dynamically enhancing them during reuse.
CFG-Cache As analyzed in Section 2.3 the conditional and unconditional outputs at the same timestep exhibit high similarity in CFG, indicating significant information redundancy. A naive approach to take advantage of this would be to directly reuse the conditional features for the cor- responding unconditional outputs the same timestep. However, this often leads noticeable degradation in detail generation. As illustrated in Fig 0 (a). this approach results in poor generation of intricate details, such the texture of the spacesuit which shows lack of details and clarity. Since both the conditional and unconditional outputs in CFG represent predicted noise, and drawing inspiration from the Dynamic Feature Reuse Strategy and FreeU (Si al., 2024), we analyze the differences between these two outputs in the frequency domain. In Fig 7 (b), we observe that, from the activation of CFG-Cache until the end of the sampling, the difference between the conditional and unconditional outputs gradually shifts from being dominated by low-frequency components to being dominated by sehhffreuency components. This indicates that the effects of CFG in the sam- pling process is primarily to influence perceptual features like layout and shape during the early and mid-stages, while contributing to detail synthesis in the later stages. A similar phenomenon can also be observed in HHiao et al. (022).. This observation suggests that despite their overall similarity, key differences in frequency components must be addressed to avoid the degradation of fine details. be found in Appendix A.3.2.
More discussion and visualization
Building on this discovery, we propose CFG-Cache. high- and low-frequency biases, coupled with a timestep-adaptive enhancement technique. Specif- full inference is performed obtain both the conditional output eæ (x,, t, c) and the unconditional output eœ( ł, 0). We then separately calculate the biases for the high-frequency (F and cow-frequency (F components between these two outputs: f, Yow.
novel approach designed to account for both
ically, shown in Fig. [ at timestep
ALF FFT (Ca (Co (Iz, t, 0))
A HFF FFT (E) 0) high (F, (,) hight
(7)
These biases ensure that both high- and low- frequecyyenc differences are accurately captured and com- pensated during the reuse process. In the subsequent n timesteps (from t 1 to t n2), we infer only the outputs of the conditional branches and compute the unconditional outputs using the cached AHF and &rr as follows:
Eo 6o( t i,0) IFFT (Foow Frigh):
(8)
Frow ALF wi +
c))tow:
(9)
Frigh AHF * W2 + FFF (2, t i, c,c))
(10)
Here, wi and wy adaptively adjusted based the sampling timestep with greater emphasis on different frequency components at distinct sampling phases. The weighting scheme is defined as:
W1
an T[( > 40), w2 1 + 02 T(( <= lo):
(11)
where a1 and a2 are hyperparameter weights, tu is the manually set switching timestep, and I[() the indicator function. This formulation ensures that mid-low frequencies are prioritized in the mid-san mid-sampling phase, while high figh-frequency components receive more attention in the later phase.
EXPERIMENTS
3.1 EXPERIMENTAL SETTINGS
Base models and compared methods To demonstrate the effectiveness of our method, we apply acceleration technique to different video synthesis diffusion models, including the Open-Sora

---

Published as a conference paper at ICLR 2025
1.2 (Zheng et al. 2024). Open-Sora-Plie IPKU-Yual Lab and Tuzhan Al etc. 2024 Latte (Ma
et al. 2024a), Cog VideoX (Yang et al. 2024). and Vchitect-2.0 (Fan et al.; (025 We compare our base models with recent efficient video synthesis techniques, including PAB (Zhao et al., 2024c) and A-DiT (Chen et al., 2024c), to highlight the benefits of our approach. Notably, A-DiT was originally designed as acceleration method for image synthesis. Here have adapted it for video synthesis to facilitate comparison. Please refer to the Appendix for more details of the base models and compared methods.
Evaluation metrics and datasets To assess the performance of video synthesis acceleration meth- ods, focus primarily on two aspects, namely inference efficiency and visual quality. To evaluate inference efficiency, we employ Multiply-Aceumula Accumulate Operations (MACs) and inference latency as metries. We utilize VBench (Huang et al. 2024). L.PISS (Zhang et al., 2018), PSNR, and SSIM for visual quality evaluation. VBench is a comprehensive benchmark suit for video generative models. is well-aligned with human perceptions and capable of providing valuable insights from multiple perspectives. LPIPS, PSNR, and SSIM measure the similarity between videos generated by the ac- celerated sampling method and those from original model. PSNR quantifies pixel-level fidelity between outputs, LPIPS measures perceptual consistency, and SSIM assesses structural similarity. In general, higher similarity scores indicate better fidelity and visual quality.
Implementation details All experiments conduct full attention inference for spatial and temporal attention modules every 2 timesteps to facilitate dynamic feature reuse. The weight wo(t) increases linearly from 0 to 1 starting from the beginning of dynamic feature reuse until the end of sampling. For CFG output reuse, full inference is conducted every 5 timesteps, starting from 1/3 of the total sampling steps (e.g., for Open-Sora 1.2, which has 30 total sampling steps, this begins at step 10). The hyperparameters 01 and are set to default value of 0.2, which performs well for most models. For more details on the selection of hyperparameters, please refer to Appendix A.5. All experiments are carried out on NVIDIA A 100 80GB GPUs using Py Torch, with FlashAttention (Dao al. 2022) enabled by default.
Table 1: Comparison of efficiency and visual quality on single GPU.
Efficiency Speedup
Visual Quality
Method
MACs (P)
Latency (s)
VBench
LPIPS
SSIM
PSNR
Open-Sora 1.2 (192 frames, 480P)
Open-Sora (72 30)
6.30
1x
192.07
78. 79%
A-DiT (N. 14.N
5,51
1.14x
168.69
77.43%
0.2834
0.7403
17.77
A-DIT (N. 28,N 22)
4.72
1.34x
143.14
76.60%
0,3321
0,7092
16.24
PAB
1.23x
156.73
78.15%
0.1041
0.8821
26.43
Ours
4.13
1.62x
78.46
0.0835
27.03
Sora--orarrn (65
frames, 512x512)
Open-Sora- Plan (7 150)
10.30
103.76
80.16%
A-DiT (N.- 14,
8.60
1.19x
86.88
78.125%
0.4515
0.4813
16.08
A-DiT (N. 28,
1.46x
77.71%
0.4819
0.4467
15.42
PAB
7.39
1.32x
78.72
80.06%
0.2423
0.7126
20.29
Ours
5.51
1.68x
61.68
80.19%
0.1348
0.8138
23.72
Latte (16 frames, 512xx512)
50)
29.22
77.05%
(NDiT (N. 14.N
2.67
1,23x
23.80
76.27%
0.1731
0.8107
22.69
A-DiT (N. 28,N 2)
2.29
1.43x
20.38
76.01%
0.2245
0.7620
21.00
PAB
2.24
1.28x
22.84
76.70%
0.2904
0,7083
18.98
Ours
1.97
1.54
18.98
76.89 %
0.0817
0.8948
28.21
Cog VideoX (48 frumnss, 480P)
Cog Videnx (7 50)
6.03
1x
78.48
80.18%
A-DiT (N.- = 4,
5.62
1.08
72.72
79.61 %%
0.3319
0.6612
17.93
A-DiT (N.- S.N
5.23
1.15
68.19
79.31
0,3822
0.6277
16.69
A-DiT (N. 12.N
4.82
1.26x
62.50
09.009%
0.4053
0.6126
16.15
PAB
4.45
1.35x
57.98
79.76%
0.0860
0.8978
28.04
Ours
3.71
1.62x
48.44
79,83%
0.0766
0.9066
28.93
Vehitect-2.0 (40 frames, 480P)
Vchilect-2.0
14.57
260.32
80%
A-DIT (N. 6,N
13.00
1.11x
233.59
79.98%
0.4153
0.5837
14.26
A-DiT (N. 12,N =3)
11.79
1.24x
209.78
79.50%
0.4534
0.5519
13.68
PAB
12,20
1.26x
206.23
79_56%
0.0489
0.8806
27.38
Ours
8.67
1.67
156.13
80.84 %
0.0282
0.9224
31.45

---

Published as a conference paper at ICLR 2025
3.2 MAIN RESULTS
Quantitative comparison Table presents quantitative comparison of our method with D-DiT and PAB in terms of efficiency and visual quality. We synthesize videos with prompts provided by VBench and the synthesized videos to compute the Vßench metrics well calculate LPIPS, SSIM, and PSNR with videos sampled by the original model. The results demonstrate that our method achieves stable acceleration efficiency and superior visual quality across different base models, sampling schedulers, video resolutions, and lengths. Open-Sora
Open- Sora- Plan
Latte
A-DIT Original
PAB
Ours
Figure 9: Visual quality comparison of different methods. Differences are highlighted in red boxes. Visual quality comparison Fig. 9 compares the videos generated by our method against those by the original model, PAB, and A-DiT. The results demonstrate that our method can effectively preserve the original quality and fine details. More visual results can be found in the Appendix.
3.3 ABLATION STUDY
To comprehensively assess the effectiveness and efficiency of our method, we perform ex- tensive ablation studies based on Open-Sora, synthesizing videos of 48 frames at 480P. Efficiency Table 2 compares the efficiency of the original Open-Sora and its variants with different ac- celeration components. There are two key observa- tions. (1) The Dynamic Feature Reuse Strategy and CFG-Cache independently contribute significant ductions in inference costs. When combined. they fur- ther minimize inference overhead. (2) Compared to
Table 2: Impact on inference efficiency.
Variants
MAC' (P) Latency (6) A (s)
Sa Dyani: crocdde
1.34
41 28
31.25
-801
1,33
33.50
-7.78
31.32
-9.96
Vanilla Feature Reuse, the proposed Dynamic Feature Reuse strategy has a negligible impact on efficiency.
-15.16
(Vamlla
represents the
reduction latency comsared to original model.)
Visual quality Table the visual quality of the original Open-Sora with its variants implementing different acceleration components. Note that vanilla feature reuse leads to a performance drop VBench and LPIPS. The introduction of the dynamic feature reuse strategy mitigates the loss of information and thereby improves the performance of these metrics (e.g., VBench: 78.34% - 78.69%). Fig. 10 (a) provides a visual comparison of the results. can be observed that vanilla feature reuse shows reduced details (e.g., the moon and snowflakes), whereas dynamic feature reuse strategy can significantly alleviate this problem. The Feature MSE curves show that adding the bias term lower the MSE between intermediate features from the original and accelerated sampling process, aligning with the visual results.
Table 3: Impact visual quality.
Table 4: Scaling to multiple GPUs with DSP.
Variants
VBench
LPIPS
PSNR
SSIM
Mahas
La A100
2x A100
4x A
$x A100
Original Open-Sora
78.99%
Full (w/ Vanilla FR)
78.34%
0.0657
28.20
0.8785
Opee-Sora
392.07 (1-)
(266g
3009
(492
50
Full Dynamic FR)
69%
0.0590
28.41
0,8938
PAR
(1.35)
11133331
90.1
CPG Cuche wis Enhascement
78.42%
0.0709
27.97
0.8727
11844 82-3
Open-Sues-Paede Paan/21
iames 512w512)
22.88.0.97.1
Finhunce LF only
78.58
0.0617
28.29
0.8894
Spee-Sor-PPaa
159.21 (1.87)
99.10-3.35x4
40.316444-
Enhance HF only
78.49%
0.0686
28.08
0.8834
PAB
14..3
(7..0012.49g)
30.33R55500
Full (w/ Tull CFG-CCache
78.69%
0.0590
28.41
0.8938
(3.65x) 57.2015.40
31.21055
FR demotes Fenture Reuse)
Referring to Table 3 it can be seen that introducing CFG-Cache without enhancement reduces the visual quality. On the other hand, CFG-Cache with dynamic enhancement of either the low- high-
8

---

Published as a conference paper at ICLR 2025
Feature MSB
WiO Feature Reuse
Feature MSE
Originall
CPG-Carte w/o E Enbencentent
Dyzaaii Festure Reuse (a) Impact of Dynamic FR on Feature MSE and visual results
Vanilia Feature Rause
Enhence LF odly Enhance HE only
Full CFG-Cache
(b) Impact CFG-Cache on Feature MSE visual results
Figure 10: Comparison of Feature MSE curves and visual results from the ablation study, frequency bias helps to improve the visual quality, and their combined effect achieves the best visual quality. Fig. [O](b) (b) shows that enhancing low- now-frequency bias improves the fidelity of low- cowfireuenccy components (e.g., clouds, tornado outlines) while enhancing nigh-frequency bias enriches high- frequency details (e.g., lightning). The Feature MSE curve of CFG-Cache without enhancement aligns with the reduced visual quality. Dynamic enhancement helps to mitigate error accumulation, leading to higher visual fidelity.
3.4 SCALABILITY AND GENER ALIZATIINN
Scaling to multiple GPUs To evaluate the sampling efficiency of our method on multiple GPUs, we adopt the approach used in PAB and integrate Dynamic Sequence Parallelism (DSP) (Zhao al., 2024b) to distribute the workload across GPUs. Table 4 fillustrates that, as the number of GPUs in- creases, our method consistently enhances inference speed across different base models, surpassing the performance of the compared methods.
Performance at different resolutions and lengths To evaluate the effectiveness of our method in accelerating sampling for videos of varying sizes, we conduct tests across different video lengths and resolutions and report the results in Fig. [1 Our method maintains stable acceleration performance when faced with increasing resolutions and frame counts videos, demonstrating its potential to accelerate sampling longer and higher-resolution videos in line with practical demands.
4Bfnemex, 0SoP
3diremes, 4B3P
102loummo, 4800
160
11644
TA.
4P-18
A14
0.7x
240tromes, 360P
48name, 720P
32fromes, 1080P
100.79
12
0.24
43:00
6.02
14.21
7.70
Figure 11: Acceleration efficiency of our method at different video resolutions and lengths. 12V and image synthesis performance We integrate our acceleration method to the state-of-the- art image-to-video model DynamiCrafter (Xing al.) (023) and image synthesis model Pix sigma (Chen et al., (024a).. As shown in Fig. 12] our method significantly accelerates sampling while maintaining visual fidelity, demonstrating its potential for extension to various base models.
RELATED WORK
4.1 DIFFUSION MODELS FOR VIDEO SYNTHESIS
Diffusion models have demonstrated potential in high-quality image synthesis (Ho et al. 2020 Rombach et al. 2022 Chen al. 2023 2024b), attracting significant attention. Subsequent works have adapted these models for video synthesis to generate high-fidelity videos (Ho et al.) 2022
Motivated by advancements in image synthesis, early studies typically employed the diffusion UNet
architecture (Blattmann et al. 2023 Wang et al. 2023 Zhang al. 2023 Wu et al. 2023 Zhang jet al.] 2024c). As the scalability of diffusion transformer (Peebles & Xie 2023 was validated in image synthesis, an increasing number of works have adopted the diffusion transformer the noise

---

Published as a conference paper at ICLR 2025
Prongt
(b) Renils of Image Synthess Based on PizAut Sigma
Prompt pt
SH0 %
Original Curs
1.30
4-9 15.56
DynamiCraftee
pisket-sigma [c] Compariaon of Inference Time
(s) Resalts of Image to Vidao Sampling A Acceleration Based on DynamiCrafter
Figure 12: Visual results and inference time of our method on 12V and image synthesis models. estimation network (Ma et al. Yang et al. 2024).
2024a Zheng et al. 2024 PKU- Yuan Lab and Tuzhan Al etc. 2024
4.2
EFFICIENCY IMPROVEMENTS IN DIFFUSION MODELS
Despite the impressive performance of diffusion models in image and video synthesis, their sub- stantial inference cost limits their practicality. Prior research on improving the efficiency of diffu- sion models has primarily focused on two perspectives, namely reducing the number of sampling steps and lowering the inference cost per sampling step. Regarding the reduction of sampling steps, most approaches achieve high-quality samples with fewer steps by employing efficient SDE or ODE
solvers (Song et al. 2020 Lu et al. 2022a b). Other methods reduce sampling steps by progressively
distilling the model (Salimans & Ho 2022 Meng et al. [202] Sauer et al. [2023] Lin & Yang 2024
Li et al. 2024b) or employing consistency models (Luo et al. 2023 [onn et al. 2023).
More works have focused on reducing the inference cost per timestep. Some approaches improve
network efficiency through pruning (Zhang al.) 2024a)
quantization (Shang et al. 2023 So
ct al. 2024a He et al. 2024, Li et al. (024a] Sui et al. 2024 /Zhao et al., 2024a). while others ob-
tain more lightweight network architectures through search techniques (Li et 2 Yang al.]
2023). However, these methods often require additional computational resources for fine-tuning or
optimization. Some training-free free approaches (Bolya & Hoffman] 2023) Wang al. 2024) focus on the input tokens, accelerating the sampling process by reducing the number of tokens to be processed by eliminating token redundancy in image synthesis. Other methods reuse intermediate features be- tween adjacent sampling timesteps. avoiding redundant computations (Wimbauer et al. 2024 So jet al., 2024b). TGATE (Zhang et al.] (0024b) accelerates image generation by caching and reusing attention outputs scheduled timesteps. DeepCache (Ma et al. 2024b) and Faster Diffusion (Li cct al. 2023b) employ a feature caching mechanism to indirectly alter the UNet diffusion for accel- eration. DiT (Chen et al. 2024c) adapts this mechanism to the diffusion transformer architecture by caching the residuals between attention layers. PAB (Zhao et al.) 2024c) caches and broadcasts intermediate features at different timestep intervals based on the characteristics of varying atten- tion blocks. Although these methods have achieved some improvements in diffusion efficiency, the efficiency enhancements for diffusion transformers in video synthesis remain insufficient.
5 CONCLUSION AND DISCUSSION
In this work, we present FasterCache, a training-free strategy that significantly accelerates video synthesis inference while preserving high-quality generation. Through analysis of existing cache- based methods, we find that directly reusing adjacent-step features in attention modules can degrade video quality. Additionally, we investigate the acceleration potential of CFG, identifying redundancy between conditional and unconditional features at the same timestep. Leveraging these insights, FasterCache integrates a dynamic feature reuse strategy that maintains feature distinction and tem- poral continuity, and CFG-Cache which optimizes the reuse of conditional and unconditional out- puts to further boost speed without sacrificing detail quality. Extensive experiments demonstrate its strong performance in both efficiency and synthesis quality across diverse video models, sampling schedules, video lengths and resolutions, highlighting its potential for real-world applications.
Limitation Despite the effectiveness shown by our method, certain limitations remain. When the synthesis quality of the model is suboptimal, our acceleration method is unlikely to yield satisfactory results either. We believe that advancements in base video models will mitigate this issue. Addi- tionally, in complex scenes with substantial video motion, method may occasionally produce degraded results. At present, this can be remedied through manual adjustments of hyperparameters. In the future, we plan to investigate strategies for adaptive caching to further enhance performance.
10

---

Published as a conference paper at ICLR 2025
6 ACKNOWLEDGEMENTS
This study is supported by the National Key R&D Program of China No.20222D0161102, and by the video generation project (intern-ccittct) of Shanghai Artificial Intelligence Laboratory- This study is also supported by the Ministry of Education, Singapore, under its MOE AcRF Tier and under the RIE2020 Industry Alignment Fund - Industry Collaboration Projects (IAF-ICP) Funding Initiative, as well as cash and in-kind contribution from the industry partner(s).
2 (M0ET2EP20221-00)
MOE-T2EP20223-0002
REFERENCES
Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models large datasets. arXiv preprint arXiv:231.15112,, 2023.
Daniel Bolya and Judy Hoffman. Token merging for fast stable diffusion. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4599-4603, 2023.
Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart-a: Fast training of diffusion transformer for photorealistic text-to-image synthesis, 2023.
Junsong Chen, Chongjian Ge, Enze Xie, Yue Wu, Lewei Yao, Xiaozhe Ren, Zhongdao Wang, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart-o: Weak-to-strong training of diffusion transformer for 4k text-to-image generation, 2024a.
Junsong Chen, Yue Wu, Simian Luo. Enze Xie, Sayak Paul, Ping Luo, Hang Zhao, and Zhenguo Li. Pixart-0: Fast and controllable image generation with latent consistency models, 2024b.
Pengtao Chen, Mingzhu Shen, Peng Ye, Jianjian Cao, Chongjun Tu, Christos-Savvar Bouganis, Yiren Zhao, and Tao Chen. 8-dit: A training-free free acceleration method tailored for diffusion transformers. arXiv preprint urXiv:2406.01 125, 2024c.
Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. Flashattention: Fast and memory- efficient exact attention with io-awareness. Advances in Neural Information Processing Systems, 15:164441,6359 2022.
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-firs International Conference on Machine Learning, 2024.
Weichen Fan, Chenyang Si, Junhao Song, Zhenyu Yang, Yinan He, Long Zhuo Ziqi Huang, Ziyue Dong, Jingwen He, Dongwei Pan, et al. Vchitect-2.0: Parallel transformer for scaling up video diffuuson models. arXiv preprint arXiv:2501.0844
2025.
Yefei He, Luping Liu, Jing Liu, Weijia Wu, Hong Zhou, and Bohan Zhuang. Ptqd: Accurate post- training quantization for diffusion models. Advances in Neural Information Processing Systems, 36, 2024.
Jonathan Ho and Tim Salimans. arXiv:2207. 12598, 2022.
Classifier-free
diffusion guidance.
arXiv preprint
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-688 6851, 2020.
Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi. and David J Fleet. Video diffusion models. Advances Neural Information Processing Systems, 35:8633- 8646, 2022.
Yi-Ting Hsiao, Siavash Khodadadeh, Kevin Duarte, Wei-An Lin, Hui Qu, Mingi Kwon, and Ratheesh Kalarot. Plug-and and--andd-layy play diffusion distillation. In Proceedings of the IEEE/CVF Con- ference Computer Vision and Pattern Recognition, pp. 13743-13752, 2024.

---

Published as a conference paper at ICLR 2025
Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si. Yuming Jiang, Yuanhan Zhang, Tianx- ing Wu, Qingyang Jin, Nattapol Champaisit, Yaohui Wang, Xinyuan Chen, Limin Wang. Dahua Lin, Yu Qiao, and Ziwei Liu. VBench: Comprehensive benchmark suite for video generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- fion, 2024.
Lijiang Li, Huixia Li, Xiawu Zheng, Jie Wu, Xuefeng Xiao, Rui Wang, Min Zheng, Xin Pan, Fei Chao, and Rongrong Ji. Autodiffusion: Training-free optimization of time steps and architec- tures for automated diffusion model accelcration. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 7105-7114, 2023a.
Senmao Li, Taihang Hu, Fahad Shahbaz Khan, Linxuan Li, Shiqi Yang, Yaxing Wang, Ming-Ming Cheng, and Jian Yang. Faster diffusion: Rethinking the role of unet encoder in diffusion models. arXiv preprint arXiv:2312.0900 2023b.
Yanjing Li, Sheng Xu, Xianbin Cao, Xiao Sun, and Baochang Zhang. Q-dm: An efficient low-bit quantized diffusion model. Advances in Neural Information Processing Systems, 36, 2024a.
Yanyu Li, Huan Wang. Qing Jin, Ju Hu, Pavlo Chemerys, Yun Fu, Yanzhi Wang. Sergey Tulyakov, and Jian Ren. Snapfusion: Text-to-image diffusion model on mobile devices within two seconds. Advances in Neural Information Processing Systems, 36, 2024b.
Shanchuan Lin and Xiao Yang. Animatedif-lighhning: Cross-model diffusion distillation. arXiv preprint arXiv:2403.127000 2024.
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu, Dpm-solver: fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in Neural Information Processing Systems, 35:5775-5787, 2022a.
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic models. arXiv preprint arX1v:2211.01095, 2022b.
Simian Luo, Yiqin Tan, Longbo Huang, Jian Li, and Hang Zhao. Latent consistency models: Synthe- sizing high-resolution images with few-step inference. arXiv preprint arXiv:2310.04372 2023.
Xin Ma. Yaohui Wang, Gengyun Jia, Xinyuan Chen, Ziwei Liu, Yuan-Fang Li, Cunjian Chen, and Yu Qiao. Latte: Latent diffusion transformer for video generation. arXiv preprint 24X1v:2401.03048, 2024a.
Xinyin Ma. Gongfan Fang, and Xinchao Wang. Deepeache: Accelerating diffusion models for free. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 15762-15777 2024b.
Chenlin Meng, Robin Rombach, Ruiqi Gao, Diederík Kingma, Stefano Ermon, Jonathan Ho, and Tim Salimans. On distillation of guided diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14297-14306, 2023.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4195-4205, 20233.
PKU- Lab and Tuzhan AI etc. Open-sora-plan, April 2024. URL https:://oi..og////
5281/zenodo. 10948109
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. arXiv preprint arXiv:2202.005/2. 2022.
Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach. Adversarial diffusion dis- tillation. arXiv preprint arXiv:2311.17442 2023.
12

---

Published as a conference paper at ICLR 2025
Yuzhang Shang, Zhihang Yuan, Bin Xie, Bingzhe Wu, and Yan Yan. Post-training quantization diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1972-1981, 2023.
Chenyang Si, Ziqi Huang, Yuming Jiang, and Ziwei Liu. Freeu: Free lunch in diffusion u-net. In CVPR, 2024.
Junhyuk So, Jungwon Lee, Daehyun Ahn, Hyungjun Kim, and Eunhyeok Park. Temporal dy- namic quantization for diffusion models. Advances in Neural Information Processing Systems, 36, 2024a.
Junhyuk So, Jungwon Lee, and Eunhyeok Park. Frdiff Feature reuse for universal training free acceleration of diffusion models, 2024b. URL htttps://arx.w.or///bb/////// 23tp//3121200011111
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502. 2020.
Yang Song, Prafulla Dhariwal, Mark Chen, and lla Sutskever. Consistency models. arXiv preprint arXiv:2303.01469, 2023.
Yang Sui, Yanyu Li, Anil Kag, Yerlan Idelbayev, Junli Cao, Ju Hu, Dhritiman Sagar, Bo Yuan, Sergey Tulyakov, and Jian Ren. Bitsfusion: 1.99 bits weight quantization of diffusion model. arXiv preprint 2406:3406.04333, 2024.
Genmo Team. Mochi 1. https: /ttps://github... http://ww...coo/ggm//////////ooa
2024.
Hongjie Wang, Difan Liu, Yan Kang, Yijun Li, 7he Lin, Niraj K Jha, and Yuchen Liu. Attention- driven training- faiee efficiency enhancement of diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16080-16089, 2024.
Yaohui Wang, Xinyuan Chen, Xin Ma, Shangchen Zhou, Ziqi Huang, Yi Wang, Ceyuan Yang, Yinan He, Jiashuo Yu, Peiqing Yang, et al. Lavie: High-quality video generation with cascaded latent diffusion models. arXiv preprint arXiv:2309.15000 2023.
Felix Wimbauer, Bichen Wu, Edgar Schoenfeld, Xiaoliang Dai, Hou, Zijian He, Artsiom Sanakoyeu, Peizhao Zhang, Sam Tsai, Jonas Kohler, et al. Cache me if you can: Accelerat- ing diffusion models through block caching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6211-6220, 2024.
Jay Zhangjie Wu, Yixiao Ge, Xintao Wang, Stan Weixian Lei, Yuchao Gu, Yufei Shi, Wynne Hsu, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7623-7633, 2023.
Jinbo Xing, Menghan Xia, Yong Zhang, Haoxin Chen, Wangho Yu, Hanyuan Liu, Xintao Wang, Tien-Tssn Wong, and Ying Shan. Dynamicrafter: Animating open-domain images with video diffusion priors. arXiv preprint arXiv:2310. 12190, 2023.
Shuai Yang, Yukang Chen, Luozhou Wang, Shu Liu, and Yingcong Chen. Denoising diffusion step-aware models. arXiv preprint arXiv:2310.033:: 2023.
Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng. al. Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint 2tt.:2240800077... 2024.
Dingkun Zhang, Sijia Li, Chen Chen, Qingsong Xie, and Haonan Lu. Laptop-diff: Layer pruning and normalized distillation for compressing diffusion models. arXiv preprint a 2024a.
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features perceptual metric. In CVPR, 2018
Wentian Zhang, Haozhe Liu, Jinheng Xie, Francesco Faccio, Mike Zheng Shou, and Jürgen Schmid- huber. Cross-attention makes inference cumbersome in text-to-image diffusion models. arXiv preprint A.X::22044002447, 2024b.
13

---

Published as a conference paper at ICLR 2025
Yabo Zhang, Yuxiang Wei, Dongsheng Jiang, Xiaopeng Zhang, Wangmeng Zuo, and Qi Tian. Con- trolvideo: Training-free controllable text-to-video generation. arXiv preprint arXiv:2305.13077 2023.
Yabo Zhang, Yuxiang Wei, Xianhui Lin, Zheng Hui, Peiran Ren, Xuansong Xie, Xiangyang Ji, and Wangmeng Zuo. Videoelevator: Elevating video generation quality with versatile text-to-image diffusion models. arXiv preprint 2xXiv:2403.05438. 2024c.
Tianchen Zhao, Tongcheng Fang, Enshu Liu, Wan Rui, Widyadewi Soedarmadji, Shiyao Li, Zinan Lin, Guohao Dai, Shengen Yan, Huazhong Yang, et al. Vidit-q: Efficient and accurate quantiza- tion of diffusion transformers for image and video generation. arXiv preprint urXiv:2406. 02540, 2024a.
Xuanlei Zhao, Shenggan Cheng, Zangwei Zheng, Zheming Yang. Ziming Liu, and Yang You. Dsp: Dynamic sequence parallelism for multi-dimensional transformers. arXiv preprint arXiv:2403. 10266, 2024b.
Xuanlei Zhao, Xiaolong Jin, Kai Wang. and Yang You. Real-time video generation with pyramid attention broadcast. arXiv preprint arXiv:2408. 12588, 2024c.
Zangwei Zheng, Xiangyu Peng, Tianji Yang. Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. Open-sora: Democratizing efficient video production for all, March 2024.
URLhttps: /i thub. http:////.cmcccccccccc//eeee-eea
14

---

Published as a conference paper at ICLR 2025
A APPENDIX
A.1 FURTHER DETAILS BASE MODELS
In this work, we applied FasterCache various video synthesis models, including Open-Sora 1.2 (Zheng et al. 2024). Open-Sora-Pian (PKU- Yuan Lab and Tuzhan Al etc. 2024). Latte (Ma)
al., 2024a). Cog VideoX (Yang al. 2024). and Vchitect 2.0 (Fan al. 2025). ). Open-Sora 1.2 integrates 2D VAE and 3D-VAE to enhance video compression and employs ST-DiT blocks for the diffusion process. Open- Saae-Sor--Plaal Plan adopts CausalVideo VAE to compress visual representations better and 3D full attention architecture to capture joint spatial and temporal features. Latte extracts spatio-temporal tokens from input videos and then adopts a series of transformer blocks to model video distribution in the latent space. Cog CogVideoX VideoX employs 3D VAE to compress videos along spatial and temporal dimensions and an expert transformer with the expert adaptive LayerNorm to facilitate the fusion between the two modalities.
A.2 FURTHER DETAILS OF COMPARED METHODS
PAB (Zhao et al. 2024c) employs a pyramid-style broadcasting mechanism to propagate attention outputs across subsequent steps. It optimizes efficiency by applying distinct broadcast strategies to each attention layer based on their respective variances. Additionally, the method introduces broadcast sequence parallelism to enhance the efficiency of distributed inference. This paper follows the default parameter configuration of PAB.
A-DIT (Chen et al. 2024c) accelerates inference by caching feature offsets instead of the full feature maps while preventing input information loss. It caches the residuals of the blocks in the latter part of DiT for approximation during early-stage sampling and caches the residuals of the blocks in the earlier part during later-stage sampling. In A-DiT, the parameters that need to be configured are the residual cache interval N, the number of cached blocks Ne. and the timestep boundary for determining the position of the cached blocks. Since the source code of A-DIT is not publicly available, we implemented its method based on the paper for accelerating video synthesis. Following the guidelines in A-DiT, we experimented with different configurations of Ne and N to balance visual quality and inference speed, allowing for a fair evaluation of the method.
A.3 MORE DISCUSSION
A.3.1 MORE DISCUSSION ON DYNAMIC FEATURE REUSE
Effectiveness of Dynamic Feature Reuse Assume that the output features of a particular layer in the diffusion model are function of the timestep denoted F()). The motivation behind Vanilla Feature Reuse lies in the observation that features at adjacent timesteps are highly similar. Vanilla Feature Reuse avoids the computation at the current timestep by directly reusing the features from the previous timestep. i.e. F(t) = F(t + At). Although F(t) and F(t + At) are very close with a minimal error F(t) F(t + At), the difference is not zero. To estimate this error, we assume that F(() is a smooth and differentiable function with respect to I, allowing us to perform Taylor expansion, yielding:
F(t+ + At) = F(t) + dF(t)
F'((t)
At
At+
-OAAt))
(12)
dt
dt
2
F(t + 3 = F(t) + dt
F()
At2
At +
3
+O(AP))
(13)
dt
dt2
2
By subtracting these expansions, we derive:
A(t+ - F(t + 3Dt) = dF(()) At) x (-2) O(AP))
(14)
Based on the statistics of approximately 200 video samples, we plotted the magnitudes of the first- order and second-orde order terms of F(t). When At (e.g., At = 1) is sufficiently small, the norm of second-order term is smaller than that of the first-order term, as shown in Fig. [3) (c). Furthermore,
we tested three different estimations for F(t), denoted as F(t): (a) F(t) = F't + 1). (b) f(t) = dF(() F(((1) Subsequently, calculated the I.1 F(t 1) dF(O) and (c) (() F(t + dt di
2de
15

---

Published as a conference paper at ICLR 2025
distance between each Ft) and F(t). As shown in the Fig. 13 (d), incorporating the second-order term yields only a marginal reduction in the LI distance compared to the first-order term. Therefore, the second-order terms contribute only marginally 78.77% + 78. 78.80%). However, the computation of second-order terms incurs significant costs in memory and latency. Considering both simplicity and efficiency, we only the first-order term for eπrоr estimation in Dynamic Feature Reuse. Based on these analyses and statistical results, we define the error term as:
the improvement in visual quality (VBench:
dF(() AL dF(() di
= F()) F(I+
( A1) F(L +3A1)) W.
(15)
The scale factor w is introduced to scale the bias term to approximate the error E. In Eq. (5), E' = Fi-1 Ftt Fenche Rs (F% heche F cache) w(t). By introducing this feature bias term, the information loss could be reduced, thereby improving the quality of the synthesis videos while maintaining computational efficiency.
As shown in Fig. 13 (a), we tried different design choices for Dynamic Feature Reuse (DFR) and found that the linear increasing strategy is a simple and effective manner for dynamically capturing missing features. Different design choices for DFR: (1) Constant weights w()). A constant weight of wi() = 0.5 is applied to the feature biases at each accelerated timesteps. (2) Learnable weights w(t). We introduced a set of learnable parameters w()., which are optimized by minimizing the MSE loss between the features output by DFR during accelerated sampling and those generated in the original unaccelerated sampling process, resulting in the learned w().. (3) Linearly increasing w(t) (Our DFR). Starting from the application of DFR the end of sampling proces, the weight function (t),, used for weighting feature biases, linearly increases from 0 to
Design choices for w(t) in Dynamic Feature Reuse
The trend of the optimized w(t) is shown in Fig 13 (a), the result indicates that w(t) obtained through optimization gradually increases as sampling progresses. This trend is primarily attributed the increasing stability of feature biases in Eq. 5 with respect the sampling timesteps and the growing reliance on bias features for synthesizing high-quality details in the later stages of sampling. The performance of different strategies shown in Table 5 All results incorporating feature biases outperform those without them. The linearly increasing w(t) achieves comparable
performance to optimized learnable wt)), both outperforming constant wt)). Given the simplicity of linear interpolation, we ultimately adopt linearly interpolated w(t) to weight the feature biases.
Table Performance of different Dynamic FR strategies.
Variants
LPIPS
PSNR
SSIM
Vanilla FR
0.0657
28.20
0.8785
Dynamic FR (Constant wtt) 0.5)
0.0615
28.33
0.8889
Dynamic FR(Learned wt))
0.0596
28.45
0.8941
Dynamic FR(Linear wt))
0.0590
28.41
0.8938
Comparison between Dynamic FR and Vanilla FR Fig. 13 (b) presents the generated results of Vanilla Feature Reuse (FR) and Dynamic FR and the differences between the features produced by Vanilla FR and Dynamic FR compared to the original features. It is evident that, due to the introduction of feature biases, the feature differences between Dynamic FR and the original features are less significant. In contrast, the features produced by the model accelerated with Vanilla FR exhibit detail loss compared to the original features, leading to noticeable detail degradation in the synthesized images (as highlighted by the red box).
A.3.2 FURTHER DISCUSSION ON CFG-CACHE
Effectiveness of CFG-Cache The reliability of CFG-Cache stems from three key factors: (a) After the early stage fearty. the similarity between conditional output cond(t) and unconditional output uncond(t) (t) the same timestep t::
uncond(L) = cond(() + A, when >= learty-
(16)
(b) The predictability of biases between conditional and unconditional output from previous timesteps, expressed as:
A uncond(t At) cond(t At) uncond(t) cond(t) +
(17)
16

---

Published as a conference paper at ICLR 2025
Learned w
Oulput
Linear w
Constant W
0.6
w(t)
Resure
0.0
10
15
20
25
30
Original Festure
|VER
Originel
|DRR
Original
(b) Difference between VFR/DFR features and original features, and coresponding outputs.
(a) Different setting of w(t)
First-arder tarm
Zeroth- order approximation
Second-order term
First-order aporasimation
approximation
0.28
of carmns
DDzzaance
(c) The norm of first-order and second-order term Figure 13: Design choices for Dynamic Feature Reuse and comparison between Dynamic Feature
(d) L1 distance of different approximation.
Reuse (DFR) and Vanilla Feature Ruse (VFR).
In practice, find that when At is sufficiently small, the can be considered negligible. Then: uncond(t) cond(() (uncond(t At) cond(t At))
(18)
(c) The dynamic variations of the frequency-domain distribution of feature biases, as illustrated in Fig. 7(b) and 14g.
Visualization of CFG biases From the onset of CFG-Cache the end of sampling, the differences between the conditional and unconditional output features progressively shift from being dominated by low-frequency features feg-freeuency features. As shown in Fig [4] this observation aligns with the feature visualization analysis: during the early and middle sampling stages, CFG primarily guides the model synthesize perceptual features such as reasonable shapes and layouts, which often represented in the low-frequency feature domain. In contrast, during the later stages of sampling, CFG contributes primarily to the synthesis of high-quality details, typically governed by freeueeyy features. This insight motivates to assign higher weights features of different frequencies at different stages, allowing to gain more emphasis, thereby preserving the visual quality.
20
Cond- Uncond
Figure 14: The variation in differences between the conditional and unconditional outputs during the sampling process.
17

---

Published as a conference paper at ICLR 2025
A.3.3 FASTERCACHE UNDER DIFFERENT CFG SCALES AND NEGATIVE PROMPTS
We compared different negative prompt settings on Open-Sora: (1) default empty negative prompt and (2) non-empty negative prompt:
"worst quality normal quality, low quality, low res, blurry, text, watermark, logo, banner, extra digits, cropped, ipeg artifacts, signature, username, envor, sketch, duplicate, ugly, monochrome, horror, geometry, mutation, disgusting, bad anatomy, bad proportions, bad quality, deformed, dis- connected limbs, oud of frume, out of focus, dehydrated, disfigured, extra arms, extra limbs, extra hands, fused fingers, gross proportions, long neck, jpeg, malformed limbs, mutated, mutated hands, mutated limbs, missing arms, missing fingers, picture frame, poorly drawn hands, poorly drawn face, collage, pixel, pixelated, grainy"
We calculated the LPIPS, SSIM, and PSNR between the videos generated by FasterCache and those generated by the original model. As shown Fig. (a) and (b), the experimental results show that FasterCache performs similarly under both prompt settings. This is consistent with expectations, as EFG-Cache caches the biases between the conditional and unconditional outputs, which are not significantly affected by changes in the negative prompt setting.
We also experimented with different CFG guidance scales g on Open-Sora. As shown in Fig 13 (b) and (c), regardless of increasing or decreasing the scale, while the adjustment affects the original Open-Sora results, FasterCache consistently maintains a high level of alignment with the original results, particularly in preserving details. Therefore, FasterCache is not affected by changes in the CFG guidance scale and maintains high-quality acceleration.
Origine
250
0.903
ResteCacch
Monvemply mpty Meg Prompr
Nonvempty Neg Promp Cmpply Prompt
Kon-emplyy Keg Foumpt
Erpply Neg Porptt
Emotyy hig Frompt
etc
35
12
>9
16
(8) Non-emply.ge g=?
FasteeCache performance souss different empty empy angative prompts.
Originll
Sache
gr3
n-7
9-9
g-11
(c) Visual results of CasterCache acmuss different CPG scales. (Empty negative promps)
Figure 15: The performance of CasterCache under different CFG scales with empty and non-empty negative prompt settings.
0.92
LPPS
SEM
0.06
0.98
alphal
#lphal
alpha] alpha2
alpha2
alpha2
0.0
02
2.1
0.6
0.
0.
0.1
3.0
63
05
63
6.4
alpha
alpha
alpha
Figure 16: Different Settings of a in CFG-Cache.
A.4 ADDITIONAL. QUALITATIVE EXPERIMENTS
More visual results on Text-to- Video models
The additional visual comparison results for Open-
Sora 1.2 (Zheng al. 2024), Open-Sora-Plan (PKU- Yuan Lab and Tuzhan Al etc., (024). and
Latte (Ma et al. 2024a) are presented in Fig. 17
Fig. 18 and Fig. 19 while further comparisons for Yang al. 2024) and Vchitect-2.0 (Fan et al. 2025 shown in Fig. 20 Our
method demonstrates reliable fidelity across various models and styles or content in video synthesis,
while simultaneously achieving acceleration.
18

---

Published as a conference paper at ICLR 2025
Additionally, Fig. 21 demonstrates the visual performance of FasterCache state-of of- offte-of-the-an models CogVideoX-5BB 5B and Mochi-10B (Team] (2024). FasterCache achieves an acceleration of 1.63 times (206s + 126s) CogVideoX-53 and 1.74 times (320s 184s) on Mlochi-10B As model scale increases, FasterCache consistently accelerates the sampling process while maintaining fidelity in synthesized videos. We also observe that the generative capability of the base model improves, FasterCache becomes more robust in synthesizing videos with complex scenes or rapid motion. For instance, in Fig. [1] the 1st example shows subtle details of small groups of fish, the 3rd example highlights intricate finger details and complex non-rigid motions, and the 4th and 5th examples exhibit rapid and large-scale movements. These results demonstrate the broad potential of FasterCache in practical applications.
More visual results on Image-to- models We conducted image-to-video sampling accel- eration experiments based on DynamiCrafter (Xing et al. 2023), achieving 1.52 speedup on single GPU. Additional visual results are provided in Fig. 22 Our method demonstrates good fidelity in the acceleration of image-to-video models, indicating broad potential for practical applications.
A.5 ADDITIONAL QUANTITATIVE EXPERIMENTS
A.5.1 USER PREFERENCE STUDY
To assess the effectiveness of our FasterCache, we additionally conduct human evaluation. We randomly selected 30 videos for each model. Fach rater receives a text prompt and two generated videos from different sampling acceleration methods (in random order). They then asked to select the video with better visual quality. Five raters evaluate each sample, and the voting results summarized in Table 6 As one see, compared to other acceleration methods, the raters strongly prefer the videos generated by our method.
Table User preference study. The numbers represent the percentage of raters who favor the videos synthesized by our method.
Method comparison
Open-Sora 1.2
Open-Sora- Open-Sora-Plan
Latte
Ours A-DiT
80.67%
78.00%
77.33%
Ours vs. PAB
69.33%
72.67%
74.00%
A.5.2
HYPERPARAMETER SELECTION
Table 7: Different Dynamic FR caching intervals. Table 8: Different CFG-Cache caching intervals.
Interval
LPIPS
PSNR
SSIM
Interval
LPIPS
PSNR
SSIM
0.0590
28.41
0.8938
0.0496
28.88
0.8964
0.0537
28.56
0.8947
0.0698
27.95
0.8853
0.0590
28.41
0.8938
0.0751
27.61
0.8823
0.0724
27.68
0.8818
5
0.0897
27.39
0.8712
0.0104
27.44
0.8706
Caching timestep interval of Dynamic Feature Reuse We experimented with different caching timestep intervals for Dynamic Feature Reuse. According to Table 7 it can be observed that as the caching timestep interval increases, the fidelity of the synthesized results gradually decreases. In practice, the caching timestep interval for Dynamic Feature Reuse can be adjusted as needed.
Caching timestep interval of CFG-Cache We experimented with different CFG-Cache intervals and found that when the interval exceeds 5 timesteps, there significant decline in fidelity. as shown in Table Therefore, to balance fidelity and efficiency, we chose a CFG-Cache caching interval of 5. This means that after CFG-Cach is initiated, the model performs full inference for both the conditional and unconditional branches every 5 timesteps and caches the features.
The configuration of a in CFG-Cache. In CFG-Cache, we experimented with different con- figurations of a, where 0n used enhance low-frequency biases and ar used enhance high-frequency biases. Through these experiments shown in Fig. [6] we found that aj = 0.2 and 02 = 0.2 works effectively.
19

---

Published as a conference paper at ICLR 2025
astronaut flying in space, tilt down
a bird building nest from twigs leaves
Origrnal
A-Di
BNB
Ouns
cat and dog
cup and couch
Origina
4-DIT
PAB
Ouss
A tranguil tableau of the Parthenon stands
umbrella and handbag
Original
b-brr
IAB
Ouns
shtray butts table, smoke flowing black bac karkgro
Fireworks
Oogginll
A-IXT
PNB
Figure 17: More visual results on Open-Sora (480P 192 frames). Zoom in for details.
20

---

Published as a conference paper at ICLR 2025
bear catching a salmon in powerful jaws
cake
A car moving slowly on an empry street, miny evening
DIT
BAB
O1DS
A corgi is plaping drum kit
panda coolingg in the kitchen
zebra
Ohigiia
A DIT
BALB
Ous
rooot DJ playnng the tumtable, in hewvy tralttionll wroughl-iron streellamp featured delicate raining futuristic tokyo roafop eyberpunk night, sal-ft, fantasy
A tranquil tablesu the village squam,
A Jellyfish foaaingg Lirough the ocean, with hiokmiinesoee tectacles
illggse patterss and amber baed glass panels
Origina
DIT
BLB
O36
Figure 18: More visual results on Open- Sora- Plan (512x512 65 frames). Zoom in for details.
21

---

Published as a conference paper at ICLR 2025
on the right dog, front view
shark swimming in the ocran, pixel art:
hind sparing gracefllly in the sky
Odiginal
4
PAB
Ous
1
hot dog the bottur pizza, frunt view
A jellyfish floating through with bioluminescent tamtacles
A jellyfish nouting through the oceam, with hioluminesceett tectacles
Oaiiinal
BAB
Clus
panda drinking coffee cafe Paris, rucking Focus
fat rebbit wearing purple robe walking throuph fantasy landsrape
super robot protecting city
Odigina
ENT
RAB
Ous
Figure 19: More visual results Latte (512x512 16 frames). Zoom in for details.
22

---

Published as a conference paper at ICLR 2025
girdlle the hünd, Lrant view
Flaworks
Turtie swimming
CayVidoX
squirrel burger
stiffing scents food
A beuntitul coastal beart, sodag, weres lapptag
featuring steady parspective
Orggin
MaT
Ouo
ursnging Nowers
bird Jonst
Cake
A-D0T
Oue
space shuttle lamnchng into orbit, with flames and smoke billowing out from enclines
bas axveeetting to gain speed
DDW
Ous
teddy beer washing the dishes
white bird
mapee spaceship landlng Mars
Ooupied
Ous
swimming
teddy
Towntiin
AADET Chipad
Oua
Figure 20: More visual results on VideoXXX0 (480P 48 frames) & Vchitect-2.0 (480P 40frames).
23

---

Published as a conference paper at ICLR 2025
CogVideoX-58 48frames 480P
Odiginl
FasterCache
Original
Paster
Mochi 10B 163frames 480P
Oriepiaal
Farden Cachh
Original
Fedeeracch
Originaal
Foster Cache
Figure 21: More visual results Cog VideoX-BB and Mochi- 10B. Zoom in for details.
24

---

Published as a conference paper at ICLR 2025
Chiegraa
Out
prozngt
Catckraal
Oues
prongi
Catrandd
Okas
promptt
Oue
grompt
CUES
Figure 22: More visual results on DynamiCrafter (1024x576 16frames). Zoom in for details.
25


Reviews:
[{"review_id": "JmkDs88DQG", "reviewer": "Reviewer_4sX2", "summary": "This paper proposes a training-free feature caching strategy for video diffusion models, named FasterCache, that improves how features are reused for accelerating inference. Instead of directly reusing the adjacent-step features as in previous works, FasterCache interpolates the neighboring features to efficiently synthesize features for the missing steps. In addition, the paper introduces CFG-Cache, which aims to effectively reuse the conditional outputs for computing the unconditional outputs. Experimental results demonstrate notable visual quality and efficiency improvements compared with the existing methods.", "questions": "1. For CFG-Cache, should we interpret the feature differences in the frequency domain in a similar way as MSE? \n\n2. In Figure 7b, it seems to me that low-frequency components do not gradually shift into high-freq as the authors mention in L300-301, since the high-freq components differ both at the beginning and near the end. What kind of insights can we gain from this analysis?\n\n3. In the experiments, PAB seems to significantly outperform $\\Delta-DiT$, but there are no PAB results for CogVideoX and Vchitect. Also, why is the performance of $\\Delta-DiT$ so low? Is it because of the inherent randomness of image diffusion models?", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper clearly demonstrates the motivation and the problem of existing works with illustrative examples (Fig. 3-7). Overall, the limitations of vanilla cache-based acceleration methods with CFG is analyzed very thoroughly.\n\n2. Qualitative results demonstrate notable improvements in visual details and the spatio-temporal consistency of the generated frames.", "weaknesses": "1. Slightly missing justifications for the design choices of the proposed method. For instance, is the linear interpolation strategy sufficient for dynamically building the missing features? Are there any other options that the authors have considered? Using the proposed Dynamic Feature Reuse, how similar are the computed features compared to the existing caching methods? (Please guide me if I missed the descriptions)\n\n2. Limited video-specific contributions. The proposed methodologies seem to applicable to any image diffusion model, of which comparison to previous works is missing, and I could not find any video-related novelties proposed. The notation $t = \\{1 ... T\\}$ in Equations seem to denote the diffusion sampling steps, and I think there is no notation for the temporal axis? Please correct me if I misunderstood any equations.\n\n3. (Minor) Figure captions are not very descriptive. I would suggest to include the main points (Fig. 2, 8) or where to focus on (qualitative figures).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wOrbZ93jJx", "reviewer": "Reviewer_4a6E", "summary": "This paper presents an innovative approach to accelerate video diffusion models without sacrificing quality. By analyzing existing cache - based methods and classifier - free guidance (CFG), the authors identify limitations and redundancies. They introduce FasterCache, which includes a dynamic feature reuse strategy for attention modules and CFG-Cache. The dynamic feature reuse strategy adjusts features across timesteps, maintaining distinction and continuity. CFG-Cache optimizes the reuse of conditional and unconditional outputs. Experimental results on multiple video diffusion models demonstrate significant acceleration while keeping video quality comparable to the baseline, outperforming existing methods.", "questions": "Questions:\n1. Is the effectiveness of the Dynamic Feature Reuse Strategy because the videos generated by existing video generation methods are relatively smooth, with little difference between adjacent frames, e.g. fixed backgrounds? If so, once the motion of the video is large and the adjacent frames change greatly, then is the proposed method still effective?\n\n2. Negative prompt will affect the quality of video generation, which generally improves it. So will CFG-Cache and the separation of high and low frequencies have an impact on negative prompt setting?\n\n3. For Open-Sora-Plan, although FasterCache achieved an acceleration of 1.68x, the generated video has significant varying in pixel domain compared to the original one (PSNR 23.72). What is the reason for this bias? Blurriness, noise, or changes in semantic level?\n\n4. Will the setting of g value in CFG formula 4 (Line 167) affect the performance of FasterCache?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "In terms of originality, the proposed FasterCache method is highly innovative. It combines a dynamic feature reuse strategy that considers subtle differences between adjacent timesteps and a CFG - Cache component for handling conditional and unconditional outputs in a novel way. This is a departure from existing methods and reveals new acceleration opportunities through a pioneering investigation of classifier - free guidance (CFG). \n\nRegarding quality, a thorough analysis of existing cache - based methods and CFG is provided. This includes identifying problems and limitations, and conducting in - depth investigations into feature reuse and CFG outputs, which forms a solid foundation for the method. The experimental design is robust, with testing on multiple video diffusion models and the use of various evaluation metrics. Ablation studies further demonstrate the effectiveness of each component. \n\nFor clarity, the paper has a clear structure, starting with an introduction to the problem and existing solutions, followed by a detailed description of the method in the methodology section. The experimental results are presented clearly, and the discussion and conclusions are well - organized. Complex concepts are explained accessibly, such as through diagrams and detailed descriptions of the key components. \n\nIn terms of significance, the research has practical implications as it can reduce the time and computational resources required for video generation, which is important for applications like video content creation, virtual reality, and augmented reality. It also advances the field by filling gaps in existing research on video diffusion models, addressing limitations of current acceleration methods, and inspiring further studies on improving efficiency and performance.", "weaknesses": "Hyperparameter Explanation. Some of the hyperparameters used in the method, such as the weighting function are not explained in sufficient detail. Although default values are provided and it is mentioned that they work well for most models, a more in-depth discussion on how these hyperparameters are chosen and their impact on the performance for different datasets and models would make the method more reproducible and understandable.\n\nUncertainty in complex scenes. The paper mentions that in complex scenes with substantial video motion, the method may occasionally produce degraded results. This may limit the practical applicability of the method once video generation model baselines are imporved and complex video content is common. More discussions refer to the Question 1.\n\nWeak theoretical underpinning: The paper focuses more on the empirical evaluation and practical implementation of the FasterCache method. Although the experimental results are strong, the theoretical foundation behind some of the proposed techniques could be strengthened. For example, a more in-depth analysis of why the dynamic feature reuse strategy and CFG-Cache work as expected in different scenarios from a theoretical perspective would add more credibility to the method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "t1KGMPgH1Y", "reviewer": "Reviewer_JHfK", "summary": "This paper propose a training-free method for efficient video generation. Their method is based on the attention feature map and output redundancy observed in the forward pass of the denoising process. To this end, they propose to (1) use the feature map dynamics to restore the missing details in the previous naive feature map reuse method. (2) use the frequency difference to compensate the cached conditional output in CFG. The proposed method has been validated with several models and showed promising results.", "questions": "Please refer to the weakness section.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is well-written and easy to read.\n- The proposed attention feature map dynamics could be a generic method and applied into all other cache based efficient generation methods.\n- the proposed frequency residual method for CFG is novel and seems also a generic method that can be applied into CFG for all diffusion based image/video generation models.", "weaknesses": "- Distillation based efficient generation methods are less discussed and not added into the comparison. I can understand that this work mainly focused on the cache based method. But:\n  - What's the delta of this method compared to distillation based method?\n  - Are they compatible to each other? If not, in real world applications, which method we need to use?\n\n- the overall method seems to perform better than all other methods. What's the contribution of each individual method? What the cost of additional computation in each individual method?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rVncPpxMHC", "reviewer": "Reviewer_zNbc", "summary": "This paper presents a method to accelerate the computation for video diffusion models. Based on the observation that adjacent-time features exhibit subtle differences,  the authors propose a dynamic feature reuse scheme. Based on the observation that the conditional and unconditional generation features have strong similarity in CFG, the authors propose a CFG-Cache scheme to save computations of unconditional stage.", "questions": "1. what is the reason of using equation (5) for feature reuse? it seems to be quite heuristic.\n2. why in cfg, the similarity of two stages has such a wired characteristics (time-varing low and high frequency similarity)? there should be more analysis and insight.\n3. in terms of saving the computation of unconditional stage, how about existing and well-adopted methods such as \"On Distillation of Guided Diffusion Models\"? \n4. how does your method work with video generation models using flow matching?", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "this paper is relatively well organized and written.", "weaknesses": "this paper lacks novelty and insight. the proposed method is not solid in terms of logical and theoretical justification. the speedup of computation could be achieved using other methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)