BWCache: Accelerating Video Diffusion Transformers through Block-Wise Caching
Authors: Hanshuai Cui
Zhiqing Tang
Zhifei Xu
Zhi Yao
Wenyi Zeng
Weijia Jia
Venue: ICLR 2026 Poster (2026)
Keywords: Diffusion Model
Video Generation
Cache
Primary area: generative models

Abstract: Recent advancements in Diffusion Transformers (DiTs) have established them as the state-of-the-art method for video generation. However, their inherently sequential denoising process results in inevitable latency, limiting real-world applicability. Existing acceleration methods either compromise visual quality due to architectural modifications or fail to reuse intermediate features at proper granularity. Our analysis reveals that DiT blocks are the primary contributors to inference latency. Across diffusion timesteps, the feature variations of DiT blocks exhibit a U-shaped pattern with high similarity during intermediate timesteps, which suggests substantial computational redundancy. In this paper, we propose Block-Wise Caching (BWCache), a training-free method to accelerate DiT-based video generation. BWCache dynamically caches and reuses features from DiT blocks across diffusion timesteps. Furthermore, we introduce a similarity indicator that triggers feature reuse only when the differences between block features at adjacent timesteps fall below a threshold, thereby minimizing redundant computations while maintaining visual fidelity. Extensive experiments on several video diffusion models demonstrate that BWCache achieves up to 2.6$\times$ speedup with comparable visual quality.

Published as a conference paper at ICLR 2026
BWCACHE: ACCELERATING VIDEO DIFFUSION TRANSFORMERS THROUGH BLOCK-WISE CACHING
Hanshuai Cuil?. Zhiqing Tang?: Zhifei Xu², Zhi Yao!?, Wenyi Zeng', Weijia Jia"
'School of Artificial Intelligence, Beijing Normal University, Beijing 100875, China Institute of Artificial Intelligence and Future Networks, Beijing Normal University, Zhuhai 519087, China
ABSTRACT
Recent advancements in Diffusion Transformers (DiTs) have established them the state-of-the-art method for video generation. However, their inherently se- quential denoising process results in inevitable latency, limiting real-world ap- plicability. Existing acceleration methods either compromise visual quality due architectural modifications fail to reuse intermediate features at proper granularity. Our analysis reveals that DiT blocks are the primary contribu- inference latency. Across diffusion timesteps, the feature variations of DiT blocks exhibit a U-shaped pattern with high similarity during intermediate timesteps, which suggests substantial computational redundancy. In this paper. we propose Block-Wise Caching (B a training-frre method to acceler- ate DiT-based video generation. BWCacche dynamically caches and reuses fea- tures from DiT blocks across diffusion timesteps. Furthermore, we introduce a similarity indicator that triggers feature reuse only when the differences be- tween block features at adjacent timesteps fall below threshold, thereby mini- mizing redundant computations while maintaining visual fidelity. Extensive ex- periments several video diffusion models demonstrate that BCCache achieves up to 2.6x speedup with comparable visual quality. The code is available at https: http::::///ittu..... com hhwl 13/p1113//CCBCCCc
80.5
BWCache
Open-Sora
0.93
BWCache
0.91
BWCache
28.0
BWCachee
Quality (VBench)
79.8
(1-PPPPS)
0.88
Quality (SSIM)
0.86
Quality (PSNR)
25.5
79.1
PAB
0.83
PAB
0.81
23.0
PAB
78.4
TeaCache
Butter
0.78
0.76
TeaCache
Butter
20.5
TeaCache
TeaCache
12
22
34
16
28
22
34
Latency (s)
Latency (s)
Latency (s)
Latency (4)
Figure 1: Quality-latency comparisons for video diffusion models. Visual quality versus latency curves are presented for the proposed BWCache method, PAB, and TeaCache using Open-Sora. -ora. BWCache demonstrates significantly superior visual quality and efficiency compared both PAB and TeaCache. Latency evaluated on a single NVIDIA A800 GPU for generating 51 frames, 480P
videos.
INTRODUCTION
Diffusion Models (DMs) Song & Ermon (2019) Ho al. 2020 Dhariwal & Nichol 2021) have emerged as the predominant technology in Generative AI (GenAl) due to their ability to generate
high-fidelity images and videos Rombach al. (2022). Early DMs primarily adopted the U-Net architecture Ronneberger et al. (2015); Ramesh et al. (2022): Saharia et al.) (2022); Blattmann et al. (203)). Recently, the success of Sora Brooks al. (2024) has spurred adoption of the Diffu- sion Transformer (DiT) architecture Peebles & Xie (202)) Bao et al. (2023) for video generation,
"Corresponding author

---

Published as a conference paper at ICLR 2026
Cache Indicator: 2 Llye (Ri <
Cache
Rouse
Diffusion Process
Latent
Noise
DiT Blocks
N DIT Blocks
x DIT Blocks
N DiT Blocks
Text
Prompt
f
Figure 2: Overview of the BWCache. An indicator based on adjacent timestep differences in block features determines whether to reuse the cache. If conditions are met, subsequent timesteps reuse cached blocks; otherwise, blocks are recomputed and the cache updated.
achieving state-of-the-art performance. Video generation with DiT requires the sequential compu- tation of multiple DiT' blocks, including spatial and temporal DiT blocks Ho et al. (2022) However, these blocks must be executed in sequence, which considerably slows down inference. As result, accelerating DiT inference has become one of the most pressing challenges faced by GenAI.
Traditional acceleration techniques for DMs primarily focus on architectural modifications such as distillation, pruning, and quantization Wang et al. (2024); Chu et al.] (2024); Yang et al. (2024); Zhang et al. (2024a); Kang et al. (2024); Zhou al. (2024). Although these methods reduce model complexity, they also compromise generation quality. More importantly. they require substantial computational resources and extensive training data to fine-tune, which is impractical for large pre- trained DMs. A training-free alternative is to cache the intermediate features of DMs to accelerate inference Ma et al. (2023); Zhang et al. (2024b); Zhao et al. (2024b); Kahatapitiya et al. (2024); Liu et al. 2022). By reusing these intermediate features across timesteps, these approaches eliminate redundant computations and significantly improve generation efficiency.
Recent work has explored block-level temporal stability in DiTs. Skip-DiT Chen al. (2025) ad- dresses feature instability by introducing Long-Skip Skip-Connections to stabilize DiT block features, enabling efficient caching. However, this method requires modifying the model architecture and retraining from scratch, which is impractical for large pre-trained models and limits their applica- bility. ProfilingDiT Ma et al. (20256) indicates that redundancy is not uniformly distributed across timesteps. While their work does not focus on block-wise analysis in DiTs, their findings are concep- tually aligned with our observation that mid-timestep regions exhibit higher redundancy. However, this method requires storing extensive intermediate features across multiple timesteps for profiling, leading to significant memory overhead.
Despite the proven effectiveness of caching in accelerating DM inference, two critical challenges remain unresolved. First, DiT is a complex structure, considering caching at coarse granularity (e.g., timesteps) Lv et al. (025): ; Liu et al.] (2024); Ma et al. (2025b) may result in the loss of essential information, while considering caching at fine granularity (e-g., attention) Kahatapitiya
eet al. (2024); Zhao et al. (2024b); Yuan et al. (2024): Zhang et al. (2025) often fails to deliver significant acceleration. Thus, the first challenge is to identify which features are appropriate for caching. Second, many existing methods assume high similarity between features across adjacent timesteps, sharing features between them to speed up inference Zhao et al. (2024)): et al. (2025); Wimbauer et al.] (2024); [Selvaraju al. (2024); Liu et al. (2025c); Chen al. (2025). In reality, however, feature similarity varies greatly depending on the generation task and specific inference process, and naive feature reuse can often degrade output details. Therefore, the second challenge is to determine when cached features should be reused or updated.
In this paper, we propose Block-Wise Caching (BWCache), a training-free method to accelerate DiT-based video generation. This method can be seamlessly integrated into most DiT-based models as a plug-and-play component during the inference phase. The core idea is cache the features from all DiT blocks at certain diffusion timesteps and reuse them across several subsequent steps. Specifically, as shown Figure 2 DiT-based video generation typically involves providing a text prompt and initializing latent noise, which is then refined through multiple iterative denoising steps, and finally reconstructed into video frames using a VAE decoder. To determine when to reuse cached features, we introduce a similarity indicator based on the differences between block features from adjacent timesteps. Periodic recomputation is applied to mitigate potential latent drift. Our method is training- ffree and memory-effciient, using lightweight similarity indicator to dynamically determine cache reuse without extensive feature storage, making it directly applicable to existing pre-trained models without architectural modifications. We evaluate BWCache across several video
2

---

Published as a conference paper at ICLR 2026
diffusion models, including Open-Sora Zheng et al. 20024). Open- Seen-Sora-Bie Sora-Plan Lin al. (2024a) Latte
2025). , and Hunyuan Video Kong et al. (2024). As shown in Ma et al. (2025a). Wan 2.1 Wan et al.
Figure BCCache achieves superior performance at comparable computational costs to TeaCache
Liu et al.
2024) and PAB /Zhao et al. 2024b). The main contributions are summarized as follows.
We analyze the components of DiT-based models during the denoising process, revealing the dynamics within DiT blocks under different generation tasks and highlighting their similarities across specific diffusion timesteps.
We propose BWCache, training- frreingg-eeeeeeee method compatible with DiT-based models that caches DiT block features and reuses them across multiple diffusion timesteps, thereby accelerating video generation inference.
Experiments demonstrate that our method outperforms existing baselines in terms of inference speed and video generation quality, justified through both ablation studies and qualitative compar- isons.
2
RELATED WORK
2.1 DIFFUSION MODEL
Early video generation models, such as Variational Autoencoders (VAEs) Kingma et al. (2013) and Generative Adversarial Networks (GANs) Goodfellow al. (2014). face several challenges, includ- ing blurry outputs, training instability. and limited text-video alignment. The emergence of DMs Ho al. (2020); Dhariwal & Nichol (2021) addresses these limitations by achieving high-quality and diverse video generation through a gradual denoising process. Initially, these models employed U-Net architectures, which demonstrated impressive results in both image and video generation tasks Ramesh et al. (2022); Chen et al. (2024a): Wei et al. (2024). More recently, advancements in DiT-based architectures Peebles & Xie (2023) Bao al. (2023) have further improved scalability
and generalization over traditional U-Net-based DMs. The success of the Sora model Brooks et al.
(2024), which utilizes the DiT architecture for long-form video generation, has led to a growing
body of research adopting DiT as the noise estimation network Hatamizadeh et al. (0024) Wang
ct al. 2025): Li al. 2025).
2.2 DIFFUSION MODEL ACCELERATION
Despite the remarkable performance of DMs, their high inference costs limit their applicability in real-world applications. Existing acceleration techniques for DMs can generally be categorized into two groups The first category relies on architectural changes, such distillation simplifying net- works Kang et al. (2024); Zhou et al. (2024); Salimans et al. (2024), pruning removes redundant parameters Yang et (2024); Zhang et (2024)); Castells et al. (2024), quantization reduces precision Wang et al. (2024); Chu et al. (2024); Huang al. (2024a). and others. However, these approaches often require additional resources for fine-tuning or optimization. The second group con- sists of training-free free methods that accelerate inference by caching intermediate features of diffusion models. DeepCache Ma et al. (2023) accelerates diffusion models by caching high-level features from sequential denoising timesteps. T-GATE Zhang al. (2024b) enhances image generation ef- ficiency by caching attention outputs. PAB Zhao et al.] (2024b) leverages the attention redundancy to significantly accelerate the diffusion process. AdaCache Kahatapitiya et al., (2024) accelerates video generation by selectively caching computations based on video content. TeaCache Liu et al. (2024) utilizes timestep embeddings dynamically determine which computations cache
celerate inference. Although these training- free methods enhance diffusion efficiency, they still face significant challenges in balancing generation quality with computational cost.
METHODOLOGY
3.1 PRELIMINARIES
Denoising Diffusion Models. Consider the data Xo q(x) sampled from real distribution. The forward diffusion process incrementally adds Gaussian noise over steps, producing a se- quence X1, XT. As increases, x, progressively loses distinguishable features, asymptotically
3

---

Published as a conference paper at ICLR 2026
Block
Embedding
VAAE
Other
S8i
70%
18%
5%
7%
0.5
20
(b)
Latency of different compo-
nents
Prompt 1: Sunset the
Step Index
1.0
Ascregated Relative
Prompt
Black Index
15
0.
20
Prompt 2: A bus the right of a traffic light, front view.
20
Step Index
10
15
20
30
Step Index
(a) Two prompts and corresponding generated vidcos. The heatmaps track the denoising evolution across 28 blocks over 30 steps.
(c) Quantitative block difference
Figure 4: Analysis of the block in the DiT-based model.
approaching isotropic Gaussian distribution when T co. With the Markov chain assumption, is expressed as:
((x,/x,-1) == (xx: V1 Br X, B,1),
(1)
where ((xe(xr-1) is the posterior probability. The step sizes are controlled by a noise schedule Bi...., Sr. The reverse process samples from g(xx-1 g(x)=1|x;) reconstruct data from noise XT (O, I). Similarly, the backward diffusion process can be written as:
p x,) = (x, M6 (x (), Ex(x;, ()),
where μg and Eg are learned parameters defining the mean and covariance, and p. denotes the probability of observing x given Xt.
DiT Block. The DiT block Ho et al. (2022); Peebles & Xie
2023) follows the two-stage Transformer design to modulate the dynamics of diffusion features across both space and time. Each denoising step involves sequential processing through consecutive DiT blocks, which is illustrated in Figure] 3 For the i-th DiT block (i E N)) in timestep f, given an input hidden state he,s, the block first performs Adaptive Layer Normalization (AdaLN) Xu et al. (2019) followed by multi-head self-attention and cross-attention:
VAE Encoder
Linear
Linear
Cross-
Attention
DIT Blocks
Self- Attention
Embed
Noise
K/s = Attention (Adaa((h.)L)) + hy,ss
Figure Overview of DiT-based video generation models.
where the residual connection preserves input information. subsequent AdaLN-modulated MLP enhances feature expression:
Mr MLP(AdaLN((k..) M4
(4)
The output hidden state h", then becomes the input to the next DiT block (c.g., hi.41 = h(,2).
3.2 ANALYSIS
Costs Diffusion Models. The video generation process using DMs comprises multiple com- putational components, including the DiT blocks, timestep and text embeddings, VAE, and others. Figure b) shows the time distribution for each part of the video generation process. As illustrated, the DiT blocks require considerably more computation time than the other components. Further- more, this proportion increases further as the video length and resolution grow, posing a substantial challenge to the efficiency of video generation.
Block Feature Variation. To better understand the internal behavior of DiT blocks, we introduce the relative L.1 distance Liu et al. (2024) to measure the feature variations of each block between
4

---

Published as a conference paper at ICLR 2026
spaated Relathve L1
Agge saaaggatdd Relatkes
6.00
2.1
1.87
2.30
6:66
1.26
2.66
1.39
1.22
6.54
0.59
2.08
0.92
2.03
Diflusion
Diffusion
Diflusion
Diffusion
Diffesion
(a) Open-Sora
(h) Open-Sora-Plan
(c) Latte
(d) Wan 2.1
(c) Hunyuan Video
Figure 5: Aggregation relative I.1 of different models.
adjacent timesteps. For the i-th block at the t-th step, the block-wise wise relative L1 distance calculated as follows:
|14e, h4t1,il1
(5)
|/4+1,011
A smaller L1 value indicates minimal feature variation within the block between adjacent timesteps, while larger value shows sigmificant changes. In Figure Ala, we present two videos generated from different prompts. The video generated with Prompt depicts relatively static scene, whereas the video generated with Prompt 2 exhibits more dynamic content (e.g.. bus mov- ing closer from a distance). We employ heatmaps to quantify the changes block features during the generation process for both videos. In these heatmaps, the color intensity reflects the relative degree of block feature variation at different timesteps, with darker regions indicating greater differ- ences. From the figures, we observe two key insights: 1) the colors are darker in the initial and final timesteps, indicating that block features change across timesteps; 2) compared to the figure below, the darker regions in the figure above occupy smaller proportion, suggesting that different prompts exert varying influences on the features of the blocks.
Similarity and Diversity. To gain more intuitive understanding of the changes in block feature different timesteps, we aggregate the block-level feature differences within each timestep and define the aggregated relative LI distance. The equation is as follows:
N
ARL1(() Σ (ha,))
(6)
where N is the total number of DIT blocks. ARL1(() quantifies the feature change at given diffusion step by summing the relative LI distances across all blocks.
Figure 4 illustrates the aggregated relative LI distance for two prompts. Generally, the differ- ences in block features between adjacent diffusion timesteps exhibit a U-shaped pattern. In the early timesteps, the LI distance is high, indicating rapid changes in noise predictions. The middle timesteps gradually stabilize, showing considerable redundancy in block calculations. While in the final timesteps, the aggregated relative I.1 distance increases, suggesting transition from struc- tured noise to high-fidelity video. Notably, for simpler or more static scenes, the feature variations among blocks stay minimal, explaining why Prompt exhibits longer duration of stable timesteps compared to Prompt 2. More detailed analysis of block feature behavior in Appendix A
To further validate the consistency of this U-shaped pattern across different architectures and training methodologies, we analyze the aggregated relative LI distance across five video diffusion models, as shown in Figure The theoretical foundation for this U-shaped pattern can be understood through the lens of frequency domain analysis Qian et al. (222)). During early timesteps, low- wwwffreuenccy components recover, causing high feature variation (left U-shape). Middle timesteps stabilize these, minimizing changes (U-bottom). Final timesteps recover high-frequency details, increasing varia- tion again (right U-shape). However, we observe that some models exhibit more pronounced early- stage feature oscillations compared to other models. This can be attributed to noise schedule and sampling implementation issues Lin et al. (2024b). Specifically. Open-Sora-Pee samplers (i.e. PNDM) that do not start from the last timestep (t T), creating training inferrncce discrepancy where the model is forced to handle inputs that deviate from pure Gaussian noise, po- tentially causing initial timestep feature prediction instability and oscillations.
uses non- non-linear
5

---

Published as a conference paper at ICLR 2026
3.3 BWCACHE
As illustrated in Figure 4 intermediate stages of video generation frequently exhibit redundant block computations. To minimize computational redundancy and accelerate inference, we propose Block- Wise Caching (BWCache). Instead of computing each DiT block at every timestep, we selectively reuse block features cached from the previous timestep. We employ the average of the accumulated relative L.1 distance as similarity indicator to determine when to reuse the cached blocks:
Σ (he,) < <8,
where Liyel is defined in 5). and is the similarity threshold. The threshold represents an acceptable level of variation, and when features change more, fewer blocks are cached, naturally adapting to the scene dynamics. Specifically, at each timestep, we calculate the relative L11 distance between each DiT block and its counterpart from the preceding timestep. After each timestep, cache all block features and calculate the arithmetic mean of their relative L1 distances. this mean is less than the indicator threshold 6, block computations can be skipped for subsequent timesteps starting from - 1, allowing the reuse of cached features; otherwise, the block will be recomputed. A smaller threshold reduces cache reuse, while a larger threshold speeds up video generation, but may adversely affect video quality. The appropriate threshold 5 should be chosen to optimize inference speed without compromising visual quality.
Reusing DiT blocks in the final diffusion timesteps significantly degrades visual fidelity, as these timesteps represent the critical phase where the latent space transitions from structured noise high-quality video. Once BWCache is triggered at the k-th step, caching reuse is restricted to the first half of the remaining steps, while the second half is explicitly computed. Therefore, the last k/2 steps are always recomputed, ensuring thorough refinement during the most sensitive stage of generation.
3.4 PERIODIC CACHE RECOMPUTATION
Continuously reusing the same block may result in latent drift and gradually erase fine-grained details Liu al. (225a)). To prevent this cache stagnation, adopt the progressive computation strategy from PAB Zhao et al. (0024b): within the caching interval, each DiT block is periodically recomputed defined reuse interval R. Specifically, after computing the block time step its features are cached and reused for the subsequent R steps (i.e., [t - 1, - R)), and the block is then updated step R 1. This process formalized as:
OH = f..., h" 1o"""
1V R.
(8)
cached reuse It steps
eached
where On denotes the output of the DIT block across all steps, and N represents the DiT block output calculated step which can be determined using Eq. 4).
EXPERIMENTAL RESULTS
4.1 EXPERIMENT SETUP
Base Models and Compared Methods. We apply the proposed acceleration techniques to various video generation diffusion models, including Open-Sora Zheng et al. (2024). Open-Sora-Plan Lin al. (2024a). Latte Ma al. (2025a). Wan 2.1 Wan al. (2025), and SunyuanVider Kong et al. 2024). We compare method with recent effective video generation techniques such as DiT Chen et al. (2024b). T-GATE Zhang et al. (2024b). PAB Zhao et al. (2024b), and TeaCache et (204 to highlight advantages. The inference configs of base models are shown in Appendix
Evaluation Metrics and Datasets, To evaluate the performance of video generation acceleration methods, primarily focus two aspects: visual quality and inference efficiency. We utilize VBench Huang et al. (2024b), LPIPS /PIPS/Zhang et al. (2018), PSNR, and SSIM Wang & Bovik (2002) for visual quality assessment. VBench is comprehensive benchmarking suite for video generation

---

Published as a conference paper at ICLR 2026
Visual Quality
Efficiency
Model
Method
VBencht
IPIPS|
SSIM
PSNR
Speedupt
Latency(S))
Original
80.33%
44.56
A-DIT
78.21%
0.5692
0.4811
11.91
1.03x
43.26
T-GATE
77.61%
0,3495
0.6760
15.50
1.19x
37.45
Open-Sora
PAB
78. 10%
0.2134
0.7798
22.02
1.38x
32.38
TeaCache
79.16%
0,1496
0.8104
22.39
1.50x
29.64
FastcrCache
79.21%
0.1165
0.8435
23,99
1.35x
32.03
BWCache
80.03%
0.0879
0.8854
27.05
1.61x
27.68
Original
80.88%
1x
99.65
A-DIT
77.55%
0.5388
0.3736
13.85
1.01
98.66
T-GATE
80.15%
0,3066
0.6219
18.80
1.18x
84.45
Open-Sora-
PAB
80.30%
0.2781
0.6627
19.49
1.36X
73.41
Plan
TeaCache
80.32%
0.1965
0.7502
21.50
4,41x
22.62
CasterCache
80.61%
0.1190
0.8103
24.06
1.51
65.79
BWCache
80.82%
0.1001
0.8435
25.87
2.24x
44.49
Original
78.95%
26.90
A-DIT
52.00%
0.8513
0,1078
8.65
1,0x
26.37
T-GATE
75.42%
0,2612
0.6927
19.55
1.13x
23.81
PAB
76..2%%
0.4625
0.5964
17.03
1.21
22.16
Latte
TeaCache
77.40%
0.1969
0.7606
22.19
1.86X
14.46
FasterCache
78.21%
0.2442
0.7304
20.66
19.56
Skip-DiT
75.76%
0,1403
0.8087
25.50
1.65
16.28
BWCache
78.28%
0.1399
0.8181
26.46
1.90
14.16
Original
82.
1x
912
PAB
80.04%
0.1623
0.7643
21.32
1.19x
767
Wan 2.1
TeaCache
81.73%
0.2407
0.6593
18.62
1.41
644
BWCache
81.99%
0.0782
0.8539
25.86
2.00
457
Original
82.29%
1x
1122
PAB
81.73%
0,1045
0.8341
26.69
1.08x
1039
Huny Video
TeaCache
82.13%
0.1630
0,8052
24.37
2.27x
493
BWCache
82.48%
0.0794
0.8903
29.91
2.60
433
Table 1: Comparison of visual quality and efficiency on single GPU. Video generation speci- fications: Open-Sora (51 frames, 480P), Open-Sora-Plant (65 frames, 512x512), Latte (16 frames, 512x512). Wan 2.1 (81 frames, 480P), Hunyuan Video (129 frames, 544P). LPIPS, SSIM, and PSNR are calculated against the original model results.
models. LPIPS, PSNR, and SSIM measure the similarity between videos generated by the acceler- ated methods and the original models. Several evaluation metrics detailed in Appendix C To
assess inference efficiency, we use the inference latency as metrics. In our main experiments, we
generate over 1,000 videos for each model under each baseline using prompts sourced from Open-
Sora gallery Lin et al. (2024a), VBench benchmark Huang et al. (2024b), and T2V-Compeench Sun al. 2024).
Implementation Details. All experiments are conducted on NVIDIA A800 80GB GPUs using PyTorch. FlashAttention Dao et al.] (2022) enabled by default across all experiments. Unless otherwise specified, ablation studies and qualitative analyses generate 51 frames, 480P videos in 30 steps. is set 0.15 and reuse interval R is set 10% of total timesteps by default.
4.2
MAIN RESULTS
Quantitative Comparison. The experimental results in Table demonstrate the effectiveness of BWCache across multiple DiT-based video generation models. Our proposed BWCache outper- forms existing optimization methods in both visual quality and efficiency. For instance, on Open- Sora, BWCache achieves advanced visual metrics while reducing latency by 1.61 x compared to the original model. Notably, BWCache surpasses the training- free TeaCache method and maintains competitive acceleration performance. The block-wise method of BWCache leverages DiT block redundancies more effectively without sacrificing visual details. The timestep-level caching of Tea- Cache less adaptive to intra-timestep variations, leading to higher quality degradation. It is worth noting that TeaCache achieves faster acceleration than BWCache This is because there is significant variation in block features during the early inference stages of the Open-Sora-Plan model. To maintain high-fidelity output quality, BWCache strategically skips
the Open Open-Sora-Plan model.

---

Published as a conference paper at ICLR 2026
Original
PAB
TeaCache
BWCache
Open: Sora
Open Sora .Plan
Latte
Wan 2.1
Hunyuan Video
Figure 6: Comparison of visual quality with the compared method. BWCache outperforms PAB and
TeaCache in both visual quality and efficiency.
50
120
1000
Latency (5)
35.7
Latency (s)
Latency
2.0x
25
.... S.
60
10:04
500
4.34
12.2
Original
GPU
Original
Orignnal
2GPU
(a) Open- Sora (51 frames, 480P)
(c) Open-Sora-Plaa Plan (51 frames)
(c)
Wan 2.1 (81 frames, 480P)
2800
Latency (s)
Latency (s)
Latency
50
6.3k
4,7x
B.x....
13
1700
10
Orignal 1PG
GGG
GGU
GPU
Orignnl 1GPU
GPU
4GU
GPU
Orignnal 1GPU 2
4GP GPU
Open-Sora (102 frames, 480P)
(d) Open-Sora-Plan (221 frames)
Wan 2.1 (81 frames, 720P)
Figure 7: Inference efficiency of BWCache at different video lengths and resolutions.
caching during the first few denoising steps where feature oscillations are most pronounced (as vi- sualized in Figure(5b). thereby sacrificing some acceleration potential in favor of generation quality. We also compare with Skip-DiT Chen et al. (2025) on the Latte model, shown in Table [ While Skip-DiT achieves competitive acceleration (1.65x speedup). it requires architectural modifications and model retraining, which significantly limits its practical applicability. The Skip-DiT repository only provides pre-trained model for Latte, and according to their documentation, training requires approximately H100 GPUs for about one week with 300k text-video pairs. This substantial train- ing requirement makes it impractical to evaluate Skip-DiT on other models compare it fairly across different architectures. We further compare with FasterCache, which is another training- free caching- based acceleration method. The official FasterCache repository provides support for Open- Sora, Open-Sora- Sora-Plan, and Latte models. BWCache consistently outperforms FasterCache across all three models in terms of visual quality metrics. More experiments be found in Appendix [
8

---

Published as a conference paper at ICLR 2026
Latencyis))
LPIPS
SSIM
Method
1 A800
2 A800
4 x A800
8 x A800
original steps
29.20
0.3139
0.6922 0.8856
16.39
Open- Spen-Sora (204 frames, 480P)
BWCache, steps
27.68
0.0879
27:08
original PAB
71.3(27x))
37.755.1x)
24.107.9x)
150.7(122))
55.3(3.3%)
29.86.12)
19,809 1x)
Table 3: Caching mechanism v.s. inference steps.
TeaCache
11..0017%)
47.0640x)
24,6(7773)
(44(1322X)
BWCache
78.1224k)
30.366.3%)
16.91113%%)
11.1(1.22))
Open-Soru-Plan (221 frumes, 512x512)
original
284,3(1x)
154,3(18x)
81,2(3.5%)
47.166.0%)
112.512.5x) 211,(1.33))
60.0447%)
14.98.1x)
TeaCache
48.2559k)
26.9116)))
15,9(1799))
10.128.1x)
BWCache
109.12266))
60,014.7%)
32.8(8.7%)
21.(13.3%))
Latte (16 frames, 512x512)
original
25.9(1x)
15.1(1.7%)
9.3(2.8x)
8.9(29%)
PAB
21.001.2x)
12.2(2.1x)
8.3(3.1x)
7.9(1.0x)
TeaCache
17.501.5x)
7.3(3.5))
5.6(4.66x)
5.01522x)
Open-Sora 30 steps
Open:
BWCacbe
7.3(66))
4.5(5.8%)
4.0(6.5%)
19 steps
30 steps
Table 2: Inference efficiency when scaling to multiple GPUs with DSP.
Figure 8: Accelerate video generation by re- ducing inference steps and using caching re- spectively.
Reuse Rate
100%
Fast
Mid
Slow
50%
0%
Step
29 30
Figure Visualization for the block reuse rate at each step.
Visual Quality Comparison. As shown in Figure 6 we visualize the videos generated by our method compared the original model, PAB, and TeaCache. The results demonstrate that BW- Cache outperforms other methods in visual quality with lower latency. More visual results are illustrated Appendix
4.3 SCALING ABILITIES
Scaling to Multiple GPUs. Table summarizes the inference efficiency of various methods when scaled across multiple GPUs using Dynamic Sequence Parallelism (DSP) Zhao et al. (2024a). Uti- lizing the DSP method, GPUs are interconnected via high-bandwidth NVLink, which provides low- latency, high-throughput communication for distributed workloads. A single GPU often cannot utilize a full batch size due to memory constraints, leading to suboptimal hardware utilization. In contrast, the DSP method overcomes this by employing dimension-awee awaar sharding and efficient all-to-all operations to minimize communication overhead, thereby enabling more effective scaling. The experiments, conducted using several models, reveal that BWCache consistently outperforms both the original model, PAB, and TeaCache in terms of latency reduction across all tested scenarios. Specifically, BWCache achieves the lowest latencies in almost all cases, demonstrating significant speed-ups as the number of GPUs increases. These results validate BWCache as universally effec- tive acceleration method for DiT-based models under multi-GPU deployments.
Performance at Different Length and Resolution. Figure demonstrates the inference efficiency of BWCache across various video lengths and resolutions using Open-Sora. Open-Sora-Plann and Wan 2.1 models. The results show that latency increases significantly as the length and resolu- tion of the video increase, but BWCache consistently performs better than the original model in all configurations. BWCache exhibits particularly notable acceleration advantages in processing high-resolution long videos. For example, with 8 GPUs, BWCache attains a latency of just 1 11.08 seconds for the Open-Sora model with 204 frames at 480P, representing a remarkable 17.16x speed- up compared to the base model. This trend is also reflected in other configurations, where BWCache consistently provides substantial improvements in efficiency. Overall, these results validate its capa- bility to improve inference speeds across various video lengths and resolutions, making it an efficient solution for real-time video generation.

---

Published as a conference paper at ICLR 2026
Interval
Latency(s))
ViBecch
LPIPS
SSIM
PSNR
Threshold
Reuse Rate
Viienc?
LPIPS]
SSIM
PSNR1
5%
34.70
80.28%
0.045]
0.9250
30.72
0.25(Fast)
59.00%
79.03%
0.1935
0.7829
21.39
10%
27.68
$0.03%
0.0879
0.8851
27.05
0.20/Mid)
53,05%
79.42%
0.1486
0,8267
23.45
15%
27.33
79 8484
0.1065
0.5009
26.15
0.1 (5((low)
41.38%
80.03%
0.0879
0.8854
27.05
79.4885
0.1415
34.
25%
25.76
79.31%
0.1567
0.8360
24.32
Table Impact of different reuse rates.
Table 5: Impact of different reuse intervals.
4.4 ABLATION STUDIES
Reducing Inference Steps. When generating videos using the DiT model, the impact of the infer- ence steps must be considered. In this paper, the default setting for the inference steps in Open-Sora 30. To accelerate video generation, the number of inference steps can be reduced. We compare the performance of the original model with reduced inference steps and BWCache in terms of visual quality and latency. The results show that, while producing similar latencies, BWCache maintains excellent visual quality, as shown in Figure [ Specific data on visual quality and efficiency is detailed in Table 3 Overall, when the number of inference steps is reduced, sampling trajectories deviate from the theoretical optimal path, which amplifies accumulated errors in the noise estimation network. This results in issues such as blurred video details and reduced color saturation.
Quality-Efficiency Trade-Off. Figure T compares the quality-latency trade-off of BCCache, PAB, and TeaCache. The indicator threshold 5 in Eq.(7) is 0.15, 0.20, and 0.25. The results clearly demonstrate that BWCache outperforms PAB and TeaCache in visual quality across all evaluated metrics, indicating a notable improvement in the fidelity of video generation. For each metric, BWCache not only achieves higher quality scores but also maintains lower latency, which reflects its efficient processing capabilities. Overall, BWCache achieves improved visual quality at lower latency than PAB and TeaCache.
Reuse Rate. Table illlusraaes how varying the reuse rates affects performance. It reveals that the "Fast" configuration achieves the highest reuse rate 59.00%, resulting the lowest latency of 19.16 seconds. The "Slow" configuration achieves a reuse rate of 41.38%, but the visual quality the highest among all configurations. Visualized results in Figure 9 complement these findings by providing the reuse rates of blocks across each timestep for various thresholds. This figure illustrates the dynamic nature of block reuse at different timesteps, with evident fluctuations that suggest varying block reuse efficiency at different stages of video generation. In general, the block features in the middle stage of the generation change less, and a high reuse rate is obtained, which is consistent with our previous analysis.
Reuse Interval. Table 5 outlines various reuse intervals, ranging from 5% to 25% of all timesteps, and details their corresponding latency and quality metrics. As the reuse interval increases, the la- tency decreases, indicating enhanced generating efficiency. Conversely, while the latency improves with increased reuse interval, there is slight degradation in quality metrics. For instance, VBench scores range from 80.28% at 5% to 79.31 % at 25%, indicating a gradual decline in fidelity. This demonstrates a trade-off between latency and visual quality, where longer reuse intervals can effec- tively accelerate generation but may destroy the details in the output.
5 CONCLUSION
In this paper, we introduced BWCache, novel caching method designed to accelerate video Diffu- sion Transformers. By analyzing feature dynamics in DiT blocks, we established that block features across timesteps had significant computational redundancy. BWCache addressed this by employ- ing dynamic block-wise caching and similarity indicator, which selectively reused block features when their differences fell below a predefined threshold. Periodic recomputation was employed to mitigate potential latent drift. Extensive experiments demonstrated that BWCache achieved robust efficiency and visual quality across diverse generation models, sampling schedules, and video pa- rameters. These results highlighted its potential for real-world applications. The current indicator threshold was preset, and future work will dynamically adjust the threshold according to different generation tasks.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENTS
This work was supported in part by the National Natural Science Foundation of China (NSFC) under Grant 62272050 and Grant 62302048; in part by the Guangdong Key Lab of Al and Multi-modal Data Processing, Beijing Kormal-Hong Kong Baptist University (BNBU), Zhuhai under 2023-2024 grants sponsored by Guangdong Provincial Department of Education; in part by the Institute of Ar- tificial Intelligence and Future Networks (BNU-Zhuhai) and Engineering Center of Al and Future Education, Guangdong Provincial Department of Science and Technology, China; part by Zhuhai Science- Tech Innovation Bureau under Grant No. 2320004002772; and in part by the Interdisci- plinary Intelligent Supercomputing Center of Beijing Normal University (Zhuhai).
REFERENCES
Fan Bao, Shen Nic, Kaiwen Xue, Yue Cao, Chongxuan Li, Hang Su, and Jun Zhu. All are worth words: A vit backbone for diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. (1669-22679, 2023.
Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models large datasets. arXiv preprint arXiv:231..55127, 2023.
Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, Clarence Ng, Ricky Wang, and Aditya Ramesh. Video generation models as world simulators, 2024.
Thibault Castells, Hyoung-Kyu Song, Bo- Bo-Kyeong Kim, and Shinkook Choi. Ld-pruner: Efficient pruning of latent diffusion models using aask-tgnottic insights. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 821-830, 2024.
Guanjie Chen, Xinyu Zhao, Yucheng Zhou, Xiaoye Qu, Tianlong Chen, and Yu Cheng. Towards stabilized and efficient diffusion transformers through -inoncccicooontiionns with spectral con- straints. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 17708-17718, 2025.
Haoxin Chen, Yong Zhang, Xiaodong Cun, Menghan Xia, Xintao Wang, Chao Weng, and Ying Shan. Videocrafter2: Overcoming data limitations for high-quality video diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7010-731222 2024a.
Pengtao Chen, Mingzhu Shen, Peng Ye, Jianjian Cao, Chongjun Tu, Christos-Savvar Bouganis, Yiren Zhao, and Tao Chen. della-dit: A training-free acceleration method tailored for diffusion transformers. arXiv preprint arXiv:2406.01125, 2024b.
Huanpeng Chu, Wei Wu, Chengjie Zang, and Kun Yuan. Qned: Quantization noise correction for diffusion models. In Proceedings of the ACM International Conference on Multimedia (ACM MM), pp. (0995-11003, 2024.
Tri Dao, Daniel Y. Fu, Stefano Ermon. Atri Rudra, and Christopher Ré. FlashAttention: Fast and memory-efficient exact attention with 1O-awareness. In Advances in Neural Information Process- ing Systems (NeurIPS), 2022.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. In Ad- vances in Neural Information Processing Systems (NeurIPS), volume 34, pp. 8780-8794 87900 2021.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in Neural Infor- mation Processing Systems (NeurIPS), volume 27, 2014.
Ali Hatamizadeh, Jiaming Song, Guilin Liu, Jan Kautz, and Arash Vahdat. Diffit: Diffusion vision transformers for image generation. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 37-55. Springer, 2024.
11

---

Published as a conference paper at ICLR 2026
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems (NeurIPS), volume 33, pp. 2840-6851, 2020.
Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffusion models. In Advances in Neural Information Processing Systems (NeurIPS), volume 35, pp. 1633-8646, 2022.
Yushi Huang, Ruihao Gong, Jing Liu, Tianlong Chen, and Xianglong Liu. Tfmq-dm: Temporal fea- ture maintenance quantization for diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7362-7371, 2024a.
Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si. Yuming Jiang, Yuanhan Zhang, Tianx- ing Wu, Qingyang Jin, Nattapol Champaisit, Yaohui Wang, Xinyuan Chen, Limin Wang. Dahua Lin, Yu Qiao, and Ziwei Liu. VBench: Comprehensive benchmark suite for video generative models. Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recogni- fion (CVPR), 2024b.
Kumara Kahatapitiya, Haozhe Liu, Sen He, Ding Liu, Menglin Jia, Chenyang Zhang, Michael S. Ryoo, and Tian Xie. Adaptive caching for faster video generation with diffusion transformers. arXiv preprint arXiv:2411.02399 2024.
Minguk Kang, Richard Zhang, Connelly Barnes, Sylvain Paris, Suha Kwak, Jaesik Park, Eli Shecht- man, Jun- Yan Zhu, and Taesung Park. Distilling diffusion models into conditional gans. In Pro- ceedings of the European Conference on Computer Vision (ECCV), pp. 428-447, 2024.
Diederik Kingma, Max Welling, et al. Auto-encoding variational bayes. Banff, Canada, 2013.
Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXh::2412.0000 2024.
Zijie Li, Henry Li, Yichun Shi, Amir Barati Farimani, Yuval Kluger, Linjie Yang. and Peng Wang. Dual diffusion for unified image generation and understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2779-27900 2025.
Bin Lin, Yunyang Ge, Xinhua Cheng, Zongjian Li, Bin Zhu, Shaodong Wang, Xianyi He, Yang Yc, Shenghai Yuan, Liuhan Chen, et al. Open-sora plan: Open-source large video generation model. arXiv preprint 11 2024a.
Shanchuan Lin, Bingchen Liu, Jiashi Li, and Xiao Yang. Common diffusion noise schedules and sample steps are flawed. arXiv preprint 230.0089.0.. 2024b.
Dong Liu, Jiayi Zhang, Yifan Li, Yanxuan Yu, Ben Lengerich, and Ying Nian Wu. Fastcache:
Fast caching for diffusion transformer through learnable linear approximation. arXiv preprint arXiv:2505.20353, 2025a.
Feng Liu, Shiwei Zhang, Xiaofeng Wang, Yujie Wei, Haonan Qiu, Yuzhong Zhao, Yingya Zhang, Qixiang Ye, and Fang Wan. Timestep embedding tells: It's time to cache for video diffusion model. arXiv preprint arX1:2241111910,, 2024.
Jiacheng Liu, Chang Zou, Yuanhuiyi Lyu, Junjie Chen, and Linfeng Zhang. From reusing to fore- casting: Accelerating diffusion models with taylorseers. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision (ICCV), 2025b.
Joseph Liu, Joshua Geddes, Ziyu Guo, Haomiao Jiang, and Mahesh Kumar Nandwana. Smooth- cache: A universal inference acceleration technique for diffusion transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3229-3322 3238, 2025c.
Zhengyao Lv, Chenyang Si, Junhao Song, Zhenyu Yang, Yu Qiao, Ziwei Liu, and Kwan- Yee K Wong. Fastercache: Training-free free video diffusion model acceleration with high quality. In Pro- ceedings of the International Conference on Learning Representations (ICLR), 2025.
12

---

Published as a conference paper at ICLR 2026
Xin Ma. Yaohui Wang, Xinyuan Chen, Gengyun Jia, Ziwei Liu, Yuan-Fang Li, Cunjian Chen, and Yu Qiao. Latte: Latent diffusion transformer for video generation. Transactions on Machine Learning Research, 2025a.
Xinyin Ma, Gongfan Fang, and Xinchao Wang. Deepcache: Accelerating diffusion models for free.
arXiv preprint arXiv:2312.00835 2023.
Xuran Ma, Yexin Liu, Yaofu Liu, Xianfeng Wu, Mingzhe Zheng, Zihao Wang, Ser-Nam Lim, and Harry Yang. Model reveals what to cache: Profiling-based feature reuse for video diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 17150-17159, 2025b.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 4195-4205, 2023.
Yurui Qian, Qi Cai, Yingwei Pan, Yehao Li, Ting Yao, Qibin Sun, and Tao Mei. Boosting diffusion models with moving average sampling in frequency domain. In Proceedings of the 1EEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hicrarchical text- conditional image generation with clip latents. arXiv preprint arXiv:2204. 061255 1(2):3, 2022.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEF/CVF Con- ference on Computer Vision and Pattern Recognition (CVPR), pp. 10684-10695, 2022.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedi- cal image segmentation. In International Conference Medical image computing and computer- assisted intervention, pp. 234-241. Springer, 2015.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. In Advances in Neural Infor- mation Processing Systems (NeurIPS), volume 35, pp. 36479-36494, 2022.
Tim Salimans, Thomas Mensink, Jonathan Heek, and Emiel Hoogeboom. Multistep distillation of diffusion models via moment matching. In Advances in Neural Information Processing Systems (NeurIPS), volume 37, pp. 36046-36070, 2024.
Pratheba Selvaraju, Tianyu Ding, Tianyi Chen, Ilya Zharkov, and Luming Liang. Fora: Fost-forward caching in diffusion transformer acceleration. arXiv preprint arXhv:2407.01425, 2024.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. In Advances in Neural Information Processing Systems (NeurIPS), volume 32, 2019.
Kaiyue Sun, Kaiyi Huang, Xian Liu, Yue Wu, Zihan Xu, Zhenguo Li, and Xihui Liu. T2v- compbench: A comprehensive benchmark for compositional text-to-video generation. arXiv preprint arXiv:2407.14500 2024.
Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojic Mao, Chen-Wer Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, Jianyuan Zeng, Jiayu Wang, Jingfeng Zhang, Jingren Zhou, Jinkai Wang, Jixuan Chen, Kai Zhu, Kang Zhao, Keyu Yan, Lianghua Huang, Mengyang Feng. Ningyi Zhang, Pandeng Li, Pingyu Wu, Ruihang Chu, Ruili Feng. Shiwei Zhang, Siyang Sun, Tao Fang, Tianxing Wang. Tianyi Gui, Tingyu Weng, Tong Shen, Wei Lin, Wei Wang, Wei Wang, Wenmeng Zhou, Wente Wang. Wenting Shen, Wenyuan Yu, Xianzhong Shi, Xiaoming Huang, Xin Xu, Yan Kou, Yangyu Lv, Yifei Li, Yijing Liu, Yuming Wang. Yingya Zhang, Yitong Huang, Yong Li, You Wu, Yu Liu, Yulin Pan, Yun Zheng, Yuntao Hong, Yupeng Shi, Yutong Feng, Zeyinzi Jiang, Zhen Han, Zhi-Fan Wu, and Ziyu Liu. Wan: Open and advanced large-scale scale video generative models. arXiv preprint ::v:::200: 2503.20333........ 2025.
Changyuan Wang, Ziwei Wang, Xiuwei Xu, Yansong Tang. Jie Zhou, and Jiwen Lu. Towards accu- rate post- post-training quantization for diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 16026-16035 2024.
13

---

Published as a conference paper at ICLR 2026
Yaohui Wang, Xinyuan Chen, Xin Ma, Shangchen Zhou, Ziqi Huang, Yi Wang, Ceyuan Yang. Yinan He, Jiashuo Yu, Peiqing Yang, et al. Lavie: High-quality video generation with cascaded latent diffusion models. International Journal of Computer Vision, 133:3059-3078 2025.
Zhou Wang and Alan C Bovik. A universal image quality index. IEEE signal processing letters, (3)811884, 2002.
Yujie Wei, Shiwei Zhang, Zhiwu Qing, Hangjie Yuan, Zhiheng Liu, Yu Liu, Yingya Zhang, Jingren Zhou, and Hongming Shan. Dreamvideo: Composing your dream videos with customized sub- ject and motion. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6537-6549, 2024.
Felix Wimbauer. Bichen Wu, Edgar Schoenfeld, Xiaoliang Dai, Hou, Zijian He, Artsiom Sanakoyeu, Peizhao Zhang, Sam Tsai. Jonas Kohler, et al. Cache me if you can: Accelerat- ing diffusion models through block caching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6211-62201 2024.
Jingjing Xu, Xu Sun, Zhiyuan Zhang, Guangxiang Zhao, and Junyang Lin. Understanding and im- proving layer normalization. Advances in Neural Information Processing Systems (NeurIPS), volume 32, 2019.
Tianyun Yang, Juan Cao, and Chang Xu. Pruning for robust concept erasing in diffusion models. arXiv preprint arXiv:2405. 2024.
Zhihang Yuan, Hanling Zhang, Lu Pu, Xuefei Ning. Linfeng Zhang, Tianchen Zhao, Shengen Yan, Guohao Dai, and Yu Wang. Ditfastattn: Attention compression for diffusion transformer models. In Advances in Neural Information Processing Systems (NeurIPS), volume 37, pp. 1196-1219, 2024.
Dingkum Zhang, Sijia Li, Chen Chen, Qingsong Xie, and Haonan Lu. Laptop-diff: Layer pruning and normalized distillation for compressing diffusion models. arXiv preprint arXiv:2404. 1098, 2024a.
Hanling Zhang, Rundong Su, Zhihang Yuan, Pengtao Chen, Mingzhu Shen Yibo Fan, Shengen Yan, Guohao Dai, and Yu Wang. Ditfastattnv2:: Head-wise attention compression for multi-modality diffusion transformers. arXiv preprint arXiv:2503.22796 2025.
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
Wentian Zhang, Haozhe Liu, Jinheng Xie, Francesco Faccio, Mike Zheng Shou, and Jürgen Schmid- huber. Cross-attention makes inference cumbersome in text-to-image diffusion models. arXiv preprint arX1v:2404.02777V 02747v1, 2024b.
Xuanlei Zhao, Shenggan Cheng, Chang Chen, Zangwei Zheng, Ziming Liu, Zheming Yang, and Yang You. Dsp: Dynamic sequence parallelism for multi-dimensional transformers. arXiv preprint arXiv:2403. 10266, 2024a.
Xuanlei Zhao, Xiaolong Jin, Kai Wang. and Yang You. Real-time video generation with pyramid attention broadeast. arXiv preprint arXiv:2408.122 12588, 2024b.
Zangwei Zheng, Xiangyu Peng, Tianji Yang, Chenhui Shen, Shenggui Li. Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. Open-sora: Democratizing efficient video production for all. arXiv preprint 4412
2024.
Zhenyu Zhou, Defang Chen, Can Wang, Chun Chen, and Siwei Lyu. Simple and fast distillation of diffusion models. In Advances in Neural Information Processing Systems (NeurIPS), volume 37, pp. 4-8600 2024.
14

---

Published as a conference paper at ICLR 2026
A BLOCK-LEVEL FEATURE REUSE
Figure 10 presents a comparative analysis of feature variations during the video generation pro- cess for the two distinct prompts introduced in Figure a Figure T illustrates the block-level feature differences, while Figure 10b depicts the timestep-level feature differences. The results re- veal that block-level features exhibit significant and discernible variations between the two prompts. Specifically, the video generated for the more static scene (e.g.. Prompt 1) demonstrates lower fea- ture variability,, enabling more granular and aggressive caching and reuse of block features. This facilitates substantial acceleration while maintaining visual fidelity. In contrast, the timestep-level features show minimal differentiation across prompts, making difficult to perform prompt-specific adaptive caching control. This fundamental limitation explains why timestep-level caching meth- ods, such as TeaCache, often result in inferior video quality compared to our proposed block-wise method.
Aggregated Relative L1
Prompt 1
Prompt
0.25
Prompt 2
Prompt
00.20
0.15
0.10
2
0.05
0
0.00
10
15
20
25
30
10
15
20
25
30
Step Index
Step Index
(a) Block-level caching performance (b) Timestep-level caching performance
Figure 10: Comparative analysis of feature variations: Block-level vs. Timestep-level.
B EXPERIMENTAL SETTINGS
Base Model. This paper focuses DiT-based video generation, evaluating five state-of offte-of--hheart spen-source models: Open-Sora Zheng et al. (202)), Open-Sora-Plan Lin et al. (2024a). Latte Ma et al. (2025a), Wan 2.1 Wan al. (2025), ) and dunyuanVideo Kong al. (2024). Open-Sora ploys Spatial-Temporal DiT Ho et al. (2022), allowing for the creation of videos up to 16 seconds long at 720p resolution. Open-Sora Plan focuses generating high-resolution videos with extended durations based on diverse user inputs, featuring components like a Wavelet-Flow Variational Au- toencoder and specialized denoisers. Latte effectively extracts spatio-temporal tokens from videos and utilizes DiT blocks to model the token distribution in a latent space. Wan is comprehensive and open-source suite of high-pes nigh-performance video foundation models. Hunyuan Video is an open-source video foundation model, integrating advanced data curation and a scalable transformer architecture. The inference configs of five models are shown in Table 6, which strictly follow the official settings.
Model
Scheduler
Steps
Text Encoder
Open-Sora
RFLOW
30
TS
Open- Sorn--Sra-PPaa Plan
PNDM
150
T5
Latte
DDIM
50
T5
Wan 2.1
FlowMatch
50
UMTS
Hunyuan Video
FlowMatch
50
MLLM
Table 6: The inference config of base models.
C METRICS
Peak Signal-to-Noise Ratio (PSNR). PSNR is common metric used to measure the quality of re- construction in lossy compression and signal processing. It compares the maximum possible power of signal to the power of corrupting noise. Higher PSNR values indicate better quality. The PSNR
15

---

Published as a conference paper at ICLR 2026
is calculated follows:
R
PSNR 10 1og10
(9)
MSF
where R is the maximum possible pixel value of the image and MSE denotes the Mean Squared Error between the reference image and the reconstructed image.
Learned Perceptual Image Patch Similarity (LPIPS). LPIPS Zhang et al. (2018) is modern perceptual similarity metric that is based on deep learning. It quantifies the similarity between two images by comparing their features at different layers of neural network. This metric is believed to better correlate with human visual perception than traditional metrics by capturing perceptually important features. The LPIPS score between two images I and K is typically computed as:
LPIPS K) T. E (()(1) - (k)
(10)
where L is the number of layers in the pretrained neural network, and Φv denotes the feature extrac- tion at layer l. For video evaluation, LPIPS is calculated for each frame of the video and averaged across all frames to produce a final score.
Structural Similarity Index Measure (SSIM). SSIM Wang & Bovik (2002) is perceptual metric that assesses the visual impact of three characteristics: luminance, contrast, and structure. Instead of measuring pixel-wise differences, SSIM compares the structural information in the images, making it more relevant measure of perceived quality. The SSIM index is calculated follows:
(2piRK + (2) + C2) SSTM K) (13 + P2 + Ci)(o} + o + C2)
(11)
where μ and μæ are the average luminance of and K, of and o are the variances of and K, diK is the covariance between and K, C1 and C2 are constants used to stabilize the division with weak denominator. For video evaluation, SSIM calculated for each frame and then averaged over all frames to provide an overall similarity measure.
D ADDITIONAL EXPERIMENTAL RESULTS
D.1 COMPARISON WITH HIGH- MEMORY METHODS ON A SINGLE GPU.
We compare BWCache with high-memory caching methods, including ProfilingDiT Ma al. (2025)) and TaylorSeer Liu et al. (2025b). These methods require caching extensive intermediate features, leading to significant memory overhead. When generating long videos, both ProfilingDiT and TaylorSeer encounter out-of -e--ooy-mmrryy (OOM) errors. To ensure fair comparison, we evaluate these methods on shorter video sequences where they can run successfully. Specifically, we test on Wan 2.1 with 9 frames at 480P and Hunyuan Video with 17 frames at 544P, which are the maximum video lengths that TaylorSeer can process on a single NVIDIA A800 80GB GPU, as shown in Table 7 We configure ProfilingDiT with T, 6, and TaylorSeer with O 1 and A = 3 for Wan 2.1, and A = for Hunyuan Video. Despite operating under more favorable conditions for the baseline methods, BWCache still achieves superior visual quality metrics while maintain- ing competitive or better acceleration performance. This demonstrates that BWCache provides a better balance between memory efficiency and generation quality compared to methods that require extensive feature caching.
and
D.2 FULL VBENCH RESULTS.
VBench Huang et al. 2024b) is a comprehensive benchmark suite designed for evaluating video generative models. It provides a standardized framework that includes diverse evaluation metrics, datasets, and protocols to facilitate in-depth comparisons of model performance across various tasks. This systematic approach ensures reliable assessments of generated videos in terms of quality, effi- ciency, and robustness. Table compares the performance of four video generation methods across each of the 16 VBench dimensions. A higher score indicates relatively better performance for particular dimension. We also provide two specially built baselines. i.c., Empirical Min and Max (the approximated achievable min and max scores for each dimension), references.
16

---

Published as a conference paper at ICLR 2026
Visual Quality
Efficiency
Model
Method
VBench"
LPIPS)
SSIM
PSNR
Speedupt
Latency(s).
Original
77.05%
287
Wan 2.1
ProfilingDiT
76.53%
0.2053
0.8221
24.20
1.24
231
(9 frames, 480P)
TaylorSeer
75.13%
0,3080
0.7231
18.75
1.87
153
BWCache
76.81
0.0929
0.9048
1.67x
171
Original
82.08%
1x
179
Hunyuan Video (17 frames, 544P)
ProfilingDiT
80.33%
0.2610
0.6662
18.77
1.72x
104
TaylorSeer
82.05%
0.4021
0.5526
15.33
1.94x
92
BWCache
81.42%
0.1804
0.7507
2.39
Table 7: Comparison with high-memory caching methods on reduced video lengths.
Subject
Background Consistency
Temporal Flickering
Motion Smoothness
Dynamic Degree
Aesthetic Quality
Imaging Quality
Object
Models
Consistency
Class
Original
95.08%
96.86%
99..11
98.80%
59.53%
95.87%
PAB
95.15%
96.70%
99.38%
99.09%
28.00%
56.32%
59.63%
96.37%
TeaCache
96.83%
58.50%
96.00%
BWCache
95. 91%
96.76%
99.43%
98.
38.00%
59.26%
61.29%
98 25%
Empirical Min
14.62%
26.15%
62.93%
70.60%
0.00%
0.00%
0.00%
0.00%
Empitiaal Max
100.00%
100.00%
100.00%
99.75%
100.00%
100.00%
100.00%
100.00%
Multiple
Spatial Relationship
Appearance
Temporal Style
Overall
Models
Objects
Action
Color
Seene
Style
Consistency
Original
36.00%
84.00%
81.25%
85.75%
59.62%
25,40%
23.03%
27.00%
PAB
38.50%
68.00%
85.64%
77.22%
57.37%
25,06%
22.20%
26.70%
TeaCache
31.50%
82.00%
81.61%
85.75%
56.
24.93%
22.73%
27.24%
BWCache
36.25%
84.00%
82.58%
86.52%
59.25%
25.27%
22.74%
27.91%
Empirical Min
0.00%
0.00%
0.00%
0.00%%
0.00%
0.00%
0.00%
0.00%
Empirical
100.00%
100.00%
100.00%
100.00%
82.22%
28.55%
36.40%
36.40%
Table 8: VBench evaluation results per dimension.
D.3 LAST STEP
In the final stage the DiT model for video generation, that is, during the last stage of denoising, the reuse of cached block features can introduce unnecessary biases and lead to a degradation in the generated video. Therefore, assuming that cache reuse is triggered at the k-th step during the denoising process, we default that the last 1/2k: steps not involve cache and reuse operations. Meanwhile, we have statistically analyzed the results for the last 1/3k and 2/3k steps, well as fixed 5 and steps, as shown Table It can seen that the choice the last steps has significant impact on latency and video generation quality. Specifically, as the number of last steps increases, latency tends to decrease, while metrics exhibit fluctuations to some extent. In general, default setting 1/2k balances the generation quality and latency.
Last Step
Latency(s))
Vbench
LPIPS
SSIM
PSNR
1/3k
29.10
79.95%
0.0922
0.8823
26.95
1/2k
27.68
80.03%
0.0879
0.8854
27.05
2/3k
25.98
80.09%
0.0853
0.8874
27.16
5
31.47
80.13%
0.0877
0,8866
27.10
8
28.62
80.00%
0.0862
0.8896
27.01
Table 9: Impact of different last steps.
D.4 STATISTICAL ANALYSIS
Table 10 shows the statistical results of the paired t-test that determine the significance level of the BWCache compared with other methods regarding LPIPS, SSIM, and PSNR. Specifically, the table provides the mean and standard deviation of PAB Zhao et al. (2024b) and TeaCache Liu et al. 2024),, alongside the valuu and corresponding confidence interval. Our null hypothesis is: "There significant difference in the performance between the BWCache and other methods". The table shows a meaningful difference between the B BWCache and other methods, while the p-value is lower than 0.05 in all cases. There is statistically significant evidence the 5% significance level
17

---

Published as a conference paper at ICLR 2026
Model
Method
Metries
Mean
Standard Deviation
P-Value
Confidence Interval
LPIPS
0.2134
0.1026
9.90c-111
[0.2048, 0.2220]
PAB
SSIM 0.7798
0.1084
8.99c-73
[0.7707. 0.7888]
PSNR
22.0191
2,8325
2.8c-135
[21.7819, 22.2564]
LPIIS
0.1496
0,07800
1.07e446
[0.1430, 0.1561]
Open-Sora
TeaCache
SSIM 0.8104
0.1034
1.09-422
[0.8017, 0.8190]
PSNR
22.3929
4,3654
2.8c-80
[22.0273, 22.7586]
LPIPS 0.0879
0.0561
[0.0832, 0.0926]
BWCache
SSIM 0.8854
0.0665
10.8798, 0.8909]
PSNR 27.0510
2,9898
[26.8006, 27.3014]
LPIPS 0.2781
0.1177
7.69c-51
[0.2602, 0.2961
PAB
SSIM 0.6627
0,1449
7.16c-37
[0.6407, 0.6848]
PSNR 19.4968
3.6949
6.76e-42
[18,9340, 20.0596]
Open-Sora-
LPIPS 0.1965
0.0928
6.64e-27
[0.1824, 0.2
TeaCache
SSIM 0.7502
0,1090
4.95e-18
[0.7536, 0.7668)
Plan
PSNR 21.5010
3,5785
6.25e-24
[20.9559, 22.0461]
LPIPS 0.1001
[0.0922, [080]
BWCache
SSIM 0.8435
0.0743
[0.8322, 0.8548)
PSNR 25.8734
3.7611
[25,3006, 26.4463)
LPIPS 0.4625
0.1481
3.84c-218
[0.4501. 0.4750]
PAB
SSIM 0.5964
0.1631
3.88e-110 1016
[0.5827, 0.6100]
PSNR 17.0299
2.2196
2.67e-189 189
[16.8400,
17.2158)
LPIPS 0.1969
0.1058
5.32c-17
[8.1880, 0.2058]
Latte
TeaCache
SSIM 0.7606
0.1329
2.39c-12
[0.7495, 0.77 18)
PSNR 22.1947
4,3783
1.02c400
[21.8280, 22.5614]
[0.1301, 1496]
BWCache
SSIM 0.8181
0.1357
[0.8067, 0.8294]
PSNR 26,4620
5.6981
[25.9847, 26,9392]
LPIPS
2.10c-88
[0.1440, 1806]
PAB
SSIM 0.7643
0.1465
4.92e-09
[0.7405, 0.7881]
PSNR 21.3210
3,5476
3.07e-22
[20.7974. 21.446]
LPIPS 0.2382
1.96:30
[0.2252, 0.2531]
Wan 2.1
TeaCache
SSIM 0.6608
0.1406
1.476331
[0.6373, 0.6843]
PSNR 18.6197
3.2942
1.11e-42
1690669, 19.1702
[0.0728, 0.0882]
BWCache
SSIM 0.8527
0.0968
[0.8365, 0.8689]
PSNR 25.7246
3,9350
[25.0670, 26.3821]
3.20e-06
[0.0857, 1.333
PAB
SSIM 0.8341
0.1642
1.47e-03
[0.8070, 0.8613]
PSNR 26.6958
3.6582
2.11e-11
[26. 1089, 27.2827]
LPIPS
0.0823
3.68c-25
[0.1547, 0.1822]
HunyuanVidee Video TeaCache
SSIM 0.7973
0.1184
2.80c-13
[0.7775, 0.8171]
PSNR 23.90004
3,9613
1.14e-25
[23.2384, 24.5623]
LPIPS 0.0768
[0.0689, 0.0846]
BWCache
SSIM 0.8922
0.0860
[0,8778, 0.9065]
PSNR
29.6333
4.2949
(28.9157, 30.3510]
Table 10: Statistical comparison of the BWCache with compared methods.
to suggest that the BWCache performs better than PAB and TeaCache. Thus, the null hypothesis is rejected, which proves that the differences are significant.
D.5 MEMORY USAGE ANALYSIS
compare the memory consumption the original models, our proposed BWCache method, and all main baseline methods (PAB, TeaCache, FasterCache, ProfilingDiT, and TaylorSeer) across five models under various video generation configurations. The results indicate that all training-free caching methods require additional GPU memory compared to the original models due to caching intermediate features from previous timesteps. This memory overhead is inherent to all caching-based acceleration approaches. For models already at absolute memory lim- its, all training-free caching methods face similar constraints. Notably, Wan 2.1 and Hunyuan Video employ full 3D self-attention Peebles & Xie (2023), which inherently demand more memory. For instance, TaylorSeer can only process Wan 2.1 with frames at 480P and Hunyuan Video with 17 frames at 544P, which the maximum video lengths that TaylorSeer process sin- gle NVIDIA A800 80GB GPU. Compared to other cashe-based methods, BWCache demonstrates competitive superior memory efficiency.
As shown in Table II
Moreover, BWCache is not an all-or-nothing design; it can be configured to reduce memory over- head for memory-constrained settings. The method can be configured to: (1) only cache subset of
18

---

Published as a conference paper at ICLR 2026
blocks instead of all blocks, (2) apply pooling compression to cached features, or (3) when only caching the first DiT block each timestep, BWCache becomes similar to TeaCache. To maintain computational efficiency, BCCache selectively uses subset of blocks for cache indicator calcula- tion in Wan 2.1 and Hunyuan Video models. of blocks used for cache indicator computation video generation quality. We compute the similar- ity indicator using uniformly spaced subset of blocks across depth. This design aims to cover both early, middle, and late blocks while reducing the amount of cached state and indicator computation. Generally, higher proportion of blocks leads to more accurate cache indicator calculation. To bal- ance latency and efficiency, we default using 50% of the blocks for cache indicator calculation in these models. Although BWCache introduces additional memory usage, it significantly acceler- ates inference and reduces overall GPU operational costs, making it highly effective solution for practical applications.
the effect of varying the proportion
Model
Open-Sora
Open-Sora-Plant
Latte
Parameter
51 frames, 480P
65 frames, 512 x 512 16 frames, 512 512
Original
15270MiB
15724MiB
15240MiB
PAB
27016MiB
26722MIB
29126MiB
TeaCache
20630MiB
21266MiB
18499MiB
FasterCache
26130MiB
22508MiB
23264MiB
BWCache
21362MiB
18022MiB
17400MiB
Model
Wan 2.1
Parameter
frames, 480P
81 frames, 480P
81 frames, 720P
Original
49757MiB
71861MiB
73955MiB
PAB
59757MiB
80285MiB
80154MiB
TeaCache
56737MiB
74197MiB
76563MiB
ProfilingDiT
57089MIB
78787MiB
OOM
TaylorSeer
79263MiB
OOM
OOM
BWCache
58047MiB
7954IMiB
80607MiB
Model
Hunyuan Video
Parameter
frames, 720P
129 frames, 540P
129 frames, 720P
Original
45124MiB
58491
74479MiB
PAB
IMiB
78787MiB
OOM
TeaCache
5245 MiB
64729MiB
78039MiB
ProfilingDiT
65124MiB
OOM
OOM
TaylorSeer
74287MiB
OOM
OOM
BWCache
49757MiB
66184MiB
80625MiB
Table 1I: Memory usage comparison of BWCache and baseline methods.
Wan 2.1 (81 frames, 480P)
Memory usage
LPIPS
SSIM
PSNR
10% blocks
72367MiB M56
0.0790
0.8524
25.79
30% blocks
0.0789
0,8533
25.83
50% blocks
79541MiB
0.0782
0.8539
25.86
Hunyuan Video (129 frames, 540P)
Memory usage
LPIPS
SSIM1
PSNR
10% blocks
60977MiB 63617MiB
0.0795
0.8921
29.95
30% blocks
0.0786
0.8884
29.86
50% blocks
661 66184MiB
0.0794
0.8903
29.91
Table 12: Impact of different block proportions cache indicator calculation.
D.6 COMPUTATIONAL LATENCY OF THE CACHE INDICATOR
we measure the absolute time spent computing the cache indicator and its relative proportion of the total inference time across five models. The percentages in parentheses indicate the fraction of the entire inference process consumed by the cache indicator computation. The results demonstrate that the computational cost of the cache indicator is relatively modest, rang- ing from 10.6% to 22.5% of the total inference time. Despite this overhead, BWCache still achieves
As shown in Table 13
19

---

Published as a conference paper at ICLR 2026
significant overall speedup because the cache indicator computation is substantially faster than computing all DiT blocks, and the cached features enable skipping a large portion of redundant computations.
Model
Open-Sora
Open-Sora-Plan
Latte
Wan 2.1
Hunyuan Video
Compufational latency (9) 6.22(22.5%)
5.05(11.4%)
3.12(22.0%)
48.23(10.6%)
68.90(13.9%)
Table 13: Computational latency of the cache indicator.
D.7 DYNAMIC VIDEO GENERATION
To thoroughly evaluate the performance BWCache in highly dynamic scenarios, we specifically analyze video generation tasks involving human motion, fast parallax, and other compositional scenes with high motion complexity. While our main experiments have already included various dynamic scenes, further extract and evaluate subset of videos that exhibit high dynamic char- acteristics. As shown in Table [4 and Figure 11 we evaluate BWCache's 's performance on highly dynamic video generation tasks across five models. VBench is comprehensive benchmarking suite that enables quantitative assessment of how well our method handles challenging scenarios with rapid motion and complex scene compositions. The results demonstrate that BWCache maintains competitive visual quality even in highly dynamic scenarios, with only minimal quality degradation compared the original models. This indicates that the block-wise adaptive caching mechanism effectively adapts to varying scene dynamics.
Open-Sora
Open-Sora-Plan
Latte
Olighal
Origiaal
BWCache
Wan2.1
HunyuanVideo
Original
BWC
Original
Figure 11: Dynamic video generation.
20

---

Published as a conference paper at ICLR 2026
Model
Open-Sora
Open-Sora-Plan
Latte
Wan 2.1
Hunyuan Video
Original
80.12%
80.48% 80.37%
78.23%
82.03%
82.14%
BWCache
79.71%
77.64%
81.90%
81.38%
Table 14: Dynamic video generation results.
Model
Open-Sora
Open-Sora-PHae
Latte
Wan 2.1
HunyuanVider Video
Threshold
0.15 0.25
0.15 0.25
0.1 0.2
0.1 0.2
0.15
VBench
79.3% 80.0%
80.2% 81.0%
75.5% 78.7%
80.4% 81.3%
81.9% 82.6%
Speedupt
1.41 1.96 x
2.02
2.07
1,75 2,48
2.94
Table 15: Recommended parameter settings.
DISCUSSION
E.I GUIDELINES FOR PARAMETERS SETTING
When applying BWCache to new DiT-based model, we recommend starting with a default value of 8 0.15, which prioritizes quality. To further optimize, generate few sample videos and plot the ARLI across timesteps to identify the optimal region for cache reuse, then iteratively test different threshold values to balance quality and speedup. Models with more stable feature dynamies may tolerate higher thresholds, while those handling highly dynamic content may require lower thresholds. During threshold selection, prioritize maintaining VBench scores within 1-2% of original model's performance, and reduce the threshold if quality degradation exceeds acceptable limits.
In practice, most users can directly the default value of 8 = 0.15 select from the validated ranges in Table 15 based on their model type, as these have been extensively tested across diverse models and generation tasks.
For the reuse interval R, we recommend setting to 10% of the total timesteps as the default value. Our extensive experiments across all five models demonstrate that interval of 10% consistently achieves an optimal balance between inference efficiency and visual quality.
E.2 FEATURE DIFFERENCE PATTERN IN OTHER SETTINGS
We also investigate whether similar patterns exist in other diffusion model settings. Specifically, as shown in Figure 12 and 13 class-conditional generation and image generation exhibit inverted L-shaped pattern, which differs from the U-shaped pattern observed in video generation models. The feature variation pattern is closely related to the noise schedule and loss function used during training Zhang et al. (2024b). In text-to-image generation, the model rapidly establishes the overall semantic structure and content layout, leading to high feature variation. Once the semantic structure established, the remaining timesteps focus primarily on refining details and improving visual quality, resulting in stable and gradually decreasing feature variations.
E.3 LATENT DRIFT FROM FEATURE REUSE
While the similarity indicator effectively identifies when features are stable enough for cache reuse, we observe that once cache reuse begins, the subsequent feature changes become unpredictable. As illustrated in Figure [4] after multiple cycles of cache reuse, the feature variations exhibit irregular patterns that deviate from the expected U-shaped trajectory. this case, each reuse step propagates small discrepancies, which accumulate over time and cause the latent representation to drift away from the intended denoising path. This latent drift introduces additional error that cannot be easily predicted corrected by dynamically monitoring the L1 distance alone.
21

---

Published as a conference paper at ICLR 2026
Aggregated Relative L1
Aggregated Relative LI
1.28
13.6
0.23
Aggregated Relative
0.17
0.88
8.23
0.69
5.53
0.07
2.82
0.02
0.29
80 100
0.12
Diffusion Process (%)
Diffusion Process (%)
Diffusion Process (%)
Figure 13: Feature changes for class-conditional genera- tion.
Figure 12: Feature changes for text-to-image generation.
Figure 14: Feature changes during cache reuse.
F MORE VISUAL RESULTS
The additional visual comparison results for Open-Sora, Open-Sora-Plan, Latte, Wan 2.1, and Hun- quanVideo are presented in Figure [5] Figure [6| Figure 17 Figure [8] and Figure [9] All prompts sourced from the Open-Sora gallery Lin et al. (2024a). VBench benchmark [Huang et al. (2024b), and T2V-CompBench Sun et al. (2024). which collectively offer diverse and representative scenarios for comprehensive evaluation. Our method demonstrates consistent fidelity across diverse models, styles, and content types in video generation while maintaining computational efficiency.
STATEMENT ON LARGE LANGUAGE MODEL (LLM) USAGE
In the preparation this manuscript, large language models (LLMs) were utilized solely for the purpose of language polishing. The LLM played no role in the generation of core ideas. All final content was thoroughly reviewed and approved by the authors.
22

---

Published as a conference paper at ICLR 2026
lapse beach moving sky
Arch
Original
PAB
TeaCahce
BWCache
A person is drawing
byclle gliding hurough snowy field
Original
PAB
ab
o
TeaCahce
64
Gkg
Gll
BWCache
ab
obb
An astrenaut flying in space
tranquil tableas of bedream
Original
PAB
BWCache TexCahce
bost
A panda standing on a surt board the ocean sunset
Original
PAB
TeaCahoe
BWCache
Figure 15: More visual results on Open-Sora (51 frames, 480P).
23

---

Published as a conference paper at ICLR 2026
blue car drives past white picket fence surny
black dog sits beside red vase rable
A harse standing on rable
Original
PAB
TeaCahce
BWCCache
panda bambooo while monkey
fat rabbit purale through any landscace
A frozen lake thawing the
rises
Original
TeaCahce
BWCache
Tuturistic cyber sunk cityscape at right with tower ing reon-lit Skyscr apers
chostly ship seiling through the clouds, navigating through sea under mooni t sky
A sunset with beautiful beach
Original
PAB
TeaCahce
BWCache
Figure 16: More visual results on Open-Sora- (65 frames, 512x512) x12).
24

---

Published as a conference paper at ICLR 2026
cat slinking the left side of coay living room
A teddy bear drops downward onte the bed,
while fluf pillow rises upwards to the air
A glass apple tumbles from wicker basket
Original
PAB
TeaCahce
BWCache
Red toaster toasting beside white rigerator
Pink surf with droplets, close-up
and explesions
Original
TeaCahce
BWCache
Deers the woods gaze the camera under the sunlight
woolen hat files off wooden en bench, carried away by a sudden gust of wind
Sunrise on top of hi high-rise buildling
Original
PAB
Figure 17: More visual results on Latte (16 frames, 512x512).
25

---

Published as a conference paper at ICLR 2026
tranquil tableau of peacef orchid garden showcased variety delicate blooms
toothbrush
Original
PAB
BWCache TeaCahoe
a giraffe taking a peacef ul walk
white vase
Original
PAB
BWCache TeaCahce
valley
person is sailing
Original
PAB
BWCache TeaCahce
Vincent Gogh is painting in the person making snowman
Original
PAB
BWCache TeaCahce
Figure 18: More visual results on Wan 2.1 (81 frames, 480P).
26

---

Published as a conference paper at ICLR 2026
black vase
bus on the right of traf fic light, front view
Original
PAB
BWCache TeaCalice
giraffe running jin herd its kind
underwater reef
Original
PAB
BWCache TeaCahoe
Vincent van Gogh ininting in the room
Volcano
Original
PAB
BWCache TeaCahce
W ind mill
Yellow flwers swing in the wind
Original
PAB
BWCache TeaCahoe
Figure 19: More visual results on Hunyuan Video (129 frames, 544P).
27


Reviews:
[{"review_id": "M89iKTnKmm", "reviewer": "Reviewer_CKpW", "summary": "The paper proposes BWCache, a training-free acceleration technique for Diffusion Transformer video generation. It starts with the observation that DiT blocks dominate latency and many intermediate timesteps present redundant computation. BWCache caches block outputs across timesteps and reuses them based on a similarity indicator. It also incorporates periodic recomputation to prevent latent drift. Experiments across multiple backbone models show up to 2.6× speedup while maintaining competitive quality.", "questions": "The paper's validation is limited exclusively to text-to-video generation. Have you explore the core \"U-shaped\" ARL1 pattern holds in other settings, like class-conditional generation, image generation?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- High Practicality and Scalability. As a training-free method, it can be integrated into existing DiT pipelines without costly fine-tuning. \n- The BWCache design is well-motivated by a clear empirical analysis of DiT block behavior. The Aggregated Relative L1 analysis in Figure 4 provides a strong justification for the caching strategy.\n- The method achieves a superior balance between inference speed and visual quality. It delivers speedups while maintaining comparable or even superior visual fidelity.", "weaknesses": "- The paper's claim to be the first method to exploit block-wise temporal stability in DiTs appears to be overstated. There is some highly relevant prior work which also investigates temporal redundancy in DiTs but not discussed in the paper. The authors must situate their work relative to it, differentiate their contribution, and (if possible) provide a comparative analysis.\n1. Towards Stabilized and Efficient Diffusion Transformers through Long-Skip-Connections with Spectral Constraints\n\n- In Table 1, TeaCache achieves a 4.41x speedup on Open-Sora-Plan. The authors attribute it to \"significant variation in block features during the early inference stages,\" but provide no data or analysis to support this claim. Why does the block-wise indicator in Eq. 7 fail in this specific scenario while TeaCache's simpler timestep-level caching succeeds? \n-  The entire caching mechanism depends on a manually-set similarity threshold in Eq. 7, which appears empirical. The paper provides no theoretical grounding for this choice, nor does it analyze its stability or the risk of error accumulation from repeated cache reuse. The ablation in Table 4 shows a predictable quality/speed trade-off but fails to explain how a user should set $\\delta$ for a new model or task.\n- The computational cost of the cache indicator is not considered. BWCache requires calculating the L1 distance for all blocks at every step to decide whether to cache. This overhead is never measured. While this cost may be negligible on large models, it could substantially diminish on smaller models, in edge-computing scenarios, or on hardware with lower memory bandwidth.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pbTkJE4CFL", "reviewer": "Reviewer_41nC", "summary": "This paper presents BWCache, a training-free inference acceleration method for DiT-based video generation. The work is motivated by an empirical analysis showing that DiT block computations are the main latency bottleneck and exhibit a U-shaped variation pattern, implying high redundancy in intermediate steps. BWCache exploits this by dynamically caching and reusing features at the block level. A similarity indicator (average relative L1 distance) determines when to reuse features, while safeguards like periodic recomputation and forced computation in the final steps ensure high visual fidelity. The method is validated across five state-of-the-art models, demonstrating significant speedups while consistently outperforming other acceleration baselines in preserving video quality.", "questions": "What is the underlying theoretical explanation for the observed U-shaped pattern in DiT block feature variations? Is this pattern consistent across different architectures and training methodologies, and how does it relate to the diffusion process dynamics? I would appreciate it if the authors could discuss this with previous related works (e.g., [1])\n\n[1] Ma, Xuran, et al. \"Model Reveals What to Cache: Profiling-Based Feature Reuse for Video Diffusion Models.\" Proceedings of the IEEE/CVF International Conference on Computer Vision (2025).", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is built on a clear, insightful, and well-visualized analysis (Figure 4). The identification of the U-shaped variation pattern in DiT blocks is a strong finding that provides a direct and logical justification for the entire method's design.\n2. The method is more sophisticated than several competitors. It correctly identifies a suitable *granularity* (DiT blocks) and, more importantly, implements a dynamic *trigger* (the similarity indicator) for caching. This, combined with the fidelity-preserving strategies (final-step recomputation), allows it to effectively balance acceleration and quality.\n3. The experimental validation is thorough. Testing on five different, large-scale video DiTs and comparing against multiple relevant baselines demonstrates the method's general applicability. The scalability tests (multi-GPU, video length) and the direct ablation against naive step reduction (Table 3) make the performance claims very convincing.", "weaknesses": "1. Notably, the trade-off of speed for memory is a weakness. By caching DiT block features, the method's GPU memory footprint increases, as acknowledged in the appendix (Appendix D.4). For very large models that already push the limits of high-end GPUs (e.g., Wan 2.1 using ~74GB), this additional overhead (to ~81GB) could make the method unusable on memory-constrained hardware.\n2. The method's effectiveness depends on manually tuning two key hyperparameters: the similarity threshold $\\delta$ and the recomputation interval $R$. The ablation studies show that these values control a sensitive trade-off between speed and quality. While the authors provide default values, these may not be optimal for all models or use cases, and the paper does not offer a strategy for setting them automatically.\n3. A critical implementation detail is hidden in the appendix (D.4). For memory-heavy models (Wan 2.1, HunyuanVideo), the similarity indicator is not calculated using *all* blocks; instead, a *subset* (e.g., 50%) is used to save memory. The main paper does not mention this, and the appendix does not specify *how* this subset is chosen (e.g., first 50%, random, etc.), making the indicator's true behavior and robustness on these models unclear.\n4. The fidelity safeguards are static: always recompute the *last $k/2$ steps* and reuse for a *fixed interval $R$*. The \"U-shape\" analysis suggests that the transition from stable to variable features (the right side of the \"U\") is gradual. A more dynamic approach, perhaps by monitoring the L1 distance to decide when to *stop* caching (rather than a fixed $k/2$ rule), might offer an even better trade-off.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "u7oAKN8VZx", "reviewer": "Reviewer_RYjh", "summary": "This paper proposes BWCache, a training-free acceleration framework for Diffusion Transformer (DiT)–based video diffusion models. Instead of modifying the model architecture, BWCache reuses intermediate block-level features across diffusion timesteps when the change between adjacent blocks is below a similarity threshold δ. A periodic recomputation strategy mitigates latent drift.\n\nComprehensive experiments across five representative video diffusion models show that BWCache achieves up to 2.6× speed-up while maintaining comparable or slightly better perceptual quality compared to TeaCache and PAB. Statistical analyses and ablations (threshold $\\delta$, reuse interval, GPU scaling) reinforce the empirical soundness.\n\nWhile the acceleration results are impressive, the approach incurs non-negligible memory overhead due to feature caching, and its stability across architectures and motion intensities remains insufficiently explored. Nevertheless, the work offers a practical plug-and-play contribution that meaningfully advances inference-time efficiency for large-scale DiT-based video generation.", "questions": "1. Are spatial and temporal DiT blocks cached jointly or independently, and did you observe temporal drift in high-motion clips\n2. Can you provide results on highly dynamic or compositional scenes (human motion, fast parallax) to test stability", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "BWCache demonstrates outstanding performance in **redundancy granularity**, **cross-scenario adaptability**, and **quality–efficiency balance**, providing a precise and general acceleration framework tailored for redundant computation in DiT-based video generation models.\n\n1. **Focusing on Module-Level Redundancy, Breaking the Granularity Limitation of Traditional Caching**  \n   Existing caching methods (e.g., TeaCache, PAB) typically operate at the *timestep level* or *attention-head level*: the former risks losing critical timestep information, while the latter incurs high computational overhead and limited acceleration:\n   - **Key discovery of module-level redundancy:** By analyzing DiT model computation time distribution, the authors find that DiT modules account for over 70% of total inference latency, and that module features exhibit a *U-shaped variation pattern* across diffusion timesteps — large differences in early (high-noise) and late (low-noise) stages (requiring computation), but small differences in mid-stages (moderate noise, high redundancy, suitable for caching).  \n   - **Relative L1 distance as a quantitative criterion:** They define the *relative L1 distance* between module features and aggregate it into  \n     $ \\text{ARL1}(t) $ — the sum of all module-wise relative L1 values — as the redundancy measure. When  \n     $ \\text{ARL1}(t)/N < \\delta $ (δ = threshold), cached features are reused, ensuring that computation is skipped *only* when true redundancy exists, preventing blind reuse and quality degradation.  \n   This design focuses on the *most time-consuming components* (modules), guaranteeing both substantial acceleration and controlled quality degradation — the key to BWCache’s superior performance.\n\n2. **Consistent Cross-Model Acceleration with Negligible Quality Loss**  \n   Across five mainstream video generation models (Open-Sora, Open-Sora-Plan, Latte, Wan 2.1, and HunyuanVideo), BWCache achieves a strong balance between *speed* and *quality*, without requiring retraining:  \n   - **Single-GPU scenarios:**  \n     - On *HunyuanVideo* (129 frames, 544 P), BWCache achieves 2.6× speed-up (latency 1122 s → 433 s), with VBench 82.48% (higher than baseline 82.29%), LPIPS 0.0794 (better than PAB 0.1045 and TeaCache 0.1630), and PSNR 29.91.  \n     - On *Open-Sora*, at 1.61× acceleration, LPIPS 0.0879 (close to the original), SSIM 0.8854, and PSNR 27.05 — all significantly outperforming baselines like Δ-DiT and T-GATE.  \n   - **Multi-GPU scaling:**  \n     With *Dynamic Sequence Parallelism (DSP)*, BWCache reaches 17.2× acceleration on 8 A800 GPUs for Open-Sora (204 frames, 480 P) (latency 11.1 s vs 190.2 s original), with near-linear scaling, enabling large-scale deployment.  \n   - **High-resolution / long-video adaptability:**  \n     On *Wan 2.1* (720 P), BWCache delivers 2× speed-up (latency 912 s → 457 s) with VBench 81.99% (only 0.18% lower than original), while TeaCache at 1.41× acceleration shows more noticeable quality loss.", "weaknesses": "Despite its impressive performance, BWCache still has room for improvement in **memory overhead** and **adaptability to dynamic scenes**.\n\n1. **Module Caching Increases Memory Consumption — Limited for Single-GPU High-Resolution Scenarios**  \n   BWCache caches the *output features of each DiT module*, leading to noticeable GPU memory growth, especially for large models and high-resolution videos:  \n   - Appendix D.4 reports that on Wan 2.1 (81 frames, 720 P), BWCache uses 80 607 MiB vs 73 955 MiB for the baseline (+9%), and on HunyuanVideo (129 frames, 720 P) +8%.  \n   - To reduce memory pressure, the authors compute caching metrics on “50% module sampling”, which lowers memory cost but slightly reduces redundancy detection accuracy (e.g., 10% sampling yields LPIPS 0.7901 vs 0.0782 for 50% sampling).  \n   This limits BWCache’s applicability for single-GPU, memory-constrained environments (e.g., edge devices or high-resolution video generation).\n\n2. **Insufficient Redundancy Detection Accuracy in Highly Dynamic Scenes**  \n   The “U-shaped feature variation” assumption holds well for static scenes (e.g., “beach sunset”) but weakens in highly dynamic cases (e.g., “car chase,” “crowded street”):  \n   - In dynamic scenes (Prompt 2: “bus approaching traffic light”), module-level differences remain larger, with mid-stage redundant steps 30–40% fewer than in static scenes, leading to lower acceleration (e.g., on Open-Sora-Plan, TeaCache achieves 4.41× while BWCache only 2.24×, Table 1).  \n   - No dynamic-aware adaptation is proposed: the method lacks an *adaptive threshold $\\delta$* based on scene dynamics (e.g., using smaller δ for fast-changing content), resulting in suboptimal speed-quality trade-offs in dynamic video generation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Nzg5tKkkpF", "reviewer": "Reviewer_wT4r", "summary": "The paper proposes BWCache, a training-free acceleration method for DiT-based video diffusion. It caches and reuses block-level features across timesteps and triggers reuse with a similarity indicator computed from the mean relative L1 distance between adjacent-timestep block features. To avoid latent drift, it adds periodic recomputation with a reuse interval. Experiments on Open-Sora, Open-Sora-Plan, Latte, Wan 2.1, and HunyuanVideo report up to 2.6× speedup with comparable visual quality, along with scaling results under multi-GPU DSP and ablations on the indicator threshold and reuse interval.", "questions": "See Weakness", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. The mechanism is training-free, compatible with a range of DiT video generators, and easy to integrate at inference time.\n\n2. The use of a block-wise mean relative L1 indicator to decide reuse is principled and aligns with the observed per-timestep dynamics.\n\n3. Results span multiple DiT-based models and include comparisons to ∆-DiT, T-GATE, PAB, and TeaCache, plus scaling behavior with DSP and ablations over threshold δ and reuse interval R.\n\n4. The paper gives quantitative and qualitative evidence that BWCache sustains visual fidelity at meaningful acceleration levels.", "weaknesses": "1. The comparison with related work in Table 1 is not sufficiently comprehensive. Several recent and relevant caching-based acceleration methods, such as FasterCache and TaylorSeer, are missing from the comparison.\n\n2. The appendix shows that the proposed method increases memory consumption compared with the original base model. It should also include the memory usage of other representative cache-based methods to demonstrate the superiority of the proposed approach.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)