DiCache: Let Diffusion Model Determine Its Own Cache
Authors: Jiazi Bu
Pengyang Ling
Yujie Zhou
Yibin Wang
Yuhang Zang
Dahua Lin
Jiaqi Wang
Venue: ICLR 2026 Poster (2026)
Keywords: diffusion model
generative model
inference acceleration
Primary area: generative models
TL;DR: DiCache is a training-free adaptive caching strategy for accelerating diffusion models at runtime.

Abstract: Recent years have witnessed the rapid development of acceleration techniques for diffusion models, especially caching-based acceleration methods. These studies seek to answer two fundamental questions: _"When to cache"_ and _"How to use cache"_, typically relying on predefined empirical laws or dataset-level priors to determine caching timings and adopting handcrafted rules for multi-step cache utilization. However, given the highly dynamic nature of the diffusion process, they often exhibit limited generalizability and fail to cope with diverse samples. In this paper, a strong sample-specific correlation is revealed between the variation patterns of the shallow-layer feature differences in the diffusion model and those of deep-layer features. Moreover, we have observed that the features from different model layers form similar trajectories. Based on these observations, we present **DiCache**, a novel training-free adaptive caching strategy for accelerating diffusion models at runtime, answering both when and how to cache within a unified framework. Specifically, DiCache is composed of two principal components: (1) _Online Probe Profiling Scheme_ leverages a shallow-layer online probe to obtain an on-the-fly indicator for the caching error in real time, enabling the model to dynamically customize the caching schedule for each sample. (2) _Dynamic Cache Trajectory Alignment_ adaptively approximates the deep-layer feature output from multi-step historical caches based on the shallow-layer feature trajectory, facilitating higher visual quality. Extensive experiments validate DiCache’s capability in achieving higher efficiency and improved fidelity over state-of-the-art approaches on various leading diffusion models including WAN 2.1, HunyuanVideo and Flux.

Published as a conference paper at ICLR 2026
DICACHE: LET DIFFUSION MODEL DETERMINE
ITS OWN CACHE
Jiazi Bu Pengyang Ling2.5+ Yujie 2ho
Yibin Wang*6
Yuhang Zang² Dahua Lin Jiaqi Wang 5.6
Shanghai Jiao Tong University UUnierriity of Science and Technology of China
*Fudan University "The Chinese University of Hong Kong *Shanghai AI Laboratory
"CIII under InnoHK https: /twj//ujaazz github. i i cache. github. io/
"Shanghai Innovation Institute
ABSTRACT
Recent years have witnessed the rapid development of acceleration techniques for diffusion models, especially caching- based acceleration methods. These studies seek to answer two fundamental questions: "When to cache" and "How to use cache", typically relying on predefined empirical laws or dataset-level priors to determine caching timings and adopting handcrafted rules for multi-step cache utilization. However, given the highly dynamic nature of the diffusion process, they often exhibit limited generalizability and fail to cope with diverse samples. In this paper, a strong sample-specific correlation is revealed between the varia- tion patterns of the Mhallow-laeer feature differences in the diffusion model and those of deep-layer features. Moreover, we have observed that the features from different model layers form similar trajectories. Based on these observations, we present DiCache, novel training-fres adaptive caching strategy for accelerating diffusion models runtime, answering both when and how to cache within a uni- fied framework. Specifically, DiCache is composed of two principal components: (I) Online Probe Profiling Scheme leverages a shallow-layer online probe ob- tain an on-the-fly indicator for the caching error in real time, enabling the model to dynamically customize the caching schedule for each sample. (2) Dynamic Cache Trajectory Alignment adaptively approximates the deep-layer feature output from multi-step historical caches based on the shallow-layer feature trajectory, facilitat- ing higher visual quality. Extensive experiments validate DiCache's 's capability in achieving higher efficiency and improved fidelity over state-of-the -het--rrt-- approaches various leading diffusion models including WAN 1, Hunyuan Video and Flux. com Bu jiazi /
Our code is available at
INTRODUCTION
Over the past few years, diffusion models (Song & Ermon 2019: Sohl-Dickstein et al. 2015 Ho
al. 2020 Dhariwal Nichol, 2021 have markedly advanced frontiers of visual synthesis.
Initially grounded in the lightweight U-Net (Ronneberger et al.] 2015) architecture, diffusion models have achieved substantial progress in both image (Rombach et al., 2022 Podell et al.) (023) and video synthesis Blattmann et al. (2023) : Guo et al. (2023); Chen et al. (2024a). Recent efforts (Esser
ct al. 2024; Labs] [024]] Chen et al. 2023) have integrated transtormer-based architectures into diffusion models for larger model capacity and improved performance, especially in the field of video generation (Kong et al.) 2024 Wan et al., 2025) HaCohen et al., 20244) Yang et al.) 2024)). Despite their effectiveness, diffusion models often suffer from substantial inference costs and low generation speed, primarily stemming from the rapid expansion in model scale and intricacy.
To mitigate these limitations, numerous approaches (Lu al. 2022 Li et al. 2024 Liu et al.
2025a) on accelerating diffusion models have been proposed.
Most training-based accelcration
strategies (Meng al. 2023 Sauer et al. 2024 Wang et al. 2023 Chen et al. 2025 Ma et al.
"Equal contribution. 'Corresponding author.

---

Published as a conference paper at ICLR 2026
(o) Previous Caching Strotegies
(b) Our proposed DiCoche
cnn Marld
need sins actro bnow acge
BB
deramer prlrr
engiricll loer
Offfcim Meld
can denuratia my cedking scheckle #reall througt cellne probed
O
Compatt
Skip
Compute
Slop
O
Compate
Compute
handkrofied/rsiss
adgphin wefinemen
combiie
(c) Visual Comporison
D3 WAN 2.1 148 (8) frones)
Hanyuaniidde (129 frames)
1.00
Venlo 10C+
Teatsche
tour plannz ahaa Squars
00244-0044 Lrian)
'Hary rallong
Villoaen stKias (ASS, duralty eightene, colorati thackeation daiy siglt"
1o Augg wwved with ango wwlkaggaaooooo.o.......... and Eiiaaing STAANS CONGATT
Figure 1: Comparison between our proposed DiCache and previous caching methods. (a) Pre- vious studies typically rely on dataset priors or empirical laws to skip timesteps, and resort to hand- crafted rules utilize multi-step caches. (b) DiCache employs online probe adaptively adjust its caching strategy at runtime. (c) A brief visual comparison between DiCache and existing state- of-the-art methods, in which DiCache demonstrates superiority in both quality and efficiency.
2024 Li et al. 2024) entail significant training costs and require supplementary training data, which
less suitable for broad deployment. Therefore, recent studies have focused on training-free ac-
celeration methods (Zhang et al., 2025b Xi et al. 2025 Ren et al. 2025 Lu et al.) 2022 Liu et al.
2025a). Among them, caching- based methods (Chen et al. [022bb Selvaraju al. 2024 Xu al.
2018 Zhao et al. 2024 Lv et al. (024) provide lightweight choice for diffusion model accelera- tion by leveraging the similarity between features at consecutive timesteps and reusing them strate- gically to reduce computation redundancy. Early uniform caching strategies reuse cached features at fixed intervals, which fail to adapt to the diffusion process's vareinayying behaviors, resulting in low inference speed and degraded visual quality. Recently, AdaCache (Kahatapitiya et al., 2024) sug- gests dynamically modifying the cache interval based on content complexity. TeaCache (Liu et al.) 2025a) employs polynomial function calibrated offline datasets to determine its cache schedule. EasyCache (Zhou et al. 2025) replaces this polynomial estimator with an empirical transformation rate law. TaylorSeer (Liu al., (0225)) leverages multi-step caches under Taylor expansion-based forecast mechanism to predict current features for enhanced visual texture and details.
Caching-based acceleration confronts two core issues: "When to cache" " and "How to use cache". Existing methods address these questions by (1) relying on predefined empirical laws or dataset- level priors to make their cache schedules, and (2) resorting to manually crafted rules for multi-step cache utilization, as shown in Fig. (a). These key limitations hinder their flexibility to adapt to diverse samples, leading to suboptimal inference efficiency and reduced similarity regarding original results, as illustrated in Fig. (c). Therefore, an adaptive paradigm wherein the model autonomously determines its reuse strategy without recourse external empirical priors is imperative.
In this paper, we uncover that (1) for a given sampling process, the difference in shallow-layer features strongly correlates with that in deep-layer features on a sample-specifie basis (Fig. 3 enabling them to serve an on-the-fly proxy for the final model output evolution. Since the optimal moment to reuse cached features is governed by the difference between model outputs at consecu- tive timesteps (Liu et al. 2025a), it possible to employ an online shallow-layer probe to efficiently obtain an indicator of output changes at runtime, thereby adaptively adjusting the caching strategy for each individual sample. (2) the features from different DiT blocks form similar trajectories
2

---

Published as a conference paper at ICLR 2026
(Fig. 4) which allows for dynamically extrapolating the deep-layer feature output through combin- ing multi-step historical caches based on the shallow-layer probe feature trajectory.
In light of these observations, introduce DiCache. novel plug- doug-and-pan and-play adaptive caching strat- egy for accelerating diffusion models at runtime, as depicted in Fig. (b). Specifically, DiCache is composed of two principal components: First, Online Probe Profiling Scheme is proposed to harness shallow-layer features of the diffusion model to estimate the caching error in real-time, thereby more accurately tailoring the cache schedule for each sample. Second, Dynamic Cache Trajectory Align- ment is introduced to adaptively approximate the deep-layer feature output from multi-step caches based on the probe feature trajectory, which further elevates visual quality. By integrating the above two techniques, DiCache intrinsically answers when and how to cache within unified framework.
Our contributions can be summarized follows: (1) Shallow-Layer Probe Paradigm: We intro- duce an innovative probe-based approach that leverages signals from shallow model layers to predict the caching error and effectively utilize multi-step caches.
(2) DiCache: We present DiCache, a novel caching strategy that employs online shallow-layer probes to achieve more accurate caching schedules and superior multi-step cache utilization. (3) Superior Performance: Comprehensive experiments demonstrate that DiCache consistently delivers higher efficiency and enhanced visual fidelity compared with existing state-of- ooe---ffthhhrrrr methods on leading diffusion models including WAN 2.1 (Wan et al. 2025), Hunyuan Video (Kong et al., 2024), and Flux (Labs 2024).
2 RELATED WORK
Diffusion Model. Diffusion models (Song & Ermon 2019 Sohl-Dickseee et al. 2015 Ho et al.,
2020 Dhariwal & Nichol 202T) have demonstrated exceptional capability to synthesize content with superb quality and rich diversity. Early pioneers like Stable Diffusion (Rombach et al. 2022) and SDXL (Podell et al., 2023) achieved high-fidelity image generation with the U-Net (Ron- neberger et al. 2015) architecture. which also inspired subsequent video diffusion models (Guo ct al. 2023 Chen et al. 2024a; Blattmann et al. 2023). Despite its success, the U-Net architecture possesses limited scalability, hindering the training and deployment of large-scale models with im- proved performance. To overcome this constraint, recent diffusion models (Esser et al., 2024 Labs
2024 Chen et al. 2023 Peebles & Xie 2023) have widely adopted diffusion transformers (DiT) as their backbones for better scalability and flexibility, delivering superior outcomes across various
domains (Wu et al. 2025 Wan al. 2025 Kong et al. 2024 Yang et al. 2024).
Diffusion Model Acceleration. Despite the striking achievements drawn by diffusion models, their inference cost increases substantially with model capacity and complexity. posing challenges to practical applications. Accordingly, acceleration techniques (Dao et al. 2022 Li et al. 2024 Liu al., (025a)) have emerged and gradually become key research field in the realm diffusion models, with existing studies generally falling into the following categories: efficient attention (Dao al., 2022; ZZhag et al. 2025b), sparse attention (Xi et al., 2025 Ren al.) 2025), model dis- tillation (Meng et al." 2023, Sauer et al. 2024; Wang et al. 2023), model quantization (Li et al.) 2024). improved SDE or ODE solvers (Lu et al. 2022). and caching-basse feature reuse strate- gies (Liu et al. 2025a b Zhou et al. 2025). Among them, caching- based methods (Liu et al.) 2025a] Zhao et al. 2024) have recently gained increasing attention due to their lightweight nature. For instance, DeepCache (Xu et al.) 2018) and Faster Diffusion (Li et al. (023) reuse the U-Net feature across timesteps to reduce computation redundancy. FORA (Selvaraju et al. (202)) and A- DiT (Chen al. 2024b) extend this idea transformer-bases backbones. PAB (Zhao et al. 2024) broadcasts attention features to subsequent steps in a pyramid style based on different block charac- teristies. AdaCache (Kahatapitiya et al., (024) dynamically modifies residual reuse strategies based on the content complexity. FasterCache (Lv et al.) (024) proposes to cache for both the conditional branch and unconditional branch of Classifier-Free Guidance (CFG) (Ho Salimans 2022). Tea- Cache (Liu et al., 2025a) leverages calibrated polynomial estimator to predict output changes from input differences. EasyCache (Zhou et | (025) replaces this polynomial function with an empirical transformation rate law. TaylorSeer (Liu et al. 2025b) suggests combining multi-step cached fea- tures in a eeyylor-xpannsion-le laws dataset-level priors, we enable the diffusion model autonomously determine the caching timings and adaptively utilize multi-step caches according to an online probe at runtime, thereby better adapting to the highly dynamic diffusion process and the diverse sample distribution.
manner. Unlike existing methods that rely on predefined empirical
3

---

Published as a conference paper at ICLR 2026
When to Cache: if Zerror 6, then reuse cache: else recompute
(1) Online Probe Profiling Scheme
fp
Ep
1
...
t
t (Current)
m K- Loyers
M
m
M
m
M
m
M
Cached
Reused
Reused
Cached
Reused
TH
How to use Cache: align with probe trajectory
Caching Enror Indicator:
Eecror
Llea(ti". .ta.)
(2) Dynamic Cache Trajectory Alignment
Probe Trojectory
t-Tp
Trejectory Alignment:
T2th
+ Pr
T1
Target Trajectory
Ttp
Traj Align
Directly Reuse
Tta
A Process:
Compute
Skip
TEp
The proposed DiCache consists Online Probe Profiling Scheme and Dynamic Cache Trajectory Alignment. The former dynamically determines the caching timing with an online shallow-layer probe at runtime, while the latter combines multi-step caches based on the probe feature trajectory adaptively approximate the feature at the current timestep.
Figure 2: Overview of DiCache.
METHOD
3.1 PRELIMINARY
Flow Matching. Flow Matching (Lipman et al. 2022) and Rectified Flow (Liu et al. 2022) are de- signed to simplify the construction of Ordinary Differential Equation (ODE) models by delineating linear mapping between disparate probability distributions. Given samples i) drawn from the noise distribution и and то from the clean data distribution To, the transition from 21 to xo is modeled as linear trajectory in the direction of the vector (zo x1). with the intermediate state at timestep t E [0, 1)) denoted as 2; = f.r + (1 f) Hence, the ODE governing Ia can formulated as dæ, (ao .I dtt. Since Zu is unknown during denoising, a learned velocity field vo (xt, t, c) is employed to approximate the direction (x5- 21), thereby constructing a neural ODE model:
dē, = va( L e)dl,
(1)
where stands for extra input conditions such textual or image prompts.
Diffusion Transformer. The Diffusion Transformer (DiT) (Peebles Xie 2023 Esser 2024)
architecture adopts a tiered layout with M cascaded blocks, in which self-attention (SA), cross- attention (CA), and a multilayer perceptron (MLP) are integrated as a unified module. Let B, denote the i-th DiT block (i E [1, M.) yf the output of the i-th DiT block at timestep t, the transformation between yf and model input latent Ia can be formulated as:
s B, B
B2 B1 (z),
in which yt represents the final model output at timestep t, i.c. the predicted velocity vo(are, t, e).
Residual in DIT. The residual between the output of the i-th DiT block vł and the initial model input latent x, at timestep & can be expressed as:
in which rM = y/M xx, vo(x velocity and its input, i.e. the residual of the entire diffusion transformer. Since the residual effec- tively captures the evolution of features across DiT blocks, we choose cache for model residuals.
Ia represents the difference between the model's predicted
4

---

Published as a conference paper at ICLR 2026
(4) Chepue Deffrrence
(b) lepui Diffzrenoe
(c) Shalllww- ayer Diflerene
Spearman Conclation Duigui
RuaussVdd
431
43
Diflaiko Process
Difininas Preces (5)
Difioce Poocss
Pode Deplr
Diffaion Frosew (S]
Daffuik Fridss (7)
Diftace Process (4)
Probe Depii
Figure 3: Variation of feature differences between consecutive timesteps (Mean & Variance), (a) Output differences (ye, 1. It exhibits large variances, indicating sample-specific nature. (b) Input differences L.1 (F, F++1). It increases monotonically with timesteps, failing to capture the variations in output differences. (c) Shallow-layer feature differences [ v in this figure). exhibits a strong correlation with the variations in output differences. (d) Spearman correlation coefficient between L (vl" 31+11 and Ll 1t+1). They already exhibit high correlation coefficient (around 0.8) with shallow probe depth (I~3 layers).
(5-th layer
3.2 DICACHE
Instead of resorting to data-driven or empirical metrics, DiCache employs an online shallow-layer probe to efficiently obtain an indicator of diffusion dynamics, offering guidance for both caching timing determination and multi-step cache utilization. Specifically, we propose Online Probe Profil- ing Scheme to address "When cache" " and Dynamic Cache Trajectory Alignment to answer "How to use Cache", " respectively. The overall framework of DiCache is illustrated in Fig.
Online Probe Profiling Scheme. We begin by analyzing the caching error. Given that the optimal moment to reuse cached features depends on the evolution of model outputs over timesteps (Liu et al. 2025a), the ideal caching error c,,+1 can be defined as the difference between model outputs at adjacennt timesteps and /. ++1
4,,+1 = L1 (et M141) = 11ve 86-1ll1
(4)
11
where Yt denotes the model output at timestep and the relative L1 distance (L1,a(-)) is adopted to measure feature differences. Once 61,4+11 is known, it is able to determine whether the cache should be used at timestep I. However, previous studies seek to anticipate these output variations using of- fline priors (Liu et al. 2025a) empirical heuristics (Zhou al. 2025), overlooking the specificity of individual samples, thus resulting in suboptimal generalizability on outlier instances. Unlike existing works, discover that: for specific sample, the variation patterns of its shallow- layer feature differences remain consistent with those of its deep-layer feature, demonstrat- ing that efficient indicator for caching error can be obtained without delving deep into the model. Consider a diffusion model with M blocks/layers, let I; denote the input and y!" the output of the m-th layer (m [1, M). yfd yt. i.e. the final model output) at timestep demonstrate that Lya"" v.+ can reflect the dynamics of (y, M with m << M. As shown in Fig. 3 (a)-(c), while (1,, Z(+1) fails to capture the variation pattern of ( yM Ly y4t lm=5 effectively characterizes it with only m 5. Moreover, as depicted in Fig. (d), the Spearman correlation coefficient between Llaa(""" y and LLaa(y 1 reaches a significant level (around 0.8) even when m is very small (m [1.
Motivated by this insight, LLa v(t1) can be utilized as an estimated caching error with m--h layer online probe:
,, LL v( = 1| vo
(5)
Based on the above analysis, we propose an Online Probe Profiling Scheme to efficiently and accu- rately determine caching timings inference time, detailed in Algorithm I
Instead of relying
5

---

Published as a conference paper at ICLR 2026
Fazars Treesdry of Raliaed
Futte Traketry ofTrch Recial
Rehttieniig bntwonn v (hmés Depia m
Verizee
035
4..0
8.59
022
0.75
(4) feature trajestories of residual probe residual (PCA projections)
(b) cchibits high similarity
Figure 4: Observation and analysis regarding Dynamic Cache Trajectory Alignment.
on pre- pre-trained estimators other empirical metrics, only infer the first (m << M) layers of the model as shallow-layer probe at each timestep to obtain a proxy of (yt MM and com- pute estimated caching error 61,+1 via Eq. 5 An accumulated caching error is maintained across timesteps as metric for caching error tolerance:
b++
Σ GLt 56<
Σ 6,441
(6)
in which ò a user-specified threshold. Specifically, after computing and caching the model residual timestep /1, we accumulate the estimated caching error 6,,+1 and reuse the cached residual for each subsequent timestep. When the accumulated error exceeds the threshold at timestep 12 (12 < t1), the residual recomputed to refresh the cache and the accumulated caching error reset to zero. is noteworthy that our strategy inherently supports resuming computation from the probed layer, thereby incurring no additional cost at timesteps requiring recomputation.
Algorithm Online Probe Profiling Scheme
1: Inputs: Model input Zy, Sampling steps T, DiT blocks B; (i [1, Extra conditions
2: Parameters: Reuse threshold 8, Probe depth m (m M)
3: Initialize accumulated estimated caching error Exror cached residual R None
4: for =
do 5: t T then
6: Yt + Bar o .... (2,,,, f) Calculate for the first step
7: R y: 24 # Initialize cache from None
8:
else
yr +- B
(1 c) Probe the first layers Eame Eeno Lha"""" 1/+1)) Accumulate caching error
11:
if Eenor S then
12:
y/ & It + R2# Reuse cached residual
13:
else
14:
Y/ By o...o Bm+1(y?"" # Resume from the probed layer
R y1 Fı, Veroo 0 # Refresh cache and error accumulator
end if
17:
end
18:
end for
Dynamic Cache Trajectory Alignment. While Online Probe Profiling Scheme resolves "When Cache", the generation quality still limited by insufficient cache utilization. To this end, further propose a simple yet effective Dynamic Cache Trajectory Alignment (DCTA) strategy to answer "How to use cache". Previous works (Lv al. 2024 Liu et al., 2025b) usually employ fixed handcrafted rules such as Taylor expansion for utilizing multi-step caches, lacking adaptability and risking deviation from original results, as depicted Fig. (c))). this work, we discover that features from different model layers exhibit similar trajectories. As shown in Fig. 4 (a), the shallow-layer probe residual from the m-th layer r?" (m 5 here) and the residual of the entire model rt display analogous dynamic trends. Since the residual represents the transformation direction from the latent space input to the velocity space output, for a well-trained DiT, these directions generally aligned across different blocks. Therefore, reliable signal can be derived from the shallow-layer probe to guide the utilization multi-step cached residuals for a better approximation of the current residual.

---

Published as a conference paper at ICLR 2026
(DiCada)
Promeed OventCh Devegere
Dynamic Trajectory Alignment on
'Setec vies of vish nountaie berignunnn""
Detail Preserved Motion Preserved
Detail Changed Motion Changed
(b) Effects of Dynaicc Cache Trijectory Aligmmen: WAN 2.1
Figure 5: Effects of Dynamic Cache Trajectory Alignment (DCTA). Best viewed zoomed in.
Assuming the two most recent recomputation timesteps are ta and t& with fa tg, and they cache model residuals rt,, and ftg. respectively. We compute the current residual in the following form:
Tt Tta + (rr,, - :t,))
(7)
where 7t stands for the residual trajectory parameter at timestep characterizes the dynamic trend of the residual's feature trajectory at each timestep. By emphasizing such trends of residual changes across timesteps, Eq. 7 enables a more precise approximation
the current residual compared to directly using the most recent cache Tt. More discussion on the design of Eq. 7 can be found in Section [ in the appendix. Nevertheless, since the diffusion process is unpredictable, the optimal value of % is unknown before the model is inferred at timestep t. Here, propose to leverage the online probe feature trajectory to dynamically estimate 7t value at each timestep. Specifically, after computing the probe feature y/"" its corresponding probe residual is calculated as:
Vi It.
(8)
Since the probe is computed at each timestep, the ground-truth values of r? r and ri are already available. Therefore, the ideal feature trajectory can be established among them:
ri (r r(;).
(9)
in which the probe residual trajectory parameter can be directly solved by:
I. (r (rg)
π L (r.) r.p)
(10)
Given the similarity between the probe residual trajectory and the full residual trajectory, f exhibits high consistency with 7/ even under small m values, as shown in Fig. (b). Consequently, 7. can be utilized as efficient substitute for T in Eq. 7
(T mi (re. - rtg),
Tt Ttp + felr,, - rrt = rto + Ll (r
(11)
which allows for adaptive adjustment of the trajectory parameter, thereby more accurately estimating the full residual with multi-step caches and facilitating hetter visual quality, as illustrated in Fig.
4 EXPERIMENTS
4.1 IMPLEMENTATION DETAILS
Experimental Settings, We conduct our experiments on three leading DiT-based diffusion models: WAN 2. 1-1.BB (Wan et al. 2025) and Hunyuan Video (Kong et al. 2024] for video generation, and Flux. 1. .0-dev (Labs) 2024) for image generation. For hyperparameters, we set probe depth m = 1 for all three models, while reuse threshold is set and = 0.4 for Flux. respectively. If not specified, experiments are conducted on a single NVIDIA A800 80GB GPU. More implementation details can be found in Section B in the appendix.
0.2 for WAN 2.1, 8 0.1 for Hunyuan Video,

---

Published as a conference paper at ICLR 2026
Baselines. The compared methods encompass (i) two naive acceleration strategies (directly reduc- ing the number of timesteps by 50% and uniform caching with different static cache intervals) as well as (ii) various of saching-basee methods, including TeaCache (Liu et al.
2025a
TaylorSeer (Liu et al. 2025b), EasyCache (Zhou et al. 2025) and ToCa (Zou et al.
2024).
Evaluation Metries. We collect 300 high-quality text prompts from VideoDPO Liu al. 2025c)
and VideoFeedback (He et al. 2024) datasets (polished by GPT-4o (Hurst et al.
2024)) for video
generation and IK captions sourced from the LAION-5B (Schuhmann et al. 2022
dataset for image
generation. Following previous works (Liu et al. 2025al Zhou et al. (022). speedup ratio and
latency are reported measure inference efficiency, while LPIPS (Zhang et al.
2018). PSNR, and
SSIM regarding original results are adopted to assess visual quality.
4.2 COMPARISON TO EXISTING METHODS
Quantitative Evaluation. The quantitative assessments presented in Tab. I We begin by com- paring DiCache with two basic cache strategies: direct step reduction and uniform caching with a static caching interval I. Despite their simplicity, they fail to adapt the dynamic diffusion process, resulting in considerable quality loss (about 30% SSIM reductions and 40% PSNR loss compared DiCache WAN 2.1). While TaylorSeer achieves an impressive 3.43x speedup WAN 2.1, its generation results deviate significantly from the original with an LPIPS of 0.5214. Moreover, its long-range feature forecast mechanism imposes severe burden on GPU memory, even encoun- tering out-of-memory issues on Hunyuan Video when using a single A800 GPU. TeaCache overly relies on data priors and is prone overfitting to training prompts, leading unstable performance on unseen cases. Though TeaCache-slow maintains relatively acceptable quality at low acceler- ation rate, notable performance drop occurs once pushing acceleration further to TeaCache-fas (with LPIPS of 0. 0.2898, SSIM of 0.8015 and PSNR of 22.01 on Hunyuan Video). EasyCache yields suboptimal efficiency across various models due to its inability to precisely capture the dif- fusion dynamics with empirical transformation rate metric. In contrast, dynamically determines its caching timings and effectively utilizes multi-step caches based on online probes, achieving unification of rapid inference speed and high visual fidelity (e.g.. LPIPS 0.1492, SSIM 0.9396, PSNR 32.79 and 2.34x speedup on Hunyuan Video). On the image generation back- bone Flux. 1.0-dev, the benefits of DiCache become even more pronounced, with markedly superior performance in efficiency and quality across all evaluated baselines. In addition, DiCache can be integrated with other acceleration methods offer even higher speed (detailed Section 4.3
proposed DiCache
Table 1: Quantitative comparison with baselines. The best result is highlighted in bold, while the second-best result underlined. "OOM" indicates CUDA out memory on the A800 80GB GPU.
Model
Method
LPIPS
SSIM
PSNR 1
Speedup
I Latency (sec) J
Vanilla (100% steps)
Vanilla (50% steps)
0.4143
0.6304
16.19
1.83
105.25
Uniform Cache (J
0.4740
0.5927
15.16
2.39x
80.44
TeaCache-slow Liu
2025a
0.1939
0,8374
22.60
1,82x
105.83
WAN
TeaCache fast
Las et m
0.2161
0.8226
20.97
2.20x
87.58
TaylorSecr
2755
0.5214
0.5485
14.32
3.43x
56.05
EacyCache
Thou
20.35
0.2013
0.8562
24.80
2.21
86.96
DiCaene (Ours)
0.1734
0,8885
26.45
2.45
78.42
Vanilla (100% steps)
1.00x
118..32
Vanilla (50% steps)
0.4138
0.7134
17.90
1.98
599.65
Uniform Cache (J
0.4111
0.7132
17.92
1.91x
622.45
Hunyuan Video
TeaCache-sluw (Liu et
TeaCache-fast (1.
2025a
0.2762
0.2898
0,8114
0,8015
22.41
22.01
2,20
1.65x
717.21
$38.49
TaylorSeer eLu
20256
00M
0OM
COM
OOM
0OM
EasyCache
0.1558
30.71
558.71
DiCache (Ours)
0.1492
0.9396
32.79
2.34x
507.24
Vanilla (100% steps)
1.00x
15.11
Vanilla (50% sleps)
0.3679
0.7370
18.04
1.98
7.63
Uniform Cache (1
3)
0.4034
0.7013
17.09
2.55
5.92
TeaCuche-slow [Liu
2025a
0.2810
0,8036
21.81
1.71x
8.86
Flux
TeaCache-fast Liu 2t
0.4053
0.7219
18.01
2,82x
5.36
ToCa (Zoi
9.92
TaylorSedr [.:i
2br
0.4709
0.6721
16.63
3.13%
4.8
EasyCache
Ziko er
2015
0.3049
0.7527
19.75
2.49%
6.06
DiCache
0.2704
0,8211
22.39
3.22%
4.69
8

---

Published as a conference paper at ICLR 2026
(a) WAN
Bi Humyuan Video
Flux
Vaoille
Toi ToiCaele
Taylomeea
Out of Memory on A800 GPU
of Memory on A800 GPU
DiCachh:
Adg seaag anves Affer Onfins TM cle gier
counter Noow Aookkrtii Moskurklrrmo "Cyne plorrcc Ne suher allgyy
Figure 6: Qualitative comparison with previous acceleration methods. Best viewed zoomed in.
Qualitative Comparison. 6 presents the visual comparison between the proposed DiCache and other baselines. While TeaCache achieves a notable speedup, the visual quality suffers significant degradation. For instance, compared with the vanilla results, the textures of the dog and the road surface generated by TeaCache exhibit a marked loss. TaylorSecr employs a Taylor expansion-base feature forecast mechanism to explicitly enhance texture details. However, it often manifests low similarity to the original results and exhibits abnormal color contrast (especially with Flux). Easy- Cache adopts predefined cache law, suffering from inaccurate caching timing determination and insufficient cache utilization. The results produced by EasyCache deviate from the original results in critical details such the dog's ears and human hands. Moreover, such a heuristic rule sometimes leads to generation collapse, resulting in degraded outputs (e.g.. the "raccoon" case generated by EasyCache with Flux). comparison, the proposed DiCache consistently outperforms the base- lines in terms of both visual quality and similarity to the original results across diverse scenarios and generation backbones. More visual comparison results are presented in Section C the appendix.
4.3 COMPATIBILITY WITH OTHER ACCELERATION TECHNIQUES
We analyze the proposed DiCache's compatibility with other acceleration techniques like sparse attention and model distillation to demonstrate its ficxibility and versatility. Experiments in this section are conducted on single NVIDIA H200 140GB GPU.
Compatibility with Sparse Attention. We integrate DiCache with the advanced sparse attention method Sparse VideoGen (SVG) (Xi et al. 2025) on Hunyuan Video (720 x 1280 resolution, 129 frames). As depicted in Fig. 14 in the appendix, by combining DiCache with SVG, we achieve a 3.08 x speedup with negligible loss in visual quality. Moreover, as shown in Tab, 2 the incorporation of DiCache into SVG leads to a significant improvement in efficiency compared to using SVG alone (from 1.67x speedup to 3.08x speedup), without noticeable degradation in quantitative metrics.
Table 2: DiCache's compatibility with the sparse attention method Sparse VideoGen (SVG).
Model
Method
LPIPS J.
SSIM
PSNR
Speedupp
Latency (sec)
Vanilla
1.00x
1747.30
VinyuanVideo
SVG Xi et
2025
0.1760
0,9034
28.18
1.67%
1047.63
SVG
Xi
at 2025
+ DiCaahhe
0.2051
0.8882
27.20
3,08%
567.45
Compatibility with Model Distillation. We combine DiCache with the distilled version of WAN 2.1B14B (by Video (Zhang et al. 2025a), with 10 sampling steps), achieving a notable accelera-

---

Published as a conference paper at ICLR 2026
tion without compromising visual quality. The quantitative evaluation is presented in Tab. the qualitative results are shown in Fig. in the appendix.
while
Table 3: DiCache's compatibility with the model distillation method Videoo
Model
Method
LPIPS 4
SSIV
PSNR
Speedup
Latency (sce)
WAN WAN 2.1-14B
AccVidea (Zhang dtal.
24
AccVideo Zhhnn et in. 2025a
DiCache
0.2737
0.7432
19.72
1,36x
30.98
4.4
ABLATION AND ANALYSIS
We conducted ablation studies on the video generation model Hunyuan Video, as shown in Tab,
Choice of Probe Depth m. m determines the number of DiT blocks/layers that are probed each sampling step. Empirically, a larger m leads to more accurate cache timing decisions but also incurs a higher probe time cost. However, as illustrated in Fig. 3 (d), Ll -3--1) and Ly y241) already exhibit a strong correlation when m is very small (m E [1, Therefore, m = 1 is chosen offer higher efficiency. Ablation results on probe depth m are presented in Tab, (a).
The value of stands for the tolerance level for caching emrors (i.e., the trade-off between quality and efficiency). As depicted in Tab. (b), adopting smaller å yields superior visual quality, yet concurrently results in increased latency, as the model is inferred at more timesteps. Conversely, a larger facilitates higher inference speed but entails a certain degree of quality loss. 8 = 0.1 is chosen in experiments to strike balance between quality and efficiency.
Choice of Reuse Threshold 8.
Effeets of Dynamic Cache Trajectory Alignment (DCTA). Dynamic Cache Trajectory Alignment aims at better approximating features at the current timestep through combining multi-step caches based on shallow-layer probe feature trajectory. As shown in Fig. | and Tab. (c), Dynamic Cache Trajectory Alignment leads to improved visual quality and similarity regarding original results. Ad- ditional visual ablation results of DCTA are presented in Fig. 15 in the appendix.
Table 4: Ablation experiments on DiCache components and hyperparameters.
Components
Values & Choices
LPIPS
SSIM
PSNR
Speedup
Latency (sec)
0.1367
0.9495
33,47
2.10x
563.68
(a) Probe Depth m
m
0.1397
0.9472
33.30
2.20x
539.83
0.1492
0.9396
32.79
2.34x
507.24
0.20
0.1886
0.8980
29.81
2.90
408.62
0.15
0.1665
0.9131
30.92
2.62x
452.33
(b) Reuse Threshold 5
0.10
0,1492
0.9396
32.79
2
507.24
5
0.08
0.1305
0.9433
34.03
2.10x
564.73
0.05
0.1047
0.9584
35.45
1.76x
672.20
w/o
0.1517
0.9314
31.98
2.34
506.82
(c) DCTA
w/
0.1492
0.9396
32.79
2.34X
507.24
CONCLUSION
We present DiCache, a novel training-frit -free adaptive caching strategy for accelerating diffusion mod- els at runtime. DiCache consists of Online Probe Profiling Scheme and Dynamic Cache Trajectory Alignment, the former dynamically determines caching timings based on a shallow-layer online probe, while the latter adaptively utilizes multi-step caches according to the probe feature trajectory. Extensive experiments validate the effectiveness of DiCache on various leading diffusion models.
ACKNOWLEDGEMENT
This project is funded in part by Shanghai Artificial Intelligence Laboratory, Shanghai Innovation Institute, the Centre for Perceptual and Interactive Intelligence (CPII) Ltd under the Innovation and Technology Commission (ITC)'s InnoHK. Dahua Lin is a PI of CPII under the InnoHK.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Záon English, Vikram Voleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint 1rXiv:231..15127, 2023.
Haoxin Chen, Yong Zhang, Xiaodong Cun, Menghan Xia, Xintao Wang, Chao Weng, and Ying Shan. Videocrafter2: Overcoming data limitations for high-quality video diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp-
7310-7320, 2024a.
Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, al. Pixart-a: Fast training of diffusion transformer for photore- alistic text-to-image synthesis. arXiv preprint arXiv:2310.00400 2023.
Lei Chen, Yuan Meng, Chen Tang, Xinzhu Ma, Jingyan Jiang, Xin Wang, Zhi Wang, and Wenwu Zhu. Q-dit: Accurate post-training quantization for diffusion transformers. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 28306-28315,, 2025.
Pengtao Chen, Mingzhu Shen, Peng Ye, Jianjian Cao, Chongjun Tu, Saristos-Savvar Bouganis, Yiren Zhao, and Tao Chen. 8-dit: A training-fre ffrn acceleration method tailored for diffusion transformers. arXiv preprint wrXiv:240.001125 2024b.
Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. Flashattention: Fast and memory- efficient exact attention with io-awareness. Advances in neural information processing systems, 13444-416,,,,9 2022.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780-8799 2021.
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first International Conference Machine Learning. 2024.
Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text-to-image diffu- sion models without specific tuning. arXiv preprint arXiv:2307.04725, 2023.
Yoav HaCohen, Nisan Chiprut, Benny Brazowski, Daniel Shalem, Dudu Moshe, Eitan Richardson, Eran Levin, Guy Shiran. Nir Zabari, Ori Gordon, et al. Ltx-video: Realtime video latent diffusion. arXiv preprint arXiv:2501.0010 2024.
Xuan He, Dongfu Jiang, Ge Zhang, Max Ku, Achint Soni, Sherman Siu, Haonan Chen, Abhranil Chandra, Ziyan Jiang, Aaran Arulraj, et al. Videoscore: Building automatic metrics to simulate fine-grained human feedback for video generation. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 2105-2123, 2024.
Jonathan Ho and Tim Salimans. arXiv:2207. 12598, 2022.
Classificr-free diffusion guidance.
arXiv preprint
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
Aaron Hurst, Adam Lerer, Adam Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024.
Kumara Kahatapitiya, Haozhe Liu, Sen He, Ding Liu, Menglin Jia, Chenyang Zhang, Michael Ryoo, and Tian Xie. Adaptive caching for faster video generation with diffusson transformers. arXiv preprint arXiv:2411.02397 2024.
Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint 142 2024.

---

Published as a conference paper at ICLR 2026
Black Forest Labs. Flux.
2024.
Muyang Li, Yujun Lin, Zhekai Zhang, Tianle Cai, Xiuyu Li, Junxian Guo, Enze Xie, Chenlin Meng, Jun- Yan Zhu, and Song Han. Svdquant: Absorbing outliers by low-rank components for 4-bit diffusion models. arXiv preprint
2024.
Semmao Li, Taihang Hu, Fahad Shahbaz Khan, Linxuan Li, Shiqi Yang, Yaxing Wang, Ming-Minn Cheng, and Jian Yang. Faster diffusion: Rethinking the role of unet encoder in diffusion models. CoRR, 2023.
Yaron Lipman, Ricky TQ Chen, Heli Ben- Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.02747, 2022.
Feng Liu, Shiwei Zhang. Xiaofeng Wang, Yujie Wei, Haonan Qiu, Yuzhong Zhao, Yingya Zhang, Qixiang Ye, and Fang Wan. Timestep embedding tells: It's time cache for video diffusion model. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 7353- 7363, 2025a.
Jiacheng Liu, Chang Zou, Yuanhuiyi Lyu, Junjie Chen, and Linfeng Zhang. From reusing to fore- casting: Accelerating diffusion models with taylorseers. arXiv preprint arXiv:2503.06923 2025b.
Runtao Liu, Haoyu Wu, Ziqiang Zheng, Chen Wei, Yingqing He, Renjie Pi, and Qifeng Chen. Videodpo: Omni-preference alignment for video diffusion generation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 8009-8019, 2025c.
Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning generate and transfer data with rectified flow. arXiv preprint arXiv:2209.03003 2022.
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in neural information processing systems, 35:5775-577 2022.
Zhengyao Lv, Chenyang Si, Junhao Song, Zhenyu Yang. Yu Qiao, Ziwei Liu, and Kwan- Yee K Wong. Fastercache: Training- faining-free video diffusion model acceleration with high quality. arXiv preprint arXiv:2410. 19355, 2024.
Xinyin Ma, Gongfan Fang, Michael Bi Mi, and Xinehao Wang. Learning-to-cant Accelerating diffusion transformer via layer caching. Advances in Neural Information Processing Systems, 37: 133282-13323 133304, 2024.
Chenlin Meng, Robin Rombach, Ruiqi Gao, Diederik Kingma, Stefano Ermon, Jonathan Ho, and Tim Salimans. On distillation of guided diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 14297-14306, 2023.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference computer vision, pp. 4195-4205, 2023.
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01955. 2023.
Sucheng Ren, Qihang Yu, Ju He, Alan Yuille, and Liang-Chieh Chen. Grouping first, attending smartly: Training-fres acceleration for diffusion transformers. arXiv preprint arXiv:2505. 14687, 2025.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedi- cal image segmentation. In International Conference on Medical image computing and computer- assisted intervention, pp. 234-241, Springer, 2015.
12

---

Published as a conference paper at ICLR 2026
Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach. Adversarial diffusion dis- tillation. In European Conference on Computer Vision, pp. 87-103. Springer, 2024.
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, al. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in neural in- formation processing systems, 35:25278-2551 25294, 2022.
Pratheba Selvaraju, Tianyu Ding, Tianyi Chen, Ilya Zharkov, and Luming Liang. Fora: Fast-forward caching in diffusion transformer acceleration. arXiv preprint arXiv:2407.01425 2024.
Jascha Sohl- Sohh-Diccstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learn- ing. pp. 2256-22662222 2265. pmir, 2015.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019.
Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang. et al. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.2033 20314, 2025.
Xiang Wang, Shiwei Zhang, Han Zhang, Yu Liu, Yingya Zhang, Changxin Gao, and Nong Sang. Videolem: Video latent consistency model. arXiv preprint arXiv:2312.09109, 2023.
Chenfei Wu, Jiahao Li, Jingren Zhou, Junyang Lin, Kaiyuan Gao, Kun Yan, Sheng-ming Yin, Shuai Bai, Xiao Xu, Yilei Chen, et al. owen-immaee technical report. arXiv preprint arXiv:2508.02324 2025.
Haocheng Xi, Shuo Yang. Yilong Zhao, Chenfeng Xu, Muyang Li, Xiuyu Li, Yujun Lin, Han Cai, Jintao Zhang, Dacheng Li, et al. Sparse videogen: Accelerating video diffusion transformers with seatial-temporal sparsity. arXiv preprint arXiv:2502.01770 2025.
Mengwei Xu, Mengze Zhu, Yunxin Liu, Felix Xiaozhu Lin, and Xuanzhe Liu. Deepcache: Princi- pled cache for mobile deep vision. In Proceedings of the 24th annual international conference on mobile computing and networking, pp. 129-144, 2018
Zhuoyi Yang. Jiayan Teng. Wendi Zheng. Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint S1iv:2248.06077 2024.
Haiyu Zhang, Xinyuan Chen, Yaohui Wang, Xihui Liu, Yunhong Wang, and Yu Qiao. Accvideo: Accelerating video diffusion model with synthetic dataset. arXiv preprint arXiv:2503. 1944622,..992, 2025a.
Jintao Zhang, Chendong Xiang, Haofeng Huang, Jia Wei, Haocheng Xi, Jun Zhu, and Jianfci Chen. Spargeattn: Accurate sparse attention accelerating any model inference. arXiv preprint arXiv:2502.18133 18137, 2025b.
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 586-595, 2018.
Xuanlei Zhao, Xiaolong Jin, Kai Wang, and Yang You. Real-time video generation with pyramid attention broadcast. arXiv preprint arXiv:2408. 12588, 2024.
Xin Zhou, Dingkang Liang, Kaijin Chen, Tianrui Feng, Xiwu Chen, Hongkai Lin, Yikang Ding, Feiyang Tan, Hengshuang Zhao, and Xiang Bai. Less is enough: Training free video diffusion acceleration via unttie--adaptiv caching. arXiv preprini 20Ni:2507.008600 2025.
Chang Zou, Xuyang Liu, Ting Liu, Siteng Huang, and Linfeng Zhang, Accelerating diffusion transformers with token-wir wise feature caching. arXiv preprint a.X1v:2410.05317, 2024.
13

---

Published as a conference paper at ICLR 2026
A APPENDIX
In the appendix. we present additional implementation details (Section additional qualitative results (Section C). text prompts used in both the main paper and appendix (Section D), more anal- ysis on the choice of reuse threshold (Section E. more discussion on Dynamic Cache Trajectory Alignment (Section[F), the limitations of our method (Section (G. the ethical statement (Section HH),
the reproducibility statement (Section T). the declaration on LLM usage (Section ( as well as the description of supplementary material
supplement the main paper.
B ADDITIONAL IMPLEMENTATION DETAILS
The detailed configurations adopted by each model in the main experiments are follows: (832 x 480 resolution, 81 frames, 50 sampling steps) for WAN 2.1, (544 x 960 resolution, 129 frames, 50 sampling steps) for Hunyuan Video, and (1024 x 1024 resolution, 30 sampling steps) for Flux.
C ADDITIONAL QUALITATIVE RESULTS
In this section, we present more visual comparison results between DiCache and existing caching- based methods in Fig. 7 Fig. 8 Fig.
Fig. TO Fig. 11 and Fig. [2] along with accelerated results
using the same prompts and different random seeds (0/1/2) presented in Fig. 13
D TEXT PROMPTS
All text prompts used to generate images or videos in this paper are listed in Tab.
E MORE ANALYSIS ON THE CHOICE OF REUSE THRESHOLD
Admittedly, the recommended value of the reuse threshold 8 varies across models, but the differences are relatively minor. These variations arise from the differences among DiT models with different training data and architectures (e.g.. Flux is T2I model with 57 layers and 12B parameters, while WAN 2. 1.B.BB is a T2V model with 30 layers and 1.3B parameters). A potential way to calibrate ò for new model is perform simple one-dimensional sweep over commonly used threshold range (e.g., a 8 E [0.1, 0.5] is recommended since it has shown effectiveness across most tested models) using a step size of 0.05 or 0.1. Furthermore, once the value of 6 determined. exhibits strong stability across different samples within the same model. Specifically, we sample 1K diverse prompts (the same evaluation set as the main paper) from the LAION-5B dataset and use Flux for image generation. The results demonstrate that DiCache not only maintains the lowest I.PIPS but also exhibits the lowest LPIPS variance compared to state-of-the-ar methods (as shown in Tab, 5) indicating its stable performance across diverse samples and strong generalization capability.
Table 5: Mean and Variance of LPIPS on Flux
Method
Mean (LPIPS)
Variance (LPIPS).
TeaCache ef
2025a
0.4053
0.010
TaylorSeer
Lou al. 20255
0.4709
0.007
CashC
Zhou et 2025
0,3019
0.008
DiCache (Ours)
0,2704
0.004
F MORE DISCUSSION ON DYNAMIC CACHE TRAJECTORY ALIGNMENT
In this section, we provide a deeper analysis on the design of Dynamic Cache Trajectory Alignment. As described in the main paper, considering the most recent cached model residuals rt. and rtg with corresponding timesteps l, < 4gg, we estimate the current model residual rt as:
Yt + (rt,.. - -tt)):
(12)
in which 7/ stands for the residual trajectory parameter which we dynamically obtained from Eq. 10
Actually, Eq. i2 in the form of first-order approximation (with respect directly using r Pt..
14

---

Published as a conference paper at ICLR 2026
Le. zero-order approximation adopted by previous works (Liu et al. 2025a Zhou et al. 2025 Since the model residual ry is a continuously varysng feature with a stable trajectory (as shown in Fig. 4 (a)), utilizing multi-order historical caches for estimation can effectively improve the accuracy of the approximation, as evidenced by the visual comparison results in Fig. 5 and the quantitative ablation results in Tab. 4 (c). Given the effectiveness of Eq. [ natural extension is to scale this approach to higher orders. Assuming the recomputation timesteps for a diffusion process form sequence to, /2, ty (4o < t2 < < tx), the k-th order Dynamic Cache Trajectory Alignment can be formulated as:
r2 = Tto +
Σ 7 5--
1 (Tio rr,),
(13)
1=1
in which rt; denotes the cached model residual at timestep 4 (i E [)) and & E [0 is a decay factor that controls the weights of different historical caches in the combination. Similarly, such relationship can also be established among k-th order probe residuals tt Th 122 rt with a probe depth m:
t/ 5i- (1% r
(14)
rm rm in which i represents the probe residual trajectory parameter. Since rio r2) r72. are already computed, f%x can be solved by:
, and r
L (r.7 rto
(15)
=1=1 5' (r rT)
Following the main paper, fe substituted into Eq. 13 replace Te::
2i=1 ci-1 (rr r%. (Tro - -r,))
(16)
E 51-1 (r r(,)
To evaluate the benefits of increasing the order of multi- cache utilizaaion,, we randomly sam- pled 500 prompts from the LAION-5B (Schuhmann et al., (022)) dataset and conducted quantitative ablation experiments on Flux (1024 x 1024 resolution). Similar to Section 4.3, experiments this section are conducted on single NVIDIA H200 140GB GPU. The decay factor & is set as 0.5. As can be observed in Tab, 6 the benefits of further increasing the cache utilization order is are relatively minor compared to the performance gain from zero-order to first-order approximation. In addition, storing features at more timesteps can impose a greater memory burden, especially for video gen- eration models with large number of frames (e.g., Hunyuan Video). Considering both simplicity and efficiency, we adopt the first-order approximation for Dynamic Cache Trajectory Alignment.
Table 6: Ablation study on the order of multi-step cache utilization in Dynamic Cache Trajec- tory Alignment. The performance gain is measured using the reduction ratio of LPIPS.
Model
Components
Values& & Choices
LPIPS
SSIM
PSNR
Performance Gain (w.rt.
k
0.2930
0.8166
21.83
0.00%
Flux
Cache Order &
k
0.2691
0.8245
22.43
8.16% 16%)
0.2686
0.8228
22.16
8.33% (+0.17%)
k 0,2710
0.8269
22.55
7.51% (-0.82%)
G LIMITATION AND FUTURE WORKS
Despite the advancements of DiCache in accelerating diffusion models, it faces certain constraints. Specifically, the two principal components of DiCache both rely on the online probe at runtime. Even though we have validated that probing shallow layers is sufficient (e.g., for the 57-layer Flux model, probe only the first layer with a probing rate of 1/57 0.018). the current probing paradigm still incurs a certain amount of time cost. Future works can focus on exploring ways to further reduce the probing cost, thereby more efficiently obtaining the output dynamics in the diffusion process.
In addition, we provide a comparison of the probing cost T Probe across different architectures, as well as the ratio of the probing cost the total inference time with DiCache (Tpoo//aal) as shown in Tab. / Since DiT models are typically deep while DiCache introduces only shallow-layer probing, the overall additional cost incurred by the probing operation is relatively minimal.
15

---

Published as a conference paper at ICLR 2026
Table Probing cost of different models.
Model
Tyutr (sec)
Tynde/ Ta
Flax
0.19
4%
WAN 2.1-1.38
3.8
5%
Honyuan
H ETHICAL STATEMENT
In this research, we affirm our commitment to upholding ethical standards in research and promoting responsible innovation. As far as aware, our study does not entail any data, methodologies, or applications that pose ethical issues. All experiments and analyses were carried out in accordance with established ethical guidelines, thereby ensuring the integrity and transparency of our research.
I REPRODUCIBILITY STATEMENT
To ensure full reproducibility of research and to contribute to the broader academic community, we will publicly release the source code of DiCache. At the current stage, code implementation of DiCache (for Hunyuan Video and Flux) is available in the zip archive included in the supplementary material. We hope these resources will provide a reference for future caching- based acceleration studies, thereby fostering innovation and accelerating progress within the community.
DECLARATION ON LLM USAGE
In this paper, we use LLMs only for minor language polishing.
K. SOURCE CODE AND DEMO VIDEO
We provide the code implementation (for Hunyuan Video and Flux) and demo video of DiCache for reference in the zip archive included in the supplementary material.
16

---

Published as a conference paper at ICLR 2026
Vanila 1,0% Spender
TeoClle 220x Spaslar
2.122
DiCath: (Ous)) (Ous)
2. 2.39 2,30x Specday
"A woman is holding stick front of group people with smoke fore the barkgraund
Zoomed In
LO0% Spoakap
Vaulla
TocCxtee 220x Speslep
EssyCooie 211Z Sposiar
DCCadic (Ours) 2.31%
"Eagle ls Flying Over The Largest City Of Moldova."
Zoomed
Viaiila 1R0x Specday
TeuCishs 2,20x Spantapp
ExcyCaudee 2.12-Speabin
DiCacho (Ouss) Spccdap
"A mun driving a sports car"
Zoomed
Figure 7: More qualitative comparison results on Hunyuan Video (1/2). Best viewed zoomed in.
17

---

Published as a conference paper at ICLR 2026
Yanila 1,00% Sponder
TooCllee 220x Sposlay
EaCCllll 2.122 Spealar
DiCath (Oun) 2. 2,39X Specday
"Alack and white car seratoles ittelf with foos.
Zoomed In
LO0% Spodaa
Yaalh
2200 Speslap
TocCyte
Ess Coulle 2122
DCCadic (Ours) 2.31%
3D anime huxster drives a go hart in 200."
Zoomed
Viniila 1RI0 Spectap
TeuCisde 2,20% Spantip
ExcCCaeee 2.12 2.12KSpeabin Spenkep
DiCacho (Ouss) 34 Spcclap
child purting topether puzzle
Zoomed
Figure 8: More qualitative comparison results on Hunyuan Video (2/2). Best viewed zoomed in.
18

---

Published as a conference paper at ICLR 2026
Vanils LO0K Spociay
TexClabe 220x Sposlep
ToyleSeer 3.63 Spociag
DKCadie (Dur)
2 65% Speotap
"Iron Mon, flies in the sky."
Vinilaa LEDX Spociaa
TeaCishs 2,20x Specile
Tockareesi 3.65% Speckdpp
DiCacle (Ours) 2 45% Speckap
'Gandalj relaxing on beach, luge trppccl hand, cinematic whot, lens
Vatilla 1,00x Spaaka
Teloalee 2 Speslap
Tofae x Sposlap
uLLK
DiCadhe (Dust 2.43 Syedaap Spodap
child standing tolking
Figure 9; More qualitative comparison results on WAN 2.1 1.3B. Best viewed zoomed in.
19

---

Published as a conference paper at ICLR 2026
Vanila 1.00 Sycckap
TeaCaati 1,83x Sposiip
DiCache (ftrr)
12x Spondou
"A fishing boat sailing on the sea with industrial plant the background.
Vanills 1.00 Spucilup
TarCsd 1.80% Sycciap
DiCade: (Chre))
2-22
"action moste of female hero with short hair vampire casile."
Visilta 1,000 Syccske
TraCacl 1.80 Speckap
DiCache (Dwrs) Sprelap
anine fight dragon from one
Vanilla 1,00 Speakep
TofCale 183k 1881 Spezlap
DiCache (Oun) 12. Spratop
tulking gostwing
Figure 10: More qualitative comparison results on WAN 2.1 14B, Best viewed zoomed in.
20

---

Published as a conference paper at ICLR 2026
TaylorSeur 3 3.13x Speclup
EnsyCache x Speedup
(Ours) 3.22x Speedup
1.00x Specdup
2.82 x Speedup
"Low-au Low-angle shot 2004 chryster 300 pulling to curb.
style, apartment which overlooks the and outside the window are palm trees, the city and the ocean.
"lebron beautiful goal in football match.
"Close up of a hamburger and potatoes plate."
"An angel with white wings sits in the background of a field flowers.
"Spider man travelling in assam a full flood simuation
"A mun and woman are sitting on a couch with dog front of television.
explodiudo..
"The person wearing square glasses, accompanied by garden dog.
"ancient fright text that glows purple light. harror like atmosphere. 2d Indie game vibe.
Ancient Fright
Aster Aviiet Denchayion!
Ancenca Steang
JNG hywsya
Ancient Fright
Figure 11: More qualitative comparison results on Flux (1/2). Best viewed zoomed in.
21

---

Published as a conference paper at ICLR 2026
Vanilla 1,00x Speedup
TeaCuche 2,82x Spectop
TaylorSeer 3.13% Spoctlp
EasyCache 2.49x Spoctlp
DiCache (Ours) 3. 3.2 Speedop
"a cinematic reolistic scene of boy wondering in the forest."
"Display the princess forming alliances with diverse group of characters.
"Show beautiful baby pigeon flodgllng perched a rouftop.
group of people taking pictures of yellow porsche car.
"logo with nōnja in it, a cinematic look, high resolation, blend of blue and white color."
"a short red fuzzy eyclops with a purple bandana around neck walking through the new mexico desert.
"fiddler the runf."
"a beautifid sleek black jaguar taking leap into the jungle."
"A robot is solving Rubik's cube."
15:23AM
15:28
16:29
15:23
"trading countdown of seconds.
Figure 12: More qualitative comparison results on Flux (2/2). Best viewed zoomed in.
22

---

Published as a conference paper at ICLR 2026
Seed
Vanill 1.00 1.00 Spedti
DICROe (Oir))
2.12% Speeoip
Seed 1
Vanilla 1200 Speecip
DIEWNe OOund 12X 2.12x Spedtig
Seed 2
Vauillo 1.80K Speedup
DICAAe (OUn))
2. 2.12N Spodlip
"A stylish woman walks donn Jokyo street filied with glowing and antmated coty signage. She vears black leather jacket, long nod dress, boots, and carvies black parse. wears snglasses red liptick She walks confidonnly counally. street danp and seflective, creating mirror effect of the colurful lights Many podestrians walk about"
Seed
Vanillo 1.00x Speedsp
DICade (CUn)) (Cun))
2. 12% Speetup
Seed 1
Vinila 5.000 Speetip
D&Cach (Omn) 2 112 Speedop
Seed 2
Vanill 1.00 1.00> Speedip
(IEWche((OOn))) 2. 1.22 Speeduu
"A cut walks on the graus realitti style"
Figure 13: Generated results using same prompts and different seeds WAN 2.1 14B.
23

---

Published as a conference paper at ICLR 2026
Vanill 1004 Spoclap
SNG 100% Spoodap
SYG SYGG DICKEN
2.12 Sposin
"viden clause fire drinking coffee nescafe.""
1000 Speodap
Vanilla
SYG 1.67% Spontap
-SVC INCadl
3.08 Spectap
linle girl louking Berkgolug with love and hug the Bedgoouugg the
Vesilla LOOK Syeodap
SNG 3477 Sposdap
*YY-- DiCadbe 08X Speakip
"A young mon tulking into microphone night.
Vasiia 1,00 Spoolap
SNG 1,07 L07x-Spslay Speadap
SVC DiCacke 3.0 8.00- Spectep
"A cate bitten playing a cat had."
Figure 14: DiCache can be integrated with the sparse attention method Sparse VideoGen.
24

---

Published as a conference paper at ICLR 2026
Vanilla
DCTA (DiCache)
web DCTA
Vanilla
w! DCTA (DiCache)
wio DCTA
Desi Preened Bone Textre
Study Dynamic
Trajectory Alignment
Vasilla
DCTA (DiCacle)
will DCTA
Alver ene recunlors large monu ray under manos"
Motion Preserved Detail Preserved
Motion Changed Degraded Detail
"Boautffdd giri watking is the sson..
Preserved Motion Preserved
Changed Motion Changed
Study Dynamic Trajectory Alignment
Figure 15: Additional ablation experiments on Dynamic Cache Trajectory Alignment (DCTA).
100X Spoaka
TiCaate (Denn) 1.880 Sporiag
A cute Pisachu emerges from mom pile.
Tisiled bAAN 1J00 Spookor
INCksbe (Owm)) 8.88 Spociap
The youngg monk is sitting and reading so scriptures
Deetiioe Www.AAAA 1,00 Sociaa
Daee INEVA dire 8.88% Spociaap
red sports our is speoding along read surrounded by trees.
Figure 16: Performance of DiCache on the distilled WAN 2.1 14B model.
25

---

Published as a conference paper at ICLR 2026
Table 8: The text prompts for each figure are listed sequentially, following the order from left to right and top to bottom.
Figure
Text Prompt
A teddy bear playing drum Times Square.
Harry Potter is talking to his classmate.
Figure
Christmas village, cute, snowing, nighttime, cinematic illumination, starry night
long cared cat with stripes walking and blinking towards the camera on
cowboy chopping a tree with an axe
Figure
bcautiful girl the gurden.
Sunset view of city with mountains in the background.
A dog walking through the streets of New Orleans.
a vidco game maintenance tochnician arranges a game on the counter. racoon cating mango on beach at the sunset.
Figure
young girl practicing yoga under large tree.
A
holding stick front group people with smoke and in the background.
Figure
Fagle Flying Over The Largest City Of Moldova.
A man is driving sports car.
black and white cal scratches itself with foot.
Figure
A 3D anime hamster drives a go-kart in 20o.
child putting together a puzzle.
in sky.
Figure 9
Gandall reluxing on beach, huge tropical drink hand, cinematic shol, 35 lens.
A child standing and talking to his three brothers.
A fishing buat is suiling on the seu with industrial plant in the background.
action movie of a female super hero with short hair in vampire castle.
Figure 10
anime fight between character goku from dragon ball and saitama from one punch man.
An
talking the cureera and gesturing with his hands.
Low-angle shot 2004 chrysler 300 pulling up curb.
miama retro style, apartment which overlooks the city and outside the window palm trees, the city and the ocean.
lebron james scores a beautiful goal in a football match.
Close up hamburger
potatoes on plate.
Figure
An angel with white wings sits in the background of a field of flowers.
Spider man travelling in assam in a full flood situation.
A man and woman are sitting on a blue couch with dog in front television. explodindo.
The person wearing square glasses, accompanied by garden dog.
ancient fright text that glows purple in a black light. horror like atmosphere. 2d Indie game vibe.
cinernatcc realistic scene of an boy wondering in the forest.
Display the princess forming alliances with a diverse group characters.
Show beautiful bahy pigeon fledgling perched on rooftop.
A group of people taking pictures of yellow porsche
logo with ninja in it, with a cinematic look, high resolution, blend of blue and white color.
Figure
short red fuzzy eyclops with purple bandana around neck walking through the new mexico desert.
fiddler the roof.
beautiful sleek black jaguar taking leap the jungle.
A robot is solving Rubik's cube.
trading countdown of 5 seconds.
A stylish woman walks down a Tokyo street filled with warm glowing neon and animated city signage. She wears a black leather jacket, a long red dress, and black bools, carries black purse, She wears sunglasses and red lipstick. She walks confidently and casually. The street damp and reflective, creating mirror effect of the colorful lights. Many pedestriuns walk about
Figure
A cat walks on the grass, realistic style.
video of santa clause tired drinking coffee nescafe.
little girl looking
hedgehog with love and trying to hug the hedgehog in the amn.
Figure 14
A young
talking into microphone night.
A cute kitten playing in cat bed.
lions running on the occan surface
Photo of a double beef cheeseburger.
Figure
diver encounters large manta ray under water.
Beautiful girl walking in the snow.
A Pikachu emerges
snow pile.
Figure 16
The young monk is sitting and reading scriptures.
A red sports car is speeding along raad surrounded by trees.
26


Reviews:
[{"review_id": "kKBVEUJ0N7", "reviewer": "Reviewer_bWWH", "summary": "This paper introduces DiCache, a training-free, adaptive caching strategy to accelerate diffusion models. DiCache addresses two core challenges in caching-based acceleration: \"when to cache\" and \"how to use cache.\" The method introduces two components: (1) an Online Probe Profiling Scheme to dynamically determine when to reuse cached outputs, and (2) Dynamic Cache Trajectory Alignment to improve multi-step feature reuse through shallow-to-deep trajectory consistency. Extensive experiments on WAN 2.1, HunyuanVideo, and Flux validate that DiCache achieves significant speedup while maintaining high visual fidelity.", "questions": "1.Given the memory and dynamic variability challenges, is there a possibility to automatically determine δ or adaptively calibrate it without per-model tuning?\n\n2.The probe is shallow but still executed at every timestep. Could the authors provide a detailed comparison of the probe cost across different architectures.\n\n3.The probe uses L1 relative distance on shallow features. Have alternative feature distance metrics been considered, especially for semantic coherence?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The motivation behind the proposed method is well articulated. The method's design is strongly supported by empirical evidence presented in the paper.\n\n* The method is completely training-free, making it highly practical and broadly applicable across different diffusion models.", "weaknesses": "* The proposed reuse threshold δ appears to require manual, per-model tuning (δ = 0.2 for WAN 2.1, δ = 0.1 for HunyuanVideo, δ = 0.4 for Flux), which may reduce generality and increase tuning effort for new architectures.\n\n* While the probe is “shallow,” it is still computed at every single timestep to accumulate caching error. It remains unclear how much speedup is offset by repeated probing on large backbones.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9WsNjqDUVF", "reviewer": "Reviewer_HkJV", "summary": "This work presents DiCache, a training-free, runtime-adaptive caching framework for diffusion models.\nIn DiCache, the Online Probe Profiling Scheme runs the first few layers to extract shallow probe features and estimate a per-sample caching error, which serves as the threshold for cache reuse.\nDynamic Cache Trajectory Alignment then uses the probe’s estimated progress to select and align deep features from nearby cached steps, reconstructing the current step without recomputing the heavy layers.\nAs a result, DiCache achieves high speedups with stronger fidelity across WAN 2.1, HunyuanVideo, and Flux.", "questions": "Mainly listed in the weakness. Below are the additional questions.\n\n1.\t**Reasoning on probing:** Explaination on why the first few layers can be so informative for predicting global caching error—e.g., what structural or representational property of DiTs enables such strong shallow–deep feature correlation would strenghten the paper.\n\n2.\t**Minor:** (i) Typo: missing space in “…probe feature trajectory,which…” (line 108). (ii) Naming inconsistency between “Online Probe Profiling Scheme” (main text) and “Online Probe Profiling Strategy” (Fig. 2 caption).", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "Novel idea on dynamic cache trajectory alignment that effectively and efficiently adapts the cached value to the current layer that reuses it.\n\ntraining-free and plug-and-play, requiring no model fine-tuning; it works at inference by wrapping around any DiT models.\n\nDiCache consistently achieves faster inference without sacrificing output quality, outperforming prior caching methods on both image and video diffusion models\n\nClear analysis and ablations with generally smooth writing and informative figures; easy to follow overall.", "weaknesses": "1.\t**Reliance on Threshold Hyperparameter:** Although DiCache demonstrates effective runtime caching under the reported experimental settings, the chosen probe depth (m) and accumulated caching error threshold (δ) should ideally generalize across different models. Alternatively, the authors could justify that the method’s effectiveness is not sensitive to these hyperparameters to substantiate the “calibration-free” claim. However, the current analysis of both hyperparameters lacks evidence of such generalization. For example, Spearman correlation analyses (e.g., Fig. 3 (d)) across multiple architectures would strengthen this point.\n2.\t**Hyperparameter Trade-offs:** There exists a strong trade-off among probe depth, reuse threshold, and achieved speedup, as well as between the reuse threshold and output accuracy. This implies that achieving the optimal quality–efficiency balance may require manual hyperparameter tuning, making the approach functionally similar to other calibration-based caching methods rather than being fully self-adaptive.\n3.\t**Overlap with Prior Works on Adaptive Caching and Probing:** DiCache’s adaptive cache decision, which is based on accumulated probe scores and thresholding, follows the same high-level “accumulate + threshold” mechanism used in TeaCache, and its distance-based error proxy conceptually resembles AdaCache’s feature-change metric. Furthermore, shallow-layer probing and caching is a common technique in transformer models [1, 2]. The authors should clarify the DiT-specific novelty in their cache-layer determination, beyond the incremental combination or adaptation of these existing ideas.\n4.\tEffectiveness of DCTA: Although DCTA is presented as a major contribution of this work, the breakdown of DiCache’s accuracy in Table 2 raises concern about its actual impact, as the observed improvement appears marginal. It would be helpful to clarify whether the effectiveness of DCTA, while seemingly limited in magnitude, remains consistent across different models.\n5.\t**Memory Usage Analysis:** The paper lacks quantitative analysis of memory usage, even though higher-order or multi-step caching inevitably trades off between memory consumption and accuracy. Reporting VRAM or feature-map memory profiles would clarify the practical scalability of the proposed method.\n\n[1] LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding, Mostafa Elhoushi, Akshat Shrivastava, et al., ACL 2024.\n\n[2] Reducing Transformer Key-Value Cache Size with Cross-Layer Attention, William Brandon, Mayank Mishra et al., NeurIPS 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Z2QptvpaEj", "reviewer": "Reviewer_MEuZ", "summary": "This paper proposes DiCache, a unified probe-driven framework that adaptively schedules and utilizes cache during diffusion inference, achieving efficient acceleration without additional training.", "questions": "See Weakness", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. The paper addresses the two fundamental challenges of cache-based acceleration through a unified probe-driven framework, which reduces reliance on empirical heuristics and offline calibration.\n\n2. DiCache can be further combined with Sparse VideoGen to achieve additional acceleration, demonstrating its complementarity with sparse attention techniques.\n\n3. The authors empirically observe a strong correlation between shallow-layer feature differences and deep-layer residuals, and find that features across different DiT blocks exhibit similar trajectories during the sampling process, providing the foundation for the probe-based error estimation and trajectory-based cache blending.", "weaknesses": "1. The coverage of baselines is somewhat limited. Although the experimental tables include TeaCache, EasyCache, TaylorSeer, and ToCa, the Related Work section also discusses other comparable methods such as FasterCache, FORA, and Δ-DiT. Incorporating these methods into the quantitative comparison tables would make the empirical positioning of the proposed approach more complete.\n\n2. The paper primarily evaluates performance using similarity metrics (LPIPS, SSIM, and PSNR) with respect to the outputs of the original model, along with the inference speedup ratio. However, it provides limited analysis of broader perceptual quality or downstream task metrics. Since DiCache does not demonstrate particularly strong fidelity compared to the base model, including perceptual quality evaluations such as VBench scores and user studies would be necessary to provide a more comprehensive assessment.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "otuCVk8VnC", "reviewer": "Reviewer_1sUf", "summary": "This paper proposes a caching mechanism for Diffusion Transformers (DiT). The core idea is to use shallow-layer residual probing to estimate deep-layer feature caching. The author verified DiCache on both text-to-image and text-to-video tasks and obtained superior results compared with the baseline method. In summary, I would say this work introduces a simple yet effective DiT caching method that is easy to use for the community. However, more experimental analysis is expected to make the work more solid and convincing.", "questions": "1. Since Wan2.1 and HunyuanVideo are tested, I suggest that the authors provide VBench results on the t2v task. Evaluating generated video quality using image-oriented metrics is not convincing enough.\n2. Compared with baseline methods such as TaylorSeer, DiCache uses fewer text-to-image metrics, making the evaluation not comprehensive enough. For example, how does DiCache affect the text-image alignment?\n3. Since m is set to 1 for all experiments and the author claimed $m\\in [1,2,3] $ is sufficient, I suggest that the author show corresponding diagrams in Figures 3 and 4, instead of only showing m=5.\n4.  What is the recomputation rate under different parameter settings? In other words, is $\\delta$ hard to tune for different DiT models? It seems this value differs across the DiT models tested.\n5. For DCTA, does it mean the current residual is continuously computed from the two most recent recomputed timesteps? How much additional overhead (in terms of FLOPS and latency) will this cause?\n6. How does DiCache perform on distilled models?\n7. What's the source of text-to-video prompts?\n\nI will raise my score when concerns are mostly addressed.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. The motivation of this paper is clear. Empirical findings are quite convincing and interesting.\n2. This method is easy to use and can be integrated with other intra-step diffusion acceleration methods (such as the sparse-attention-based method). \n3. On one hand, the speedup is non-trivial which is especially useful for video diffusion model acceleration. On the other hand, the quality loss is marginal and it can maintain high consistency with the vanilla samples.", "weaknesses": "The main weakness of this paper lies in the presentation and insufficient experimental analysis. Please see the questions below.\n\nSome advice on the presentation:\n1. I strongly recommend redesigning Figure 2 as it increases the understanding cost.\n2. Axes numbers in Figure 3 are too small.\n3. Abbreviations are better introduced at the first time they appear, such as DCTA.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)