Relational Feature Caching for Accelerating Diffusion Transformers
Authors: Byunggwan Son
Jeimin Jeon
Jeongwoo Choi
Bumsub Ham
Venue: ICLR 2026 Poster (2026)
Keywords: Diffusion transformer
Feature Caching
Primary area: generative models

Abstract: Feature caching approaches accelerate diffusion transformers (DiTs) by storing the output features of computationally expensive modules at certain timesteps, and exploiting them for subsequent steps to reduce redundant computations. Recent forecasting-based caching approaches employ temporal extrapolation techniques to approximate the output features with cached ones. Although effective, relying exclusively on temporal extrapolation still suffers from significant prediction errors, leading to performance degradation. Through a detailed analysis, we find that 1) these errors stem from the irregular magnitude of changes in the output features, and 2) an input feature of a module is strongly correlated with the corresponding output. Based on this, we propose relational feature caching (RFC), a novel framework that leverages the input-output relationship to enhance the accuracy of the feature prediction. Specifically, we introduce relational feature estimation (RFE) to estimate the magnitude of changes in the output features from the inputs, enabling more accurate feature predictions. We also present relational cache scheduling (RCS), which estimates the prediction errors using the input features and performs full computations only when the errors are expected to be substantial. Extensive experiments across various DiT models demonstrate that RFC consistently outperforms prior approaches significantly. Project page is available at https://cvlab.yonsei.ac.kr/projects/RFC

Published as a conference paper at ICLR 2026
RELATIONAL FEATURE CACHING FOR ACCELERATING DIFFUSION TRANSFORMERS
Byunggwan Son 1+ Jeimin Jeon
Jeongwoo Choi
Bumsub Ham 1:2f
Yonsei University
Korea Institute of Science and Technology (KIST)
[byunggwan.som son, jeimin, jeongwoo. choi, bumsub. htt.uu..aamyyyosee.... ac. kr
ABSTRACT
Feature caching approaches accelerate diffusion transformers (DiTs) by storing the output features computationally expensive modules at certain timesteps, and exploiting them for subsequent steps reduce redundant computations. Re- cent forecasting- based caching approaches employ temporal extrapolation tech- niques to approximate the output features with cached ones. Although effective, relying exclusively on temporal extrapolation still suffers from significant pre- diction errors, leading performance degradation. Through detailed analysis, we find that I) these errors stem from the irregular magnitude of changes in the output features, and the corresponding output. Based on this, we propose relational feature caching (RFC), novel framework that leverages the input-output relationship to enhance the accuracy of the feature prediction. Specifically, we introduce relational fea- ture estimation (RFE) to estimate the magnitude of changes in the output features from the inputs, enabling more accurate feature predictions. We also present re- lational cache scheduling (RCS), which estimates the prediction errors using the input features and performs full computations only when the errors expected to be substantial. Extensive experiments across various DiT models demonstrate that RFC consistently outperforms prior approaches significantly. Project page is available at
input feature of module is strongly correlated with
INTRODUCTION
Diffusion models (Ho et al. 2020 Song & Ermon 2019) have recently achieved remarkable success in generative tasks, such as text-to-image generation (Rombach et al. [022 Sahariia et al.) (2022), and video generation (Blattmann al. 2023alb) b). While early approaches (Ho et al.) 2020)] Dhariwal & Nichol 2021) relied on U-Net architectures (Ronneberger et al.) 22015), recent works have shifted to- ward diffusion transformers (DiTs) (Peebles Xie (023), which demonstrate superior performance, particularly when model and data scales increase. However, such gains come with significant com- putational costs, since DiTs require performing expensive forward passes over numerous denoising timesteps, which hinders their practical application. To address this, feature caching approaches (Ma al. 2024| Wimbauer et al.) 2024 Liu et al. 2025b) have emerged, offering promising solution for reducing redundant computation in DITs. These methods are based on the observation that the intermediate features in DiTs are highly similar across adjacent timesteps. Specifically, they perform full computations at certain timesteps, cache the output features of computationally expensive mod- ules (e.g., attention and MLP), and exploit the cached features at subsequent timesteps to reduce redundant computations.
Early caching approaches (Ma et al. 2024 Selvaraju et al. 2024) typically reuse cached features di- rectly without adaptation. While this improves efficiency, the discrepancy between cached and fully computed features accumulates over timesteps, which turn degrades the generation quality. To alleviate this, recent works have proposed forecasting- based methods, that predict features through
Equual contribution.
Correspondence author.

---

Published as a conference paper at ICLR 2026
0.6
0.5
Relative L, Brror
SPD
10
0.3
0.2
POSA
0.3
7eleter (tnd)
20
20
40
Timestep
Timestep
Modall Index
FLOPs (T)
(a) Output difference.
(b) Input difference.
(c) Prediction emror.
(d) Performance.
Figure 1: Feature analysis and comparison between existing approaches (FORA (Selvaraju et al. 2024). Tay- lorSeer (Liu et al. 2025b)), and our method (RFC) using DiT-XL/X (Peebles & Xię 2023). (a-b) Min-max normalized Ly distances of output and input features, measured between consecutive timesteps. While the vari- ations of feature changes irregular, those of input and output remain closely aligned with each other. (c) The prediction errors across different modules. We measure the relative I. error hetween output features with and without applying caching methods and average the values over the timesteps. (d) Quantitative results on ImageNet (Deng al. 2009) evaluated in terms of FLOPs and sFID (Nash et al. 2021.
temporal extrapolation techniques, with an assumption that features evolve smoothly over timesteps.
For instance, FasterCache (I.v al. 2025) and GOC (Qiu et al. 2025) perform linear extrapola- tion technique based on features from the two most recent full-compute steps, while TaylorSeer (Liu al. 2025b) leverages the Taylor expansion to estimate feature changes along timesteps. However, our observations reveal that the magnitude of changes in the output features differs significantly across timesteps (Fig. [()), leading to substantial prediction errors (Fig. [T)). Specifically, Tay- lorSeer achieves slight reductions in prediction еттоr compared to FORA (Selvaraju et al.) (2024). which reuses features without adaptation. This degrades the generation quality severely, especially when employing large temporal intervals between full computations (i.e., lower FLOPs in Fig. (d).
To address this limitation, we propose relational feature eaching (RFC), a simple yet effective frame- work that leverages the relationship between the input and output features of modules (e.g., attention and MLP), rather than relying solely on temporal extrapolation techniques. To this end, we propose relational feature estimation (RFE), which estimates the magnitude of changes in the output feature by exploiting the differences in input features. RFE is based on the observation that the magnitude of changes in the output features is highly correlated with that of the input features (Figs. T [[-b)), hancing the accuracy of the feature estimation. We also introduce relational cache scheduling (RCS), dynamic caching strategy that determines when perform full computations based estimated output errors. Since directly measuring output errors requires full computations, we instead employ errors in the input prediction as an efficient proxy, leveraging the relationship between the input and output features. Extensive experiments show that our proposed RFC consistently outperforms previous caching methods across variety DiT models, demonstrating the effectiveness of our method. We summarize our contributions as follows:
We propose RFE, forecasting method that leverages input feature variations to more ac- curately estimate output features.
We propose RCS, a dynamic strategy that performs full computations adaptively by esti- mating the prediction error of the output features from that of input features.
Extensive experiments across various DiT models demonstrate that using RFE and RCS consistently outperforms existing caching methods in terms of both the generation quality and computational efficiency.
2
RELATED WORK
In this section, we review representative works related to accelcrating diffusion models.
2.1 TIMESTEP REDUCTION
The iterative generation process of diffusion models (Ho et al. 2020 Song & Ermon 2019) leads to a slow inference and high computational costs, as each sample requires hundreds or even thou- sands of denoising steps. To mitigate this, many works attempt reduce the number of timesteps.
2

---

Published as a conference paper at ICLR 2026
introduces non-Markovian deterministic sampler that preserves the DDPM Ho et al. (020) training objective, while skipping intermediate timesteps. DPM-Solve (Lu et al.. 2022 extends this idea with higher-order ODE solvers, approximating nonlinear terms through the Taylor expansion. While these approaches achieve a faster sampling, their performance degrades significantly when the number of steps becomes extremely small.
DDIM (Song al. 2020)
Another group of methods distill a pretrained diffusion model into a student model that generates comparable results with only few denoising steps. Progressive distillation (Salimans & Ho 2022) iteratively trains a student model, which then becomes the teacher for the next stage with fewer steps. Consistency models (Song et al. 2023) learn direct mappings from noisy images at arbitrary timesteps to noise-free free images, enabling skipping intermediate denoising steps. Although these ap- proaches generate high-quality images with limited timesteps, they typically incur a significant train- ing overhead.
2.2 FEATURE CACHING
Feature caching methods aim to reduce the computational cost across timesteps by storing fully computed features at certain timesteps and exploiting them for subsequent steps avoid redundant computations. Early approaches (Ma et al. 2024 Selvaraju et al. 2024) So et al. (024) use uniform intervals for full computations, but this ignores the fact that cache errors vary significantly across timesteps. To address this, CacheMe (Wimbauer et al. 2024) sets schedule for the full computa- tion before the denoising process based on the cache error, but this predetermined schedule does not consider varying cache eтrors across samples. To mitigate this, TeaCache (Liu et al. (025a) caches input features at full-com full-compute steps and adaptively triggers full computations when the current input feature deviates significantly from the last cached feature. It however requires an additional calibra- tion step to predict output cache errors from input differences. Furthermore, all the aforementioned caching approaches reuse the cached features directly without adaptation, which can cause sub- stantial cache errors, particularly under long intervals between successive full computations. The works of (Zou et al. 2225 2024) aim to reduce cache errors of important tokens by selectively per- forming full computations on them, while still relying on cached features for the majority of other tokens.
More recent works address the cache error by predicting features from cached ones rather than reusing them directly. FasterCache (Lv et al. 2025) observes that the change in the output features directionally consistent, and uses a linear extrapolation technique to predict subsequent features. GOC (Qiu al. (025) also adopts a linear extrapolation technique, while reusing the cached fea- tures for timesteps where the extrapolation is less accurate, TaylorSeer (Liu et al. 2025b) leverages the higher-order Taylor expansions to better capture nonlinear feature dynamics, providing more accurate predictions than the linear extrapolation and improving the generation quality.
3
METHOD
In this section, we describe diffusion models and feature caching in DiTs (Sec. 3.1) We then intro- duce our RFC framework in detail (Sec. 3.2
3.1 PRELIMINARIES
Diffusion models. Diffusion models consist of forward and a reverse process. In the forward process, Gaussian noise e, sampled from a normal distribution ((0 1), added to the input image To sequence of timesteps. Specifically, given timestep t, the noisy image x; is defined as:
Var.o
ozc,
6~ I,,,
(1)
where 0r denotes a predefined noise schedule timestep t. The reverse process recovers the original image zo from the noisy input by iteratively denoising it through series of steps. At each timestep the model predicts the noise component, which is then used to compute the denoised image for the previous timestep It-1. For instance, DDIM sampler (Song et al.) 2020) defines the reverse process as follows:
F1
VI aico(ze,t) + /I (1-160(2
It-1 = Vai-1
(2)
C2
3

---

Published as a conference paper at ICLR 2026
10
10th module
Relative L. Error
0.6
6X9
20th module
30th module
RSD (%)
00.4
0.2
0.0
50
40
30
20
10
50
40
30
20
16
Timestep
Timestep
Variation of 8k-
(b) Relation between &o and El-
Figure 2: Empirical analyses in DIT-XL/2 (Peebles & Xie 2023 (a) RSD of &u (t - k) with varying t. (b) Relative L1 errors of the output and input features, i.e., Eo (t) and (t), respectively, Please see the text for details.
where is the noise prediction model parameterized by 0. While early diffusion models (Ho al., 2020 Dhariwal & Nichol 202T) adopt U-Net architectures (Ronneberger et al.) 2013), DiTs (Pee- bles & Xie (223) instead parameterize εş with transformers. DiTs have been shown to be effective for lcage generation tasks, but at the cost of a substantial computational overhead.
TaylorSeer (Liu et al. 2025b). TaylorSeer reduces cache errors in diffusion transformers by pre- dicting output features through the Taylor expansion around the last fully computed timestep I. Given a cache interval N, the m-th order prediction of the output feature for the 1-th module timestep t k (0 < k < N) is computed as follows:
m K2 O')(() (
O' - =0
N
where (t) is the output feature of the I-th module at timestep t, and A denotes the i-th order finite difference operator with the stride N defined recursively as:
(1) - o' A -
O'((t) (0'0) 0 O'+ (t+
i i 1, i> 1.
(4)
For the clarity and convenience of notation, we omit the superscript i for first-order differences and the module index I in the remainder of this paper.
3.2 RELATIONAL FEATURE CACHING
While TaylorSeer (Liu et al. 20256) achieves the state-of-the-art performance over previous caching approaches, still shows considerable prediction errors (Fig. Tc). This is because the magnitude of the feature changes is irregular across the timesteps (Fig. Ia)), making it difficult to accurately estimate the subsequent features based only on temporal extrapolation techniques. To this end, we propose RFC, a framework that enhances feature prediction by leveraging the relationship between input and output features through two complementary components, RFE and RCS. RFE estimates the magnitude of changes in the output features, while RCS identifies when predictions become unreliable and performs full computation accordingly.
Relational feature estimation (RFE). We have observed in Fig. aa-b) that the magnitudes of changes in the input and output features for the same module are highly correlated, suggesting that differences in input features can serve effective predictors of the output variations. To quantify this relationship, we define the ratio between the magnitudes of changes in output and input features as follows:
Su k) | -
(5)
T 1 k)ll2
where k) and [(t-k) k) denote the output and input features of the same module, respectively, at timestep t -- k. and A is the first-order difference with stride & in Eq. 4 To evaluate the consistency of the ratio, we compute st k) for & [1, 9). and measure the relative standard deviation (RSD)
4

---

Published as a conference paper at ICLR 2026
over the resulting values. We can from Fig. 2[a) that the RSD values remain consistently low (typically around 2%), indicating that s (t k) is highly consistent across timesteps. To support the empirical consistency of sx(t k), present the following proposition:
Proposition 1. Assume that the mapping from input to output features is locally linear, and the direction of the difference vector It k) remains constant for 1 S k S N, where N is an interval between full computations. Then, the ratio sy(1 - k) is approximately invariant w.r.t
Proof. Suppose the output feature (tt) is a locally linear to the input feature I(t), i.e.,
( Al(t)
(6)
where and b denote the weight matrix and bias vector of the linear transformation, respectively. Then, the change in output features over the temporal offset is can be written
AtO(t k) = O k) - Ot))
= A(I(t - k) I(t))
(8)
AA&I(t k).
(9)
Taking the L2-norm of both sides gives:
|| - k)ll2 A [(---(((A k)ll2
(10)
Auuu k))l2llA k))l2.
(11)
where ut - 1:) is the normalized direction of the input change defined as:
Ax k)
ux.tt k) 1 k)ll2
(12)
By dividing both sides of Eq. with T (t. k)) 22. we have the following:
At, k)ll2 | k)ll2.
(13)
[x k)ll2
By the assumption that ut(t w.r.t. k.
k) remains constant for 1 < k S N, the ratio sy(t - k) is invariant
Note that the two assumptions in Proposition T are commonly observed in diffusion models. First, changes in input features usually small (Ma et al. 2024| Selvaraju et al. [0244] So al., 2024). which allows for the output features to be locally approximated using Taylor's 's theorem. Second, the
works of (Lv et al. 2025 Qiu et al., 2025) show that the direction of feature changes remains largely consistent across timesteps. We provide empirical validation of the assumptions in Sec. A
Based on the above finding, we propose RFE, simple yet effective method that estimates the mag- nitude of changes in output features O k)) l|2 by leveraging the changes in input features || 2-) Specifically, we can rewrite Eq. as follows:
|| - 12 = 8 (t It k)112-
(14)
Given the consistency of st k) across timesteps, we can approximate Eq. (4p as follows: 1t 1) (t)) k)) l
(15)
where v (t) is the ratio computed between the two most recent full computations with the interval of N. Note that obtaining an input feature T(t k)) is efficient, since it only requires lightweight
operations (e.g., LayerNorm (Ba et al. 2016), scaling, and shifting). Consequently, RFE refines the
predicted magnitude of feature changes in Eq. (3) follows:
k ANO()) Σ
m
ORFE(I- - k) = 0() + (sx T(L k))l2)g
(16)
N
where g(-) is La normalization function as follows:
g xr) ||0||2
(17)
RFE can capture irregular dynamics of feature changes, reducing the error of feature prediction, efficiently and effectively.
5

---

Published as a conference paper at ICLR 2026
Relational cache scheduling (RCS). Although RFE significanlly improves the accuracy of the feature prediction, estimation errors are unavoidable and tend to fluctuate over timesteps, making fixed caching interval N suboptimal. To address this, we propose RCS, dynamic scheduling strategy that determines when to perform full computation based on the estimated prediction error. To this end, we define the error of the output prediction timestep t k follows:
Eo k) O(t - k) - OREe(t - k).
(18)
However, computing Eo k) is not feasible during sampling, since it depends on ((t -). which requires costly computation (e.g., attention MLP). Instead, propose to estimate this error using the error of the input prediction (t k) as proxy:
Et k) I[t k) Itayoor -k),
(19)
where the predicted input feature Taylor k) is computed with the Taylor expansion as follows
Σ ky (t)(t)
Taylor (t- k) I(1++
(20)
1 N
Our intuition is that prediction errors in the output tend to increase when the feature changes abruptly, and such changes are highly correlated with the variations the corresponding in- put (Figs. aaa and 2aa)). Therefore, errors in the output prediction are likely to be correlated with those in the input prediction. To support this, we compare the relative L2 error of the output and input features, which is defined follows:
Eo(1-A) - k:) Eo(t - k)ll
= |E - -k]l T k)]l2
(21)
10 k)]l1"
As shown in Fig. b), the trends the input and output errors closely align across timesteps. Based on this, aim to estimate the accumulated errors in the output prediction during the sampling process by tracking the relative L1 error of the input prediction for the first module. We then perform full computation when the accumulated emrrr exceeds predefined threshold follows:
E
(22)
The left term Eq. (22) effectively reflects the accumulation of errors in the output prediction. By adjusting RCS controls the trade-off between the generation quality and efficiency. RCS performs full computations more frequently at timesteps, where the accumulated errors are large, reducing the cache error and improving the generation quality.
4 EXPERIMENTS
In this section, we first describe our implementation details (Sec. 4.1 We then provide quantitative and qualitative comparisons of RFC with state-of-the-art methods (Sec. 4.2), and present an in-depth analysis of RFC (Sec. (43).. More experimental results and a discussion on the limitation of our work can be found in Appendix.
4.1 IMPLEMENTATION DETAILS
Datasets and models. We apply RFC various DiT models and perform extensive experiments on standard benchmarks for the class cans-condiiional, text-to-imane image, and text-to-video generation tasks. For classass-ee coaasscondiiinal generation, we perform experiments using the DiT-XL/2 (Peebles & Xie (023) model trained on ImageNet (Deng et al. (209). We use FLUXX.1 dev (Batifol et a (025) for text-to- image generation and Hunyuan Video (Kong et al. 2024) for text-to- video generation, and evaluate them DrawBendd (Saharia et al. 2022) and VBench (Huang et al. 2024), respectively. We use the DDIM sampler (Song et al. 2020) with 50 steps for DiT-XL/2, while using the Rectified Flow sampler (Liu et al. 2023) for F dev and Hunyuan Video with the same number of steps.
RRE not applied to input prediction, as it is defined exploit oututt relationships.

---

Published as a conference paper at ICLR 2026
Table 1: Quantitative comparisons on class-conditional image generation for DiT-XL/2 (Peebles & Xie, 2023) on ImageNet (Deng et al. (2009). Numbers in bold indicate the best performance, while underscored ones are the second best among similar computational costs. N is the interval between full computations, and refers to the order of the Taylor expansion.
Methods
NFC FLOPs (T) Latency (s))
FID,
sFID,
FIDEFCL sFID2FC
1St
Full- Cull-Cunpute
50
23.74
7.61
2.32
4.32
241.25
FORA (N 4)
(Selvarnju al.
2024
14
6.66
2.37
4.33
7.38
1.90
6.32
215.33
ToCa (N = 4)
Zou et c.m 2023
17
8.73
3.64
3.03
5.02
0.86
3.56
229.01
DuCa (N 1)
Zou al. 2024
7.66
2.94
3,39
4.95
1.19
4.22
224.03
TaylorScer (N
2025b
14
6.66
2.84
2.55
5.30
0.44
2.17
232.26
RFC (m
14.01
6.67
2.86
2.52
4.60
0.30
1.33
231,00
FORA (N 5)
(Selvarnju 2024
11
5.24
2.03
6.12
10.51
3.57
10.19
196.04
ToCa (N = 5)
Zou et al.. 2025
14
7.43
3.35
3.53
5.36
1.28
4,50
224.07
DuCa (.V 5)
Zou al. 2024
6.32
2.57
6.02
6.71
4.19
8.18
196.96
TaylorSeer (N
2025b
11
5.24
2.52
2.70
5:45
0.56
2.74
228.76
RFC (m 2)
11.01
5.24
2.52
2.71
4,80
0.48
2.14
227.72
FORA (N
Selvaraju 2024
10
4.76
1.82
8.12
12.94
5.32
12.87
179.11
ToCa (N = 6) Zou al. 2025
13
7.01
3.27
3.54
5.54
1.44
5.18
226.32
DuCa
al..
7.86
TaylorSeer (N
2025b
10
4.76
2.27
3.10
6.47
1.00
4.63
222.85
RFC (m 2)
10.04
4.78
2.46
2,75
5.02
0.54
2.43
226.93
FORA (N 7) (Selvaraju 2024
8
3.35
1.73
12.63
18.49
9.66
19.11
147.06
ToCa (N = 7)
Zou et al. 2025
14
6.25
2.90
4.04
5.72
1.62
4.72
214.25
DuCa 7)
7ou
Toreor (N
2025b
8
3.82
2.02
3.46
6.97
1.30
5.61
217.18
RFC (m 2)
8.02
3.83
2.15
3.12
5.07
0.81
3.10
219.20
TaylorSeer (N
2) Liu
2025b
7
3.35
1.89
4,90
7.92
2.33
7.35
198.46
RFC (m 2)
7.04
3.37
1.99
3.40
5.21
1.03
3.66
215.39
Evaluation and metric. Following the previous approaches Zou et al. 2025 Liu al. 2025b), we evaluate 50K images of size 256 256 for class-conditional generation using standard metrics, including Fréchet Inception Distance (FID) (Heusel et al. (017). sFID (Nash et al. 2021). and In- ception Score (IS) (Salimans et al., 2016). In addition, we introduce FID2FC and sHI22FC, which quantify the degradation introduced by caching through measuring the FID and sFID scores, spectively, between the images obtained from full computations and feature caching methods. For the text-to-image generation task, we generate 200 images of size 1000x 1000 from Drawheench prompts (Saharia et al.] 2022) and evaluate the image quality and text-to-image aligmment using ImwgeReward (Xu et 2023) and the CLIP score (Hessel al. (021). respectively. We also eval- uate PSNR, SSIM (Wang et al., 2004). and LPIPS (Zhang et al. 2018) computed between images generated by full computations and caching approaches. For text-to-video generation, we produce 2,838 videos by generating three videos for each of the 946 prompts. We evaluate the generated videos with the VBench score (Huang et al. 2024), as well as PSNR, SSIM, and LPIPS. For all tasks, we report FLOPs and latency evaluate the computational efficiency. For fair comparison, we reproduce the results of state-of oftte-of-the-ar methods using the official source codes, and adjust the threshold 7 in Eq. (22) to ensure that the average number of full computations (NFC) matches that of other methods.
4.2 RESULTS
Quantitative results. We show in Tables 1 quantitative comparisons of RFC and state-of ohe art methods (Selvaraju et al. 2024 Zou et al. 2025; 2024 Liu et al. 2025b) for class-condition image, text-to-image,
text-to-video generation tasks, respectively. We summarize our findings as follows: (1) RFC outperforms existing caching approaches across various generative tasks, by a large margin. For example, we can see from Table [ that RFC with 3.37 TFLOPs outperforms TaylorSeer (Liu et al. 2025b) with 4.76 TFLOPs by 1.26 in terms of sFID. We can also see that the improvement significant for metrics that compare the quality of generated images from full computations and caching approaches (e.g., FID2FC, sFID2FC, PSNR, SSIM (Wang et al. 2004), and LPIPS (Zhang et al. (2018), suggesting that RFC can accurately estimate output features by leveraging the relationship with the corresponding input features. (2) RFC performs particularly
7

---

Published as a conference paper at ICLR 2026
Table Quantitative comparisons on text-to-image generation for FLUX.1 dev (Batifol al. 2025)
on DrawBench (Saharia et al. 2022). Numbers in bold indicate the best performance, while under- scored ones are the second best among similar computational costs. N the interval between full
computations, and in refers to the order of the Taylor expansion.
Methods
NFC FLOPs (T)
Latency (s)
PSNR* SSIM LPIPS!
IRT
CLIP
Full-Compute
50
2813.50
26.210
0,9655
17.0720
FORA (NV 3)
(Selvrraju al. 2024
957.33
10.489
17,4967
0.7236
0,4209
0,9111
16.9515
ToCa (N 5)
Zou 2025
14
1056.26
18.591
18.0333
0.7269
0.3490
0.9457
17.0063
TaylorSeer (N
1, m
Liu et
2025b
788. .9
8.863
19,5722
0.7695
0.3226
0,9204
17.0689
TaylorSeer (N
4, m
788.59
8,985
19.7733
0,907
17,0527
RFC (m 1)
14.02
789.82
9.148
20.2707
0.7905
0.2934
0,9448
17.0288
RFC (m- 2)
13.80
777.44
9.364
20.3524
0.7930
0.2952
0.9499
17.0394
FORA (N 7) Selvaraju 2024
451.10
6.065
15.8656
0.6692
0.5557
0.6121
16.8531
ToCa (N = 12) Zou et al
673.88
11.504
16,2240
0.6479
0,5512
0.6238
16.9020
TaylorSeer (N 9, m
Liu et
2025h
451.10
6.382
1346
0.6736
0.5261
0.7602
17.0769
TaylorSeer (N 9,
451.10
6.796
16.5492
0.6563
0.5328
0.7996
12.0534
RFC (m- 1)
8.00
451.23
6.407
16. 7952
0.6979
0.4645
0,9142
17.0165
RFC (m 2)
8.03
452.91
6.849
16.9188
0.6936
0.4708
0.9189
16.9641
Table 3: Quantitative comparisons on text-to-video generation for Hunyuan Video (Kong et al. on VBench (Huang al., 2024). Numbers in bold indicate the best performance among similar computational costs. is the interval between full computations, and m refers to the order of the Taylor expansion.
p024
Methods
NFC FLOPs (T)
Latency (6)
PSNRT SSIM LPIPS VBencht
Full- Call-Compute
50
7520.00
263.480
81.40
FORA (N 6)
Selvaraju al. 2024
9
1359.19
55,339
15.5221
0.4349
0.2743
78.51
TaylorSeer (N
et
20256
10
1510.21
60.051
16.3606
0.5126
0.1985
80.77
TaylorSeer (N 6, m 1)
9
1359.19
55.872
15.5321
0.4606
0.2448
79.52
RFC (m 1)
8.96
1354.65
57.786
18,5408
0.6352
0.1329
80.83
FORA (N 8) Selvaraju al. 2024
7
1058.45
48.340
14,8878
0.4114
0,2959
77.50
TaylorSeer (Liu et RFC (m 1)
1)
7
1058.45
48.736
15. 1999
0.4408
0,2624
79.59
7.09
1072.65
49.684
18.2521
0.6155
0.1436
80.49
well under limited computational budgets. In such cases, prior approaches suffer from large errors in feature prediction, mainly due to irregular changes of features across timesteps, which degrades the performance. Even with limited computation, RFC accurately estimates the output features with RFE and effectively performs full computations with RCS, preserving the generation quality. (3) With similar NFC, the overhead of RFC compared TaylorSeer (Liu al.) (0255) is minimal. This is because computing the input features in RFC only requires lightweight operations such as LayerNorm (Ba et al. 2016), shifting, and scaling, which adds negligible computational costs, while improving the generation quality significantly.
Qualitative results. We show in Fig. 3 the qualitative comparisons of images generated using DIT-XL/2 (Peebles & Xie, (023) and FT.UX.1 1 dev (Batifol et al., 2025). We observe that RFC produces more realistic images that are visually closer to those generated by full computations. For example, in the first row Fig. righht, RFC better preserves the brick structure compared to the prior approaches (Selvaraju al. 2024 Liu et al., 2025b). closely matching the image generated by full computations, indicating that RFC effectively minimizes the prediction errors in the output features. We also show in Fig. generated videos using Hunyuan Video (Kong et al.) (024). We can see that RFC produces high-quality videos that are similar to those generated by full computations, demonstrating the effectiveness of RFC once more. Please Appendix for more results.
Video
4.3 DISCUSSION
Ablations. We show in Table 4 the ablation study for each component of RFC. We can that applying either RFE or RCS alone already outperforms TaylorSeer (Liu et al. 2025b) regardless of NFC. For instance, when NFC is 11, applying RFE and RCS to TaylorSeer reduces the sFII)
8

---

Published as a conference paper at ICLR 2026
PORK F.A
RPC 6227
$0.0
10000
+822
cude
poica hat green greas
Apuecader
Acido den clin Bald.
Figure 3: Qualitative comparisons of (left) consiconni image generation for DiT-XL/2 (Peebles & Xie 2023) on ImageNet (Deng et al. 2009), and (right) text-to-image generation for FLUX.I dev (Batifol et al. 2025) on Draw Bench Saharia et al. 2022).
Methods
ecks shilustton keny olur
ahae carning a
Tul-Compcle 1.00
FORA x
TralodSerr
RPG
Figure 4: Qualitative comparisons of text-to-video generation for Hunyuan Video (Kong et al. 2024) on VBench (Huang al. 2024). Please the supplementary material for the actual video clips.
from 5.85 to 5.22 and 5.21, respectively. This indicates that both RFE and RCS can effectively improve feature prediction by leveraging the input-output feature relationship. Moreover, combining both components (i.e.. RFC) further reduces the sFID to 4.88, suggesting that RFE and RCS are complementary each other.
Relational feature estimation (RFE). Feature forecasting methods based on linear extrapolation techniques (Lv al. 2025 Qiu et al. 2025) can be formulated as:
& N w (t)ANN((t)
Opea(t (t- k) = O(t) +
(23)
where O (t (t) the predicted output feature at timestep k A is the difference hetween features from the two latest full-compute steps, N is the interval between the full computations, and w(t) is a scaling factor at the timestep t. Existing approaches differ in how w(t) is determined. For instance, FasterCache (Lv al. 2025) linearly increases w(() from zero to one increases, while GOC (Qiu et al. 2025) employs a fixed hyperparameter for w(t). We show in Table quantitative comparisons between RFE and these strategies. We can see that RFE consistently outperforms all the others significantly. This suggests that simple linear extrapolation fails to capture the irregular changes in the output features, while RFE improves the prediction by leveraging the input-output relationship.
Relational cache scheduling (RCS) RCS performs full computations based on the prediction error in input features of the first module. To evaluate the effectiveness of RCS, we provide in Table quantitative results of various scheduling strategies on ImageNet (Deng et al. (209).. First, we adopt distance-based strategy (Liu al. 2025a) and schedule full computations based the distance

---

Published as a conference paper at ICLR 2026
Table 4: Quantitative comparison of different components for DiT-XL/2 PPeblees & Xie
2023 on
ImageNet (Deng et al. 2009). All results are obtained using the first-order approximation (m 1).
Methods
NFC
FID
sFID
FID2FC1
SFID2FC4
TaylorSeer (Liu 2025b
2.65
5.60
0.57
2.77
RFE
2.52
5.18
0.43
2.02
RCS
14
2.52
4.76
0.36
1.88
RFC (RFE RCS)
14
2.51
4.66
0.31
1.41
TaylorSeer (Liu et al.
2025b)
11
2.87
5.85
0.73
3.53
RFE
11
2.69
5.22
0.55
2.55
RCS
2.77
5.21
0.62
3.09
RFC (RFE + RCS)
11
2.71
4.88
0.51
2.30
Table 5: Quantitative comparison of different strategies for estimating the change in output features
for DiT-XL/2 (Peebles Xie 2023) on ImageNet (Deng et al. 2009). All results are obtained using the first-order approximation (m 1).
Methods
NFC
FID2FC1
sFID2FC.
Latency (8).)
Linear
0.73
3.40
2.85
(f) 0.8
14
0.73
3.36
2.84
(x()) 1.0
14
0.57
2.77
2.84
ur(f) 1.2
0.52
2.51
2.84
RFE
0.43
2.02
2.86
Linear
0.93
4.08
2.52
w() 0.8
11
0.94
4.17
2.51
w = 1.0
11
0.73
3.53
2.51
uz(A 1.2
11 0.67
3.24
2.52
RFE
11 0.55
2.55
2.52
Table 6: Quantitative comparison of different strategies for the cache scheduling using DiT- XL/2 (Peebles & Xie 2023) ImageNet (Deng et al.. 2009). All results are obtained using the first-order approximation (m 1).
Methods
NFC
FID2FCI
sFID2FCI
Latency (s)
Input distance
14.03
0.50
2.53
2.62
All modules
14.02
0.37
1.85
2.79
RCS
14.00
0.36
1.88
2.63
Input distance
11.15
0.72
2.23
All modules
11.00
0.61
3.03
2.41
RCS
11.01
0.62
3.09
2.26
between the current and the latest cached input features, instead of exploiting the prediction error. We can see that RCS achieves stronger performance, suggesting that the prediction error more reliable for forecasting-basee prediction errors of all modules rather than using the first module only. RCS remains comparable to this variant, indicating that using the error from the first module is sufficient.
methods than the simple distance measure. Second, we add the input
CONCLUSION
We have shown that output feature changes in DiTs are highly irregular across timesteps, while still maintaining strong correlation with their corresponding inputs. Based this, we have introduced a new caching framework, dubbed RFC, that leverages the input-output relationship for accurate feature prediction, consisting of two novel components, RFE and RCS. We have demonstrated that RFC achieves the state of the art on standard benchmarks, and further validated the effectiveness of each component through detailed analysis.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENTS
This work was partly supported by ITTP grant funded by the Korea government (MSIT) (No.RS- 2022-00143524, Development of Fundamental Technology and Integrated Solution for Next- Generation Automatic Artificial Intelligence System, No. N0.20022--00124, RS-2022-1220124, Devel- opment of Artificial Intelligence Technology for Self-Improving Competency-Awer Learning Ca- pabilities), the KIST Institutional Program (Project No.2E33001-24- and Samsung Electronics Co., Ltd (IO240520- 10013-001.
REFERENCES
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. arXiv preprint, 2016.
Stephen Batifol, Andreas Blattmann, Frederic Boesel, Saksham Consul, Cyril Diagne, Tim Dock- horn, Jack English, Zion English, Patrick Esser, Sumith Kulal, al. Flux.1 kontext: Flow match- ing for in-context image generation and editing in latent space. arXiv preprint, 2025.
Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint, 2023a.
Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhor, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with latent diffusion mod- els. CVPR, 2023b.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. hierarchical image database. In CVPR, 2009.
ImageNet: A large-scale
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat GANs on image synthesis. In NeurlPS, 2021.
Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. CLIPScore: A reference-fre free evaluation metric for image captioning. In EMNLP, 2021.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In NeurIPS, 2017.
Jonathan Ho, Ajay Jain, and Picter Abbeel. Denoising diffusion probabilistic models. In NeurIPS, 2020.
Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianx- ing Wu, Qingyang Jin, Nattapol Chanpaisit, al. Vbench: Comprehensive benchmark suite for video generative models. In CVPR, 2024.
Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint, 2024.
Zhiqiu Lin, Deepak Pathak, Baiqi Li, Jiayao Li, Xide Xia, Graham Neubig, Pengchuan Zhang, and Deva Ramanan. Evaluating text-to-visual generation with image-to-text generation. In ECCV, 2024.
Feng Liu, Shiwei Zhang, Xiaofeng Wang, Yujie Wei, Haonan Qiu, Yuzhong Zhao, Yingya Zhang, Qixiang Ye, and Fang Wan. Timestep embedding tells: It's time to cache for video diffusion model. In CVPR, 2025a.
Jiacheng Liu, Chang Zou, Yuanhuiyi Lyu, Junjie Chen, and Linfeng Zhang. From reusing to fore- casting: Accelerating diffusion models with taylorseers. In ICCV, 2025b.
Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. In ICLR, 2023.

---

Published as a conference paper at ICLR 2026
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. DPM-Solver: A fast ODE solver for diffusion probabilistic model sampling in around 10 steps. In NeurIPS, 2022.
Zhengyao Lv, Chenyang Si, Junhao Song, Zhenyu Yang, Yu Qiao, Ziwei Liu, and Kwan-Yer Yee K Wong. PasterCache: Training-free video diffusion model acceleration with high quality. In ICLR, 2025.
Xinyin Ma, Gongfan Fang, and Xinchao Wang. DeepCache: Accelerating diffusion models for free. In CVPR, 2024.
Charlie Nash, Jacob Menick, Sander Dieleman, and Peter Battaglia. Generating images with sparse representations. In 2021.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In ICCV, 2023.
Junxiang Qiu, Lin Liu, Shuo Wang, Jinda Lu, Kezhou Chen, and Yanbin Hao. Accelerating diffusion transformer via gradient-optimized cache. In ICCV, 2025.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In CVPR, 2022.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-Net: Convolutional networks for biomedi- cal image segmentation. In MICCAI, 2015.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Bureu Karagol Ayan, Tim Salimans, et Photorealistic text-to-image diffusion models with deep language understanding. In NeurIPS, 2022.
Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. In ICLR, 2022.
Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training GANs. In NeurlPS, 2016.
Pratheba Selvaraju, Tianyu Ding, Tianyi Chen, llya Zharkov, and Luming Liang. FORA: Fast- forward caching in diffusion transformer acceleration. arXiv preprint, 2024.
Junhyuk So, Jungwon Lee, and Eunhyeok Park. Frdiff: Feature reuse for universal training-free acceleration of diffusion models, 2024b. In ECCV, 2024.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In ICLR, 2020.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. In NeurIPS, 2019.
Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. In ICML, 2023.
Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero Simoncelli. Image quality assessment:
from error visibility to structural similarity. IEEE TIP, 2004.
Felix Wimbauer, Bichen Wu, Edgar Schoenfeld, Xiaoliang Dai, Ji Hou, Zijian He, Artsiom Sanakoyeu, Peizhao Zhang, Sam Tsai, Jonas Kohler, et al. Cache me if you can: Accelerating diffusion models through block caching. In CVPR, 2024.
Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagereward: learning and evaluating human preferences for text-to-image generation. In NeurlPS, 2023.
Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao. Lsun: Construction of large-scale image dataset using deep learning with humans in the loop. arXiv preprint, 2015.
Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features perceptual metric. In CVPR, 2018.
12

---

Published as a conference paper at ICLR 2026
Chang Zou, Evelyn Zhang, Runlin Guo, Haohang Xu, Conghui He, Xuming Hu, and Linfeng Zhang. Accelerating diffusion transformers with dual feature caching. arXiv preprint, 2024.
Chang Zou, Xuyang Liu, Ting Liu, Siteng Huang, and Linfeng Zhang, Accelerating diffusion trans- formers with losen-wise feature caching. In ICLR, 2025.
13

---

Published as a conference paper at ICLR 2026
1.0
1.0
1.0
0.8-
Costne Sim.
0.8
Cocine Sim.
0.8
0.6
0.4
0.4
10th modile
10th module modille
0.2
0.2
15th modile
module
45
40 35 30 25
30
Timestep
TimeStrp
TimeStep
(a) Linearity.
(b) Input feature consistency.
(c) Output feature consistency.
Figure 5: Feature analyses using DiT-XL/2 (Peebles & Xie 2023 ImageNet (Deng et al. 2009). (a) lin- earity analysis of diffusion features. The solid lines present the linearity between the input and output features, while the dashed lines show the linearity between the timesteps and output features. (b-c) The directional con- sistency of the input and output features.
In the Appendix, we first validate the conditions underlying Proposition 1 (Sec. A). followed by further analyses of the input-output feature relationship (Sec. B). We then provide more quantita- tive and qualitative results (Sec. C) with additional discussions (Sec. D). Finally, we present the limitations RFC (Sec. and describe usage of LLMs (Sec.
A CONDITIONS FOR PROPOSITION
Proposition D relies two key conditions commonly observed in diffusion models: (1) locally linear relationship between input and output features, (2) directional consistency of feature changes across timesteps. The first condition is supported by the fact that feature changes between timesteps are typically small (Ma et al. 2024 Selvaraju et al. 2024 So et al., 2024), allowing for local linear approximation by the Taylo for's theorem. The second condition is grounded in empirical observations from prior works (Ly et al. 2025 Qiu et al. (025), which observe that the direction of feature changes remains largely consistent throughout the denoising process. To further verify these, we provide additional empirical analyses on these conditions.
Linearity. To validate the local linearity between the input and output features, we fit a linear model via least squares between the input and output features of the same module (e.g., attention and MLP),, using the features of 5 consecutive timesteps (e.g., L ~ (L - (4). We show in Fig. 5 a) the average coefficient of determination (R²) score of the fitted linear model. We show consistently high values (e.g., close to 1), demonstrating strong linearity between input and
that R scores output features. For comparison, we also compute the R² scores to quantify the linearity between the timesteps and their corresponding output features. We observe a substantial drop in R² scores, which indicates that the relationship between the output features and the timesteps is highly non- linear. This is because the magnitude of changes in the output features varies significantly. which limits effectiveness of previous forecasting methods (Liu al. 2025b Lv et al.) 2025 Qiu al.) 2025 (see Fig. ()).).
Directional consistency. To validate the directional consistency of feature changes, we com- pute the pairwise cosine similarities of the feature differences A, k) and A, - k). where k ranges from 1 to 4. Specifically, we compute the difference vectors of features across timesteps (e.g., At
[(t k) [(t) for ilaritics among them to assess how consistent their directions similarities to quantify the overall degree of directional consistency. As shown in Fig. 5]b-c), the average similarities remain consistently high across modules, indicating that the direction of feature change is preserved over time. These empirical observations align well with the findings of prior
to 4), and measure the pairwise cosine sim-
time. We then average these
studies (Lv et al. 2025 Qiu al. 2025)
B MORE ANALYSES
We have shown the strong correlation between the input and output features in the main paper. To further validate the generalizability of the analyses, we provide in f additional analysis on the
14

---

Published as a conference paper at ICLR 2026
1.0
1.0
Distaince
0.5
ISDD (16)
25th modiaee
15th media
Dilation 1 three
0.2
0.2
-00
40
20
10
Timestop
Tinestop
Timestop 67
12-
1949 maddak
Diettaac
0.8
0.5
medak
Boxrs Tome
1000
0.6
0
0.4
0.4
RSD
Wil
0.0
MW
0.0
Timestep
Timesoop
Timestop
(a) Output difference.
(b) Input difference.
(c) Variation of 8i.-
(d) &o vs. Er.
Figure 6: Analyses using FLUX.1 dev (Baiifll et al. 2025 on DrawBench (Saharia et al.
2022) (upper row)
and Hunyuan Video (Kong et al. (024) on VBench (Huang et al. 2024) (lower row). (a-b) Min-max normalized
L2 distances of output and input features, measured between consecutive timesteps. (c) RSD of 8t k) in
Eq. 5 with varying (d) Relative Lı errors of the output and input features, i.c., (t and Er in Eq. 21 respectively.
Table 7: Quantitative comparison under extremely high acceleration ratios for DīT-XL/2 (Peebles & Xie, (023) on ImageNet (Deng al. 2009 All results are obtained using the second-order approximation (m = 2).
Methods
NFC
FID
SFID
FID2FCJ
sFID2FCJ
Full-Compute
2.32
4.32
TaylorSeer (Liu al., 2025b)
13.57
13.19
10.68
15.19
RFC
4.31
5.39
1.83
4.80
TaylorSeer (Luu 2025b
45.81
27.64
42.29
RFC
6.29
6.13
TaylorSeer [Tiu
2025b
160.85
135.55
157.53
137.65
RFC
7.14
5.71
8.78
Table 8: Quantitative comparison on distilled model, FLUX. 1 schnell (Batifol et al. 2025) with denoising steps, DrawBench (Saharia al. 2022). All results are obtained using the first-order approximation (m = 1).
Methods
NFC
PSNR1
SSIM
LPIPS!
Full-Compute
1.0000
0.0000
Step reduction
26.5377
0.8855
0.1164
TaylorSeer (Liu et al. 2025b
29.3435
0.9242
0.0736
RFC
32.7436
0.9275
0.0635
Step reduction
25.02.14
0.8580
0.1511
TaylorSeer (Liu et al. 2025b
22.2134
0.7491
0.3687
RFC
27.1185
0.8931
0.0983
relationship between input and output using FL.UX.1 1 dev (Batifol al., 2025) on DrawBench (Sa- haria et al. 2022), and Hunyuan Video (Kong et al. 2024) VBench (Huang et al. 2024). Similar we can see that the feature changes are irregular across timesteps, while those of input and output remain closely aligned with each other. We can also see that the ratio between the magnitude changes in output and input features (i.e., (t - k)) is highly consistent along timesteps. Finally, and E, (t), respectively. are closely aligned with each other, demonstrating the strong correlation between the input and output features.
to the findings in Figs. [ and 2
that the relative L1 error of the output and input features, & (t)
15

---

Published as a conference paper at ICLR 2026
Mctlods
Guirea
Cooket
PollCreppdd 1.00
FORA
Figure 7: Qualitative comparisons of class-conditional image generation for DiT-XL/2 Peebles &
Xie 2023)
ImageNet (Deng al. 2009
C MORE RESULTS
Quantitative results on higher acceleration ratios. We show in Table quantitative results us- ing DiT-X1/2 (Peebles & Xie 2023) ImageNet (Deng al.) 2009) under extremely low full computation budgets (iee., NFC 6, 5, 4). We can see that the performance of TaylorSeer (Liu al. 2025b) degrades significantly. This indicates that relying solely temporal extrapolation is insufficient when intervals between full computations are large. In contrast, RFC maintains strong correlations, which enables more accurate prediction even when feature changes become irregular and difficult to extrapolate.
performance by leveraging
Quantitative results on distilled model. We show in Table X quantitative results on the distilled model, FLUX.1 schnell (Batifol et al. 2025). with 6 denoising steps. For the distilled models, the overall number of denoising steps is significantly smaller, which leads to larger changes in features between consecutive timesteps and makes feature prediction more difficult. Under this setting, can see that TaylorSeer (Liu al. 20256) shows notable performance drop, while RFC maintains consistently strong performance across all metries. This demonstrates the effectiveness of leverag- ing the onput-output relationship for stable and accurate feature prediction, even when the output features of distilled models highly fluctuate.
More qualitative results. We provide in Figs. 9 additional qualitative results for class- conditional image, text-to-image, and text-to-video generation tasks. Consistent with the obser- vations in Sec. 4.2, RFC produces higher-quality images and videos compared to those of state- of-the-art approaches (Selvaraju al. 2024. Liu et al. 2025b). These results further validate the effectiveness and superiority of RFC.
D
MORE DISCUSSIONS
Details on choosing т. In our framework, T is key parameter RCS that controls the trade-off between generation quality and computational efficiency. Specifically, larger allows more pre- diction steps before triggering full computation, thereby reducing the number of full computations and improving efficiency. This role is similar to adjusting 5 in (Liu et al. 2025a) or N in (Liu al.)
20256 Selvaraju et al. 2024 Zou et al. 2024), where users can tune the parameter based on their desired efficiency level.
For fair comparison with other methods, we tume 7 to match the target NFC. Specifically, we conduct grid search over candidate values, generate 10 samples per setting, and measure the average NFC.
16

---

Published as a conference paper at ICLR 2026
Temeer Lalan cade
ataa and The
STOP
FORA 6.23
STOP
TaylorSer 6.23
STOP
RFC 6.25
STOP
Figure 8: Qualitative comparisons of text-to-image generation for FLUX.1 dev (Batifol al. 2025 on DrawBench (Saharia et al., 2022).
Methods
Meloni, soyle chay
cs Corll
surtin rytoppuis right Calnny
Ful-Compcie 1.00
FORA
Teal
Figure 9: Qualitative comparisons of text-to-video generation for Hunyuan Video on VBench (Huang et al. 2024).
(Kngg et al.
2024
We then fix the т value that yields the desired NFC, and use this setting for reporting quantitative results. For detailed settings for Tables T are shown in Table We also provide the trade-off of our approach with varying 7 in Fig. T0 Please note that we have observed that NFC remains highly consistent across different samples, requiring no extensive tuning.
Reliability of CLIP score. RFC does not achieve the highest CLIP score in Table 2 due to the limitations of the CLIP score. The CLIP score mainly reflects coarse semantic alignment between the images and texts rather than detailed correctness, demonstrated by prior work (Lin et al.) 2024]. Also, we can see in Table D that the CLIPP score of TaylorSeer frequently outperforms fully computed counterparts, which shows its unreliability.
To further validate this, we show in Fig. Il qualitative results where CLIP scores fail to reflect true generation quality. We can see that even when the generated images exhibit visual artifacts, they can still receive high CLIP scores when they roughly match the prompt. For instance, TaylorSeer (Liu al. 2025b) generates dog with distorted face and cat with unrealistic colors (first row), clock where the number "9" is inaccurately shaped (second row), and an umbrella with unnatural textures (third row), but these examples show the highest CL.P scores.
17

---

Published as a conference paper at ICLR 2026
Table 9: Settings of for various models.
Model
Order (m)
NFC
0.19
14.01
0.38
11.01
DIT-XL/2
Peobles
0.48
10.04
0.30
14.02
1.10
8.00
Batifol
0.34
13.80
1.15
8.03
0.70
8.96
Hunyuan Video Kong
2024
7.09
6.67
3.0
II]
1.5
1.0
0.2
0.4
0.6
0.8
1.0
1.2
Figure 10: Trade-off between generation quality and computational cost on ImageNet (Deng et al, 2009) using DiT-XL/2 (Peebles & Xie 2023) under varying values of The plot shows the change in performance (i.e., sFID2FC) as J increases, with the corresponding FLOPs indicated by the size and color of each circle.
Table 10: Quantitative results for layer-wise caching strategies on ImageNet (Deng et al. 2009) using DiT-XL/2 (Peebles & Xie 2023). We reduce the frequency of full computations by half for the first (shallow skip) or last (deep skip) 7 blocks out of 28. We adjust to use more full computations for these variants to match FLOPs. All results are obtained using the second-order approximation (m 2).
Methods
NFC
FLOPs (T)
FID.
sFID
FID2FCI
SFID2FCL
IST
Full- Coll--oopute
50
23.74
2.32
4.32
241.25
Shallow Skip
3.09
Deep Skip
6.66
3.47
2.82
211.54
RFC (m
14
6.66
2.52
4.60
0.30
1.33
231.00
Shallow Skip
8
3.35
11.70
9.13
9.10
11.71
147.88
Deep Skip
10.20
6.59
149.10
RFC
5.21
215.39
Layer-wise caching strategy. We investigate whether skipping full computations for certain lay- ers leads to better trade-off between image quality and efficiency. Specifically, we consider two strategies: skipping full computations more often for (1) the shallow layers (i.e., the first blocks), or (2) the deep layers (i.e., the last 7 blocks). In these settings, the selected layers perform fewer full- compute steps (i.e., at half the frequency) than the remaining layers during the denoising process. We can see from Table 10 that both variants result in worse performance compared our method, indicating that naive layer-wise skipping does not lead to better overall results. We believe, however, that exploring the strategies for layer-wise skipping could be promising direction for future work.
18

---

Published as a conference paper at ICLR 2026
Pomm
Compute 11.00
Tayledeer 5.22
5.23
one cog ettingo the gless.
103443
4,2903
naree table.
20,2544
20,003
905001
Anumbella top spoon
Figure 11: Clip scores of text-to-image generation for FLUX.1 dev (Batifol al. 2025 Draw- Bench (Saharia et al. 2022).
Table 11: Quantitative comparison of FORA Selvaraju et al. 2024). TaylorSeer (Liu et al.
2025b
and RFC using the DDIM (Song al. 2020 model trained LSUN-bedroom (Yu al. 2015 with 50 timesteps. We use the first-order approximation (m = 1).
Methods
NFC
FID2FCJ
sFID2FCJ
FORA (Selvaraju et al.
2024
(N=5)
11
78.27
69.84
TaylorSeer (Luu et al.
2025b
(N=5)
11
18.83
11.83
RFC
8.55
7.59
FORA (Selvaraju al.
2024
(N=6)
10
103.48
89.17
TaylorSeer (Liu al.
2025b
(N=6)
37.81
21.35
RFC
10
11.64
8.25
Adaptability to U-Net architecture (Ronneberger et al., 2015). Although mainly focus on DiTs, RFC can also applied to U-Net architectures. validate its effectiveness, we compare in Table IT RFC with prior methods (FORA (Selvaraju et al. 2024) and TaylorSeer (Liu et al.) 2025b)) using the DDIM (Song et al., 2020) model trained on LSUN-bedroom (Yu et al.) 2015). We can that RFC significantly outperforms others, demonstrating its generality and architectural adaptability.
Non-uniform intervals. Our RCS dynamically determines when to perform full computations by monitoring the accumulated eиror feature prediction. To investigate how this error varies across the denoising process, we show in Fig. the average interval between two successive full-compute steps across 1,000 samples. We observe that the average interval between early full computations (i.e.. close to noises) tends to be large, and the interval progressively shortens for the later timesteps (i.e.. close to clean images). This indicates that prediction error accumulates more slowly at the beginning of the denoising process.
This phenomenon aligns with the generative process, where early stages form coarse, low www-frequency structures with more stable feature dynamics, while later stages refine fine- grained, high- high-frequency details, involving more dynamic feature changes. RCS naturally adapts to this behavior by perform- ing full computations less frequently during the early timesteps, when feature prediction is easier,
19

---

Published as a conference paper at ICLR 2026
Inbervaal 8
Interval
Full-compute step
Full-compute slep
(a) 14 NFC.
(b) 11 NFC.
Figure 12: Analysis of non-uniform intervals scheduled by RCS using DiT-XL/2 (Peebles Xie 2023 on
ImageNet (Deng et al.] (209). We visualize the average interval between foll-compute steps for (a) 14 and (b) 11 NFC settings. + 2 + at the x-axis denotes the i-th and (i + 1)-th full computation, and the y-axis shows the average interval. The results averaged across 1,000 samples.
Table 12: Quantitative results of DiT-XL/2 PPebles & Xie 2023 on ImageNet (Deng et al. 2009
terms of NFC, order in the Taylor expansion, GPU peak memory, FID2FC, and sFID2FC.
Methods
NFC
Order
Peak Memory (MB)
FID2FC,
<FID2FCJ
FORA (Seharriju et al.
2024
3121
5.32
12.87
TaylorSeer [iu 20255
10
3207
1.42
6.17
RFC
10.04
3335
0.58
2.79
TaylorScar
2025b
3333
1.0
4,86
RPC
10.02
2
3463
0.54
2.43
and increasing the frequency in the later timesteps as prediction becomes more challenging, thereby optimizing the efficiency-qualie trade-off.
LIMITATION
Table 13: Time overhead of RFC components compared to the baseline (TaylorSeer (Liu et al. 2025b)) using DiT-XL/2 (Peebles & Xie 2023) on ImageNet (Deng et al. 2009) with 14 NFCs. Percentages in parentheses indicate the time increase relative to the baseline.
Methods
Time (Overhead %)
Baseline (TaylorSeer (Liu et al. 2025b
2.840s
Input feature computation
0.019s (0.67%)
Input feature prediction
0.002s (0.07%)
RFC effectively improves the generation quality but ineurs additional memory costs store input features, similar to TaylorSeer (Liu et al. 2023b). which stores more output features for higher-order approximations. Notably, as shown in Table 12] RFC generally achieves greater improvements the generation quality than increasing the approximation order by one in TaylorSeer, while requiring comparable amount of memory, highlighting the effectiveness of our approach.
As shown in Table 13 RFC incurs slight time overhead compared to the baseline. Compared to the baseline, RFC needs (1) input feature computation for RFE and RCS, which requires lightweight operations (e.g., LayerNorm. scaling and shifting); and (2) input feature prediction using the Taylor expansion for RCS. To reduce the cost of input feature prediction
RCS, we have demonstrated in that computing the prediction error from only the first module, rather than all modules (e.g., 56 modules DiT-XL/2), sufficient. For instance, under the NFC setting, computing predic- tion enror across all modules increases runtime by approximately 7%, while yielding only marginal improvement performance (0.62 vs. 0.61 in terms FID2FC). We conjecture that this is because errors in early modules affect subsequent modules, suggesting that the prediction error of the first module can be a reliable indicator the overall prediction error. To further reduce the cost of input feature computation, promising direction for future work could be to explore hybrid strategy
20

---

Published as a conference paper at ICLR 2026
that applies RFE only to subset of important modules, while using the baseline method for the remaining ones.
There some failure cases in RFC, where the generated image does not align well with the prompt. This typically occurs when full computations fail to capture the prompt correctly, for example, as shown in the third row of Fig. where astronaut is riding horse, instead of a horse riding an astronaut. This is because RFC accurately approximates the full computations, it also reproduces such errors.
F LLM USAGE
We used large language model (LLM) solely for proofreading and polishing the manuscript. The model was not involved in generating research ideas, designing methods, conducting experiments, or interpreting results.
21


Reviews:
[{"review_id": "PuqRvz84Tk", "reviewer": "Reviewer_gkp8", "summary": "This paper addresses inefficiencies in diffusion transformers (DiTs) by advancing feature caching techniques used to accelerate inference. Prior approaches speed up computation by temporally extrapolating and reusing output features of expensive modules, but these methods can incur significant prediction errors. Through detailed analysis, the authors find that such errors arise from irregular feature magnitude changes and that there is a strong correlation between a module’s input and its output features. Building on these insights, they introduce Relational Feature Caching (RFC), a novel framework that leverages the relationship between inputs and outputs to improve feature prediction accuracy. RFC includes two key components: Relational Feature Estimation (RFE), which uses input features to predict output changes more reliably, and Relational Cache Scheduling (RCS), which estimates likely prediction errors from inputs and recomputes outputs only when large errors are expected. Experiments on various DiT models show that RFC consistently outperforms earlier techniques, significantly improving efficiency and accuracy.", "questions": "How does the model perform in terms of efficiency when the RCS component has a different scheduling policy?\n\nIs there a clear trade-off pattern for the method?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "This work introduces novel components—Relational Feature Estimation (RFE) and Relational Cache Scheduling (RCS)—that have not been explored in prior work. It utilizes input–output relationships, enabling a more accurate prediction of output features. The paper clearly articulates the motivation, challenges, and contributions, and provides a general framework that could inspire further research for DiTs.", "weaknesses": "Clarity of Mathematical Formulation (RELATIONAL FEATURE CACHING section):\n\nThe presentation of equations in the RELATIONAL FEATURE CACHING section lacks clarity, making it difficult for readers to follow the mathematical foundations of the approach. For instance, the connection between the two components, RFE and RCS, is not clear at the beginning, and the logical flow from one equation to the next is not always well-motivated.\n\nActionable suggestion: Including intuitive descriptions or intermediary steps (potentially with illustrative diagrams or simplified toy examples) would help clarify the input-output relationship modeling and the estimation process.\n\nExperimental Evaluation Coverage:\n\nWhile the experiments show consistent gains across several DiT models, the paper primarily focuses on performance improvements and does not fully explore scenarios where the method may fail or be less effective (e.g., with particularly noisy or weak input-output correlations).\nActionable suggestion: Include ablation studies or failure case analysis to identify situations where the relational estimation approach may struggle or need further modification. Examining a broader range of conditions, such as varying degrees of input-output correlation, would provide a more comprehensive understanding of the method's robustness. For Table 4, provide a brief written summary in the main text to guide the reader through the table and highlight the most important results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "79msHDirrB", "reviewer": "Reviewer_6daa", "summary": "Objective: Accelerate Diffusion Transformers by improving feature caching accuracy. The paper identifies that forecast-based caches suffer from large errors due to irregular output changes and strong input–output correlations in modules. It proposes Relational Feature Caching (RFC) with two components: Relational Feature Estimation (RFE) to predict output-change magnitudes from inputs, and Relational Cache Scheduling (RCS) to estimate prediction error from inputs and trigger full computation only when errors are likely high. Experiments across multiple DiT models show consistent, significant improvements over temporal extrapolation baselines, with planned code release upon acceptance.", "questions": "line 302-304, \"For a fair comparison, we reproduce the results of state-of-the-art methods using the official source codes, and adjust the threshold τ in Eq. (13) to ensure that the average number of full computations (NFC) matches that of other methods.\" can you elaborate more on how to adjust the threshold tau?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "This paper studies an important topic of feature caching, which is critical for optimizing the efficiency of diffusion transformers.\n\nThe identification of input–output correlation as a predictor of output changes is insightful and well supported by the empirical analysis.\n\nThe proposed RFC showcase on multiple metrics and settings, demonstrating a comparable to superior performance. The authors also present qualitative results, which are promising.", "weaknesses": "The organization of the manuscript needs improvement. There is overlapping and duplicate content across the first three sections. It may be better to defer the detailed discussion of related work to a later section and to reorganize Section 2 and Section 3.1.\n\nWhile the empirical correlation between inputs and outputs is demonstrated, the paper offers little formal analysis explaining why this correlation should hold across arbitrary architectures or datasets.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "n5Mb3aUEQL", "reviewer": "Reviewer_h2W6", "summary": "This paper proposes Relational Feature Caching (RFC) to accelerate Diffusion Transformers by exploiting correlations between input and output features, rather than relying solely on temporal extrapolation as in prior methods like TaylorSeer. The approach introduces Relational Feature Estimation (RFE) to predict output changes from input variations and Relational Cache Scheduling (RCS) to adaptively trigger full computations based on estimated errors. Experiments on image, text-to-image, and video generation tasks show consistent improvements over existing caching methods in both quality and efficiency.", "questions": "- While RFC significantly accelerates DiTs, what happens at higher acceleration ratios (larger N)? A discussion on generation quality degradation at extreme speedups would help readers assess its robustness.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Clear motivation with strong empirical evidence showing the limitation of purely temporal forecasting.\n- Simple yet effective design—RFE and RCS are well-justified and complementary.\n- Comprehensive experiments across multiple diffusion models and tasks.", "weaknesses": "- In Table 2, TaylorSeer achieves the highest CLIP scores; the paper should discuss why RFC does not consistently outperform it on semantic alignment metrics.\n- The paper could analyze RFC’s applicability to U-Net–based diffusion models to better demonstrate generality and architectural adaptability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Z9CNxxLaLz", "reviewer": "Reviewer_xGHL", "summary": "This paper proposes an improvement of TayloSeer, motivated by the highly correlated magnitudes of change of input and output features. It uses the change of input features to estimate the change of output, therefore reducing the computation to a large extent. On both text-to-image and text-to-video tasks, RFC manages to achieve better or comparable performances against baselines. Visual results from the paper also support the effectiveness of this method.", "questions": "1. How does RFC perform on distilled models?\n2. What is the recomputation rate under different parameter settings? In other words, is $\\tau$ hard to tune for different DiT models? It seems this value differs across the DiT models tested.\n3. The empirical results (Figures 1 and 2) are obtained on ImageNet and DiT, which are relatively simple. Are the findings the same in a different case? For example, it might be more convincing to also show empirical results on FLUX.1 dev or HunyuanVideo.\n4. RFC seems to be slower than the baseline. Where does the main overhead come from, and did the author think about how to reduce it?\n5. Did the author study whether shallow layers / early timesteps should be skipped or not to achieve better results?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The motivation of this paper is clear. The empirical results in Figures 1 and 2 serve as strong evidence for the proposed method.\n2. The discussion of related work is thorough, further enhancing the novelty of this paper.\n3. The experimental analysis is comprehensive. The authors validate their method not just on one model or task, but across three distinct, large-scale generative tasks (class-conditional, T2I, T2V) using modern, powerful models.", "weaknesses": "Please see the weakness of the method and experiments in the questions.\n\nHere are some weaknesses in the presentation:\n1. The related work section is a bit hard to read with the long paragraph. I suggest the author reorganize this section to improve the readability.\n2. I suggest using the full name of RFE and RCS as the paragraph header.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)