Diffusion on Demand: Selective Caching and Modulation for Efficient Generation
Authors: Hee Min Choi
Hyoa Kang
Dokwan Oh
Nam Ik Cho
Venue: NeurIPS 2025 poster ()
Keywords: Generative models
diffusion
inference acceleration
Primary area: deep_learning
TL;DR: Diffusion on Demand

Abstract: Diffusion transformers demonstrate significant potential for various generation tasks but are challenged by high computational cost. Recently, feature caching methods have been introduced to improve inference efficiency by storing features at certain timesteps and reusing them at subsequent timesteps. However, their effectiveness is limited as they rely only on choosing between cached features and performing model inference. Motivated by high cosine similarity between features across consecutive timesteps, we propose a cache-based framework that reuses features and selectively adapts them through linear modulation. In our framework, the selection is performed via a modulation gate, and both the gate and modulation parameters are learned. Extensive experiments show that our method achieves similar generation performance to the original sampler while requiring significantly less computation. For example, FLOPs and inference latency are reduced by $2.93\times$ and $2.15\times$ for DiT-XL/2 and by $2.83\times$ and $1.50\times$ for PixArt-$\alpha$, respectively. We find that modulation is effective when applied to as little as 2\% of layers, resulting in negligible computation overhead.

Diffusion on Demand: Selective Caching and Modulation for Efficient Generation
Hee Min Choi chm. choi @samsung. com
Hyoa Kang 12+ hyoa, kang@samsung com
Dokwan Oh dokwan. oh@samaung com
Nam Ik Cho 2f ni nichossuu.c ac kr
Samsung Electronics Co., Ltd.
2 Scoul National University
Abstract
Diffusion transformers demonstrate significant potential for various generation tasks but are challenged by high computational cost. Recently, feature caching methods have been introduced to improve inference efficiency by storing features at certain timesteps and reusing them at subsequent timesteps. However, their effectiveness is limited they rely only choosing between cached features and performing model inference. Motivated by high cosine similarity between features across consecutive timesteps, propose cache- based framework that reuses features and selectively adapts them through linear modulation. In our framework, the selection is performed via a modulation gate, and both the gate and modulation parameters are learned. Extensive experiments show that our method achieves similar generation performance the original sampler while requiring significantly less computation. For example, FLOPs and inference latency are reduced by 2.93 x and 2.15x for DiT-XL/2 and by 2.83x and 1.50 for PixArt-a. respectively. We find that modulation is effective when applied to as little as 2% of layers, resulting in negligible computation overhead.
Introduction
Diffusion models 14
have emerged as promising generative methods across different recent years. Across various backbone architectures for diffusion 49 are increasingly popular for generating high-quality images 35
45
46
modalities such as images 8
37
models, transformers
I
54 3
videos (27 36 6 and 3D content [32 Although diffusion transformers leverage the scalability of the architecture, they are challenged by high computational cost and slow inference speed.
Numerous frameworks have been developed to improve inference efficiency in diffusion models. As the sampling cost scales proportionally with the number of sampling steps and the model size, efficiency improvements primarily aim to reduce the number of steps and network inference cost. Several studies reduce the number of sampling steps by optimizing the denoising trajectory 44 25
26 or employing distillation methods
33
47 28
16
30 Other approaches mainly focus on model
compression by pruning [15 20 11] quantization 19
48
12 4 Different works use dynamic
inference mechanisms by allocating models of varying sizes for different timesteps [53 34
"Equal contributions Corresponding author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

100
Sample Frequency (%g
Attention
80-
0-
Timestep (+)
Actention valuee
500
0.5
R-
40-
8
20-
1009
59
49
29
19
0.0
0.2
0.4
0.6
0.8
1.0
-1000
500
nooe
Timestep (+)
Image
Cosine Similarity
Allention
(a) Similarity hcatmap
(b) Similarity at and t (c) 28th attention f13 and f11
Figure I: Visualization of attention features in DiT-XL/2 with 50 DDIM timesteps {4x {4=)22) == Heatmap (a) reveals high cosine similarity features across consecutive timesteps. Histogram (b) shows the cosine similarity distribution between feature pairs at a two-timestep interval {(tg. (=+2))ss with most values near 1. Scatter plot (c) visualizes feature values of the 28th attention layer at timesteps 13 and 11, showing a strong linear relationship.
A new approach to dynamic inference in diffusion models involves special caching mechanism. The main idea is to cache intermediate features at certain timesteps for reusing them at subsequent timesteps by leveraging high similarity between features consecutive steps. Some eaching-based methods for diffusion transformers 5 56] rely on handcrafted deterministic strategies to select features for caching. In contrast, other studies 30 42 propose to learn optimal caching schemes, yet their improvements are limited because they only exploit adjacent step redundancy. Efficiency gains in both training-free free and learning- baased methods for diffusion transformers are constrained, as they only involve choosing between cached features and performing network inference.
In this paper, propose cache-based framework for diffusion transformers that improves inference efficiency by reusing features and selectively adapting them through transformations. Our key idea builds the observation, which is also noted in the previous work [41], that attention and multi-layer perceptron (MIP) features in diffusion transformer blocks exhibit high cosine similarity
across consecutive timesteps (Figure Instead of choosing between reusing features or performing inference, we propose leveraging lightweight linear modulation to approximate the original inference given the high cosine similarity.
In our framework, we learn modulation gate scores in a differentiable manner to decide whether to apply the modulation, while simultaneously training scale and translation parameters of the modulator. At inference time, we use a threshold to enable hard selection. Both the modulation gate and modulator time-dependent but input invariant, enabling the construction of static computation graph for inference similar to the prior work [0 422 Also, the optimization does not require model parameter updates, ensuring cost-efficiency and ease of implementation.
We perform experiments using popular diffusion transformers. Our framework obtains comparable generation performance to the original sampler with significantly less computational cost. For instance, FLOPs are reduced by 2.93x and 2.83 x DiT-XL/2 (35) and PixArt-a 3 respectively.
The inference speed-up is 2.15x and 1.50x for DiT-XL/2 and PixArt-ar Also, the proposed method achieves superior generation quality compared to existing cache-based approaches and fewer-step samplers, while maintaining similar FLOPs. We find that the modulation is effective when applied to a small percentage layers as low as 2%, resulting in negligible computational overhead.
In summary, made the following contributions:
We propose cache-based framework for diffusion transformers that improves inference efficiency by reusing features and selectively adapting them through linear modulation. To knowledge, this is the first method approximate diffusion transformer inference using lightweight cache transformations.
We show that applying the modulation to as little as 2% of layers is effective, leading to negligible computational overhead.
Experiments demonstrate our method generally achieves similar generation performance with significantly less computation compared the original sampler, while outperforming cache-based approaches and fewer-step samplers in generation quality under similar FLOPs.

---

Inferonce and cache foatures
Modulation Gale Scoree (c:0.9)
Use cached featuree
P--1
Br-2
Use Modulaled fealures
0.27
Cachod features
MLP
MLP
MLP1
0.10
0.91
Modulated festures
9
Modulstinn Gale
SA
SA
SA
Modulation Parameters a.y
45-1
MLP
1.10
0.35
3.30
2.71
timestep 4
limestep.
timestep 5--
-2.12
-0.08
noise
Image
Figure 2: Example of the sampling process in the proposed framework with activation cycle 3/=3 at i-th basic block. Given N consecutive timesteps 14,1, (2-2] network inference is performed timestep la and cache features of attention I and MLP A/ in the i-th basic block. Then, for each step m E {s 1, 2} and layer, the modulation gate score Sm, is greater than predefined threshold the linear modulation with scale and translation Ym is applied, and the modulated feature replaces the layer computation at timestep fm. Otherwise, the cached feature itself does this.
2 Related Work
Transformers diffusion. Transformers 49 have been adopted as an alternative to UNet 38 in diffusion models. Recently, DiT [35] demonstrates the scalability of diffusion transformers, enabling various applications such as text-to-image and video generation 36 27 and 3D generation [2]
Efficiency improvement for diffusion. Diffusion sampling involves multiple network inferences, making computationally inefficient. DDIM
44 improves the original DDPM [14 non- proposes advanced approximations of diffusion ODE 50) use distillation techniques to efficiently reduce the number of sampling steps. Model compression strategies have also been explored, such network pruning [15 20) and quantization
Markovian cases, whereas DPM
25
26
solutions. Some methods [39
47
28
16
19
48
12
Other studies use dynamic inference, allocating
models of varying sizes to different steps
3
Cache in diffusion for image generation. Feature caching methods have been introduced to improve inference efficiency by storing features from previous timesteps and reusing them in subsequent timesteps, leveraging feature similarity. Early works focus on UNet architecture [31 5T One such work [51] applies parameter-heat
channel-wise linear transformation as post-processing to all cached features selected by a deterministic criterion, whereas our method jointly learns both whether to apply modulation and the corresponding lightweight, layer-wise modulation parameters. More recently, caching strategies have been extended diffusion transformers [18
43 Training-
free strategies utilize different features for caching such attention features [23 attention and
MLP features 41, feature residuals [5], and token-wise selected features [57]
56 yet they rely
on handerafted deterministic criteria for caching decisions. In contrast, other works 30 42 learn optimal caching strategies but achieve limited improvement by only exploiting redundancy between adjacent steps. Although prior work explored transformation of cached features in the image domain, a concurrent study [29] applies modulation the video generation, where two cached features combined using timestep-dependent deterministic weighting function.
3 Method
We propose a cache-based framework for diffusion transformers that improves inference efficiency by reusing features and selectively modulating them through learned linear transformations. We begin with preliminary overview diffusion, then present caching and modulation mechanism, followed by the training objective that unifies these components.
3.1 Preliminary
Diffusion. The forward process of diffusion transforms a sample up from the data distribution ((p)) by gradually adding Gaussian noise: 2; K200 + where (0, and Kr and o; noise

---

coefficients. In the reverse (denoising) process, given x, at timestep & > 0 and < is computed as 25
Hig ( c- (A) da
Dt
2s
(I)
Fig
where A = log((s/ar)) and (x(-) is the inverse function of A that satisfies & (x,)) = t. Here, ea (-)-) represents the learned model, which in our case, the diffusion transformer. The integral term in
can be approximated by adopting Taylor expansion at X, in the first-order [4 or higher-orders [25
Diffuusio transformer. Diffusion transformers consist of a sequence of basic blocks with self- attention fSA, multi-layer perceptron fMLLP and cross-attention fCA (for conditional generation) blocks: fz o...o f2 fi where f; = { PCA fMLPP) } and L denotes the depth of the model. Each block can be represented
(el () 1) = 21' g:( 14 ,0) LE (SA, CA,M)P))
consisting of residual connection. Here, g:(t) is timestep t-conditioned scalar, h is the self- attention, cross-attention, or MLP layer in the i-th basic block, and & denotes the input to the i-th ( block timestep f. For simplicity, suppress the condition y in conditional generation.
3.2 Caching for Diffusion Transformers
The main idea of feature caching methods is to approximate (-(-) by reusing intermediate features. We first describe the naive feature caching scheme 4T For the first of N consecutive timesteps {tr-1, tr-22-2- fr-x) } in the I stsp diffusion model, the naive caching performs network inference the first timestep fr-1 and stores all intermediate features of self-attention, cross- rros-attentiinn and MLP layers, h4( (T-1)
(2). Then the next N timesteps, the corresponding operations
Algorithm Training
Algorithm 2 Sampling
I: Input: Data distribution ((). difffusion model c.()) optimizer, ODE solver (r). total steps T', the step schedule [t= cycle.V
1: Input: Diffusion model c+(.), modulation gate score 3, modulation parameters a, 7. ODE solver (()). total steps Y', the step schedule {tx {t.).=u in
T-1 in vr()) and activation
(.).. activation cycle N and threshold
2: Randomly initialize 8, and
2: Compute r == (T 1))%N
3: Compute = 1)%N
Ør-1 N(O, to do 4: for = =
repeat
5: Io g(z0) and n 2/00, -1-1
for / (S.A, CA,, MLP)
do
5 N' +
6:
if s%N = then Calculate f((,,t) (w) and cache (,(;, tx)'s in Fq 2 # Inference and cache features
# Step t. for calculating states for caching
7:
x, ~ Nx,;; KzHu, o21)
8:
co(x,, t.) and cache t,,,t is 201,N -
8:
else
9:
9:
gl. Sigmoid(
10:
10:
- (at"'>>)
# Step fm for using cached states
11:
i. + (ea,ttostm
11:
Calculate f((,4)) by Eq
12:
Im+1 N N (Im+1; Km+1100, x2
12:
end if
6mt1 60
13:
14:
Im + (cm4+ 1, Emaz,, tm)
14:
if
s%N then
(igmoid((3)
15:
e, + ca (.)
# Optimize
Original inference
16:
16:
Calculate éo (itm tmi a, 7) by Eq with
17:
Ca La; 8, a, 7) Use cached features
Eq
17:
C || (2m, (m) cs (Im, bm) + A
18: end if
18:
Compute gradients for B, a,7 with respect
19: I.-1 <- V(ec, Lo, 41.)
loss C and update them with optimizer
20: end for
19:
until converged
21: return

---

are substituted with the cached features. That is, for E {SA, MLP), = 1...., and im E ,....,.... t -
ta tm) ((((
where == indicates the assignment operation. After completing the N timesteps, the feature cache resets and begins a new cycle by re-initializing the cache as described above. Here, N is called activation cycle.
The effectiveness of feature caching is attributed to high similarity between features at the same depths across adjacent timesteps. However, the features not identical, resulting in degradation of the generation quality, for which fine-grained caching methods are proposed (30 [7]. Although the advanced upproaches improve the quality issue, they offer limited gains in computational efficiency as they rely only on choosing between caching and network inference.
3.3 Cache Modulation
key question now is how similarity between features can be effectively leveraged to emulate full T-step network evaluation tx) Our core idea is that we approximate some parts of network inference using transformations of the cached features, while the approximation would not increase computational cost too much. Specifically, for consecutive timesteps (fs, ..t......,.......... (t=-++) with A activation cycle and cached feature (x(2 ts), we find lightweight transformation L that approximates the original inference [= (.m) in the basic block (2) by:
Re tm) == 2 + (:mmm * Chi, (n tx)),
(4)
where / E (SA, CA, MLP). i i= 1,...,, and 4m E (1a-1,.... - +1
1- Based on observations
from previous methods (30 41 that feature differences vary across timesteps and layers, we timestep-wise (fm) and layer-wise (I, transformations.
We now specifically define the lightweight transformation. Motivated by high cosine similarity and strong linear relationship between features among consecutive timesteps illustrated in Figure [ we propose using linear modulation scheme. Meanwhile, experimental results in the previous study [41] indicate that the naive caching mechanism (3) alone ensures decent generative performance, so we add a modulation gate to allow selective incorporation of the modulation. In summary, we define the lightweight transformation Cli in (4) by:
chi (M/(
2.d = (1 - ( (t)) ta); Shi Ls 7m)
(1 te) + Shi (ata h4(22 te) + t/a) (5)
where and E and wh E IR are scale and translation parameters of the modulation, and 3 E {0. 1) is the modulation gate score. We denote the approximation of the original network evaluation (0 fm) using (4) with (5) by éØ (Imm tmi B, a, 7) where = {a } (3h:1 and = (nm }.
One simple way to find a and vd. in (5) is taking the least square solution for each timestep and layer in data-driven way. Specifically, set 3ld and for many input z's to i-th / layer at timestep bm. find af and 7/i such that
minimize Σ -m) 12,
Then, given af and d.1 the modulation gate S can be determined by comparing mean square errors between the full network inference output eø c fmm and all possible approximations 60(Tm Emi 3, a, 7)'s obtained from selection combinations B = { where each & E {0, 1}. However, evaluating all possible configurations computationally too expensive, as the number of combinations grows exponentially with the model's depth L. Furthermore, this method cannot effectively handle the sequential nature of diffusion, where an error in a particular layer or timestep can propagate and affect subsequent layers well future timestep inferences.
5

---

3.4 Training Objective
To address this, we propose to learn the modulation gate score 3 = (Bhh and modulator parameters a = { and y = {nt) Recall that our goal is to find Éø bmi 3, a) that approximates the original inference ea mm: (m) with minimal computational cost. We can write this regu- larized optimization problem. To be specific, given T timesteps {tz [ - and N activation cycle (ta, fs-1...... 1a,. with caching step ls, for Im E (fs-1,.... 11-N++)
arg min [T mm 3, a. - co(Tm, 1l 8.0ay
Σ 1.1
(7)
m
where À is the regularization hyperparameter, / E (SA, CA, MLP) and = I,...,,, Although 3's need to be discrete at inference time, design to be continuous and normalized between [0, 1] using the sigmoid operation to make the computation differentiable when training. After training, we use a threshold to discretize be either or I, where / turned become modulation gate, We note that the model parameters in the diffusion transformer are frozen and only modulation gate score 3 and modulator parameters a and y are updated in our algorithm. We provide a pseudo code for training and sampling in Algorithms 1 and and illustrate an example of the sampling process
with activation cycle N 3 in Figure 2
4 Experiments
4.1 Implementation Details
Model settings. We perform experiments using popular diffusion transformers across different generation tasks, including DiT-X1/2 (35] for class-conditional image generation and PixArt-a (3]
Sana-0.6B [2 and SD3-medium [T0 for text-to-image generation. We utilize diverse sampling
methods with different number of sampling steps. DDIM [4] and DPM++ [26] are used for DiT-XL/2
and Pit-rt-a, respectively, and Flow-Euler [22] is adopted for Sana-0.6B and SD3-medium.
Training, We test multiple activation cycles A E 3........ 10]. very small number of trainable parameters.
The optimization is performed only
on the modulation gate score and modulator, leading For example, the number of parameters is 3,276 for PixArr-- with 20 sampling steps and / 3. We the training set of ImageNet 7 for DiT-XL/2 and COCO2014 [2]] for rext-to-image generation models. We train the modulation gate scores and modulator parameters on 4 H100 GPUs with AdamW optimizer
24 and learning rate 10-3 for 200K iterations.
Evaluation. For class-conditional image generation, we generate 50K images with resolution of 256 x 256 by randomly sampling IK classes ImageNet. For rext-to-image generation, randomly select 5K/30K captions from the COC02014 validation set and generate one image per caption, testing at different resolutions: 256 x 256, 512 x 512 and 1024 x 1024. We use the default classifier-free guidance for each model (e.g., 1.5 for DiT-XL/2). Unless specified, the threshold c for the modulation gate is set 0.9. To evaluate generation quality,
Frechet Inception Distance (FID), sFID [33]. Inception Score (IS) 40), precision and recall [17]. and we also report CLIPScore
13 for text-to-image generation. FLOPPs calculated using Storch-Oppounter [55], and inference latency measured on a single H100 GPU. More implementation details are in the appendix.
Table Qualitative results for class-conditional 50K image generation with DIT-XL/2 and DDIM 50 steps on ImageNet IK classes. Our method requires significantly less computation than the baseline to achieve comparable image generation quality and outperforms other approaches with similar FLOPs in terms of generation performance.
Method
FLOPs +
Speed t
FID
sFID +
IS
Precision t
Recall
DIT-XL/2 steps, bascline)
11.44
1.00x
2.29
4.32
239.54
0.81
0.60
50% steps
5.72T
1,99x
2.92
4.51
228.15
0.80
0.58
36% steps
4.12T
2.76x
3.94
5.16
216.29
0.78
0.58
FORA (W=3) 41
3.91T T
2.16x
3.28
5.72
228.47
0.80
0.56
ToCa (V=3, R=93%) 57
5.12T
1.40x
3.04
4.74
0.80
0.57
ToCa (N=4, R=93%)
4.37 T
1.49
3.60
5.11
0.79
0.56
Ours (N=3)
3.91 T
2.15x
2.90
4.60
231.54
0.80
0.58

---

Table 2: Qualitative results for text-to-image generation with PixArt-a and DPM++ (20 steps, 256x256) on COCO2014 30K valida- tion prompts. Our method significantly reduces FL.OPs compared to the baseline while slightly improving image quality and outperforms other approaches with similar FLOPs in terms FID.
Table 3: Text-to-image generation performance of SD3-medium and Sana-0.6B with Flow-Fuler sampler (20 steps, 256x256) on COC02014 30K validation prompts. Our method outperforms the naive caching method in image quality, while in- curring only marginal increase in FLOPs. The second column (Mod) indicates the percentage of layers with modulation applied.
Method
FLOPS
FID
C
PixArt-a (20 steps, baseline)
11.43
27.76
15.90
Method
Mod
FLOPs
FID J
40% steps
4.57T
28.50
16.18
SD3 (buseline)
10
11.08
23.58
FORA (N=2)
41
5.75
26.17
16.03
FORA (N=3)
0.00%
3.92T
47.27
FORA (3==) 41
4.04T
27.03
16.07
Ours (N=3)
1.50%
3.92T + 1LU1M
33.60
ToCa (3=3, R=60%) S7
6.33T
24.17
16.18
Sana (haseline)
52
4.45T
20.40
ToCa (3=3, R=90%) [77
T.26T
26.12
16.13
FORA (A==)
0.00%
1.62T
23.19
Ours (V=3)
T.04T
24.20
16.13
Ours (N=3)
1.83%
1.62 + 1180M
22.25
Table 4: Modulation across varying steps. Modu- lation improves generation quality, especially at fewer sampling steps, with a minimal increase in FLOPs. The second column (Mod) shows the percentage of layers with modulation applied.
Table 5: Large activation cycles. The proposed method achieves significantly better image gener- ation performance than DDIM with 12% steps un- der comparable FLOPs. Furthermore, our frame- work requires substantially less computation than DDIM with 18% steps in order to attain similar image quality.
Steps
Mod
FLOPs +
FID +
20
0.00%
3.22T
14.16
0,55%
+
9.44M
11.33 (-2.83)
Method
FLOPs 1
FID
50
0.00%
7.81T
3.28
DiT-XT/22 (teps)
22.89 T
2.29
0.38%
7
16.52 M
2.90 (-0.38
18% steps
4.12T
16.48
100
0.00%
15.63T
2.44
12% steps
2.74T
52.17
0.35%
15.63 T 30.67 M
2.38 (-0.66)
Ours (W=10)
2.33T
16.62
4.2 Main Results
Class-conditional image generation. Quantitative results of our framework for class-conditional image generation using DiT-XL/2 (35) are in Table [T] In comparison to the baseline DDIM 44 50
steps, our method requires 2.93x fewer FLOPs and 2. less inference latency to obtain similar generation quality. Against DDIM 36% steps, which operates with similar FLOPs, our method demonstrates substantially higher generation quality. For example, our approach achieves superior IS of 115255. Our framework also outperforms existing caching-based methods, FORA (N=3) [4] and ToCa (N==,, (3 3)
57 which have similar FLOPs, across all image quality metries.
Text-to-image generation. We show the performance of our method using PixArt-o 3 with DPM++ 26 20 steps in Table 2 To achieve comparable generation performance our method, DPM++ 20 steps, ToCa (3=3, R=60%) and FORA (W=2) require 2.83 1.57 and 1.42x x more computation, respectively. Our inference speed-up over the bascline 1.50x. Also, the proposed method outperforms + 40% steps and existing caching- based approaches, FORA (W=3) and ToCa (N=3, R=90%) in terms of FID, while maintaining similar FLOPs.
We also evaluate our method using 3D3-medium [00] and Sana-0.6B [52 with Flow-Euler [22 20- step sampler. Table B that the proposed method with 33 consistently yields better generation quality than the naive caching method (FORA), with negligible additional FLOPs.
4.3 Ablation Studies
We conduct ablation studies on class-conditions 50K image generation using DLL- 35 with sampler on ImageNet- 1K classes
DDIM 44
For ToCa, "W=3, R=93%" means that the activation cycle is 3, performing full inference once every timesteps, and for the other timesteps, on average, R=93% of tokens reuse cached values, while (1-R)=7% of tokens require network inference.
7

---

DIT-XL/2
FLOPs
300
20 steps
25
FORA
1.17T
50 stops
20
200
FLOPs 1. 35.39
FID
15
100
10
3.59
2.90
3
4
8
10
0.4 0.6
N
Gate Score Threshold
Figure 3: Modulation across activation cycles
Figure 4: Relationship between thresholds and FID. 0.9 threshold, where the majority of cached features are used without modification and small portion is modulated, is optimal.
(W). Modulation consistently enhances image generation quality, with larger N leading greater improvements.
Effectiveness of modulation. A key component of our framework lies in the use of modulation. To its effectiveness, we compare configurations with (Ours) and without the modulation (FORA (IT) We first evaluated different number of actívation cycles N E {2, 3,.3..... 10} with 50 DDIM steps. that modulation enhances generation quality, with larger activation cycles N leading to more noticeable gains. Furthermore, we investigate the effect of varying the number of sampling steps with activation cycle =/3. Table 4 improves the generation performance, with greater improvement observed when fewer steps used. These results highlight the effectiveness of modulation, as models with larger activation cycles and fewer steps tend to have lower feature similarity, making modulation-batt significant. Moreover, since the modulator is applied under % of the layers, it results negligible computational overhead.
[egure@demonstraale
shows that the use of modulation consistently
adjustments more
Recall that if the modulation gate score (3) exceeds a predefined threshold (c), the modulation is applied the cached feature; otherwise, the cached feature itself is used. In Figure we examine the relationship between thresholds and generation quality using DiT-XL/2 with 20 and 50 DDIM steps and the activation cycle 3V=.. A 0.9 threshold, where most cached features are utilized without modification and only few features are modulated (0.55% for the DDIM 20 steps and 0.38% for the DDIM 50 steps Table 4). marks the sweet spot for optimal performance. We see that when the threshold is larger (utilizing more cache without modulation), FID tends to be better compared smaller thresholds (applying modulation more). We conjecture that the modulation only needed for few features, since the similarity between features at the same depth across steps is very high (Figure []. Also, the modulation involves simple scale and translation operations, so its limited representation power may hinder generative performance if applied too extensively.
Choice of threshold.
6
Timestep
2
0.90
0.99
0.96
0.98
1.00
Gate Score Threshold
2.5
Figure 5: Various activation cycles (A) and mod- ulation thresholds (c). The optimal threshold ap- pears around 0.9 for all activation cycles. Mod- ulation becomes more effective with larger N compared to the caching-only case (c=1.00).
Block
Figure 6: Learned pattern of modulation gate scores for attention. The modulation gate scores are higher later denoising steps and deeper lay- ers, which tend to need more modulation.

---

Table 6: High-resolution generation performance of Sama-0.6B with Flow-Euler sampler (20 steps, 512x512 and 1024x 1024) COCO2014 5K validation. Our method shows better quality than the naive caching method with negligible addi- tional FLOPs. The second column (Mod) is the percentage of layers with modulation applied.
Table 7: Low-step generation performance of Sana-0.6B with Flow-Euler sampler (8 steps, 512x512) on COCO2014 5K validation prompts. Under similar FLOPs settings, our framework out- performs both fewer-step and caching- baasin ap- proaches in terms of image quality.
Method
FLOPs 4
FID
Method
Mod
FID
Sana-512 [52]
4.88 T
31.08
Sana-512 (52
12.19T
31.21
62.5% steps
3.07T T
34.45
FORA (N==)
41
0.00%
4.33
29.66
50% steps
2.44T
48.87
Ours (A=3)
1.10%
4.33 28.31 M
29.16
Sana- 1024 52]
4
29.91
L2C (N=2) 30
2.49T
32.37
FORA (V==)
0.00%
15.16T
29.45
FORA (W=2) 41
2.46T
32.46
Ours (A=33) =3)
0.55%
15.16TT + 66.06 M
29.17
Ours 22)
2.46T
31.89
Different number of activation cycles. We investigate the relationship between different activation cycles (A) and gate thresholds (c) using DiT-XL/2 with DDIM 50-step sampler. When examining performance across thresholds in coarse increments of 0.1, the best performance is observed at 0.9 (Figure 4). Therefore, we now explore the range between 0.9 and 1.0 in a more fine-grained way. Figure shows that the optimal threshold appears around 0.9 for all V E {2, 3,, 5). Similarly, as shown in Table 4 the modulation becomes more effective as the feature gap between timesteps increases (i.e.. N 1). For example, with W=5, FID decreases by more than 2 compared to the caching-only case (c=1.00), whereas with =22, the improvement is less than 1.
4.4 More Analysis
Large activation cycles. We verify the effectiveness of our framework under a large activation cycle N' setting. We use DiT-XL/2 [35 with DDIM 44 50-step sampler and set the modulation gate threshold (c) to 0.93. Table shows that DDIM with 1 88% steps achieves comparable image quality to our method with -=10, but requires 1.76x more FLOPs. DDIM at 12% steps matches the FLOPPs of our method with A-10, but yields significantly worse FID scores (+35.55). High-resolution performance. To evaluate whether the proposed modulation remains effective for high-resolution text-to-image generation, we apply it to Sana-0.6B OB (52) models generating 512 x 512 and 1024 x 1024 images on COC02014 5K validation set (21 Table shows that applying the modulation less than 2% of the layers effectively improves generation quality with negligible computational overhead. This agrees with the results Table 3
Low-step performance. We now investigate the effectiveness of our method in a low-step setting using Sana-0.6B (512 512) with 8-step Flow-Euler sampler 22 Table 7 shows that our framework
DIT-XL/2 256x226
SANA-0.6B 512x512
Baseline
Ours
Figure 7: Qualitative results. The first three images in the top row are generated by DIT-XL/2 (ItageNet, 256 x 256, 50-step DDIM) and the others by Sana-0.66 (COC02014, 512 x 512, 20-step Flow-Fuler), while the bottom shows results from our method (A = 3). Generation quality is almost identical, despite our method reduces FLOPs by 2 for DIT-XL/2 and 2.82x for Sana-0.6B.

---

outperforms fewer-step and other brching--bsedd methods under comparable FLOPs. Learned pattern of the modulation gate. We plot learned modulation gate scores / for attention features using DiT-XI/2 (DDIM 50-step, 256 x 256) under A=3. Figure o that later denoising stage features and deeper layer features tend to need more modulation (i.e., larger scores).
Qualitative analysis. We qualitatively compare the generation performance of our method and baseline methods in Figure The first three images are generated with DiT-XL/2 for ImageNet 7 classes at 256 x 256 resolution, and the other three with Sana-0.6B for COCO2014 validation prompts at 512 x 512 resolution. The top row shows baseline outputs: 50-step DDIM for DiT-XIJ22 and 20-step Flow-Fuler for Sana-0.6B. The bottom row shows our results with activation cycle / The qualitative generation performance is nearly identical, while our method reduces FLOPs by 2.83x for DiT-XL/2 (Table and 2.82x for Sana (Table 6). Additional results are in the appendix.
Limitations and Conclusion
Limitations. The core idea of the proposed method is to selectively adapt cached features through learned linear modulation for efficient diffusion transformer inference. We find applying modulation to all features leads to image quality degradation. We conjecture that the modulation is intentionally lightweight using real-valued scalars which limits its representational capacity.
Conclusion. We have introduced cache-based framework for diffusion transformers that enhances inference efficiency by reusing features and selectively adapting them through lightweight linear modulation. Experimental results demonstrate our method generally achieves similar generation quality to that of the original sampler with significantly less computation and outperforms existing approaches under similar FLOPs settings. To our knowledge, this is the first method to improve inference efficiency by approximating diffusion transformer inference through lightweight cache transformations. We hope that our work inspires further exploration of advanced caching methods.
References
[1] Fan Bao, Shen Nie, Kaiwen Xue, Yue Cao, Chongxuan Li, Hang Su, and Jun Zhu. All are worth words: A ViT backbone for diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
[2] Ziang Cao, Fangzhou Hong, Tong Wu, Liang Pan, and Ziwei Liu. Large-vocabulary 3D diffusion model with transformer. In International Conference on Learning Representations (ICLR), 2024.
[3] Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Zhongdao Wang. James Kwok, Ping Luo, Huchuan Lu, and Zhenguo Li. PixArt-a: Fast training of diffusion transformer for photorealistic text-to-image synthesis. In International Conference on Learning Representations (ICLR), 2024.
[4] Lei Chen, Yuan Meng, Chen Tang, Xinzhu Ma, Jingyan Jiang, Xin Wang, Zhi Wang, and Wenwu Zhu. Q-DiT: Accurate post-training quantization for diffusion transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025.
[5] Pengtao Chen, Mingzhu Shen, Peng Ye, Jianjian Cao, Chongjun Tu, Christos-Save Savvas Bouganis, Yiren Zhao, and Tao Chen. A-Dit: A training-free acceleration method tailored for diffusion transformers. arXiv, abs/2406.01 125, 2024. URL, htt.::: //tpx/aaa/vvv//////////eaa org/ abs/ /406. .4066001255
[6] Shoufa Chen, Mengmeng Xu, Jiawei Ren, Yuren Cong, Sen He, Yanping Xie, Animesh Sinha, Ping Luo, Tao Xiang, and Juan-Manuel Perez-Rua. GenTTro:: Diffusion transformers for image and video generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), 2024.
[7] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large-scale hierarchical image database. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2009.
[8] Prafulla Dhariwal and Alexander Nichol. Diffusion models beat GANs on image synthesis. In Advances in Neural Information Processing Systems (NeurIPS), 2021.
10

---

[9] Stefan Elfwing, Eiji Uchibe, and Kenji Doya. (eemoid-weighnn linear units for neural network function approximation in reinforcement learning. Neural networks, 107:3-11, 2018.
[10] Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhor, Zion English, and Robin Rombach. Scaling rectified flow transformers for high-resolution image synthesis. In International Conference Machine Learning (ICML., 2024.
[11] Gongfan Fang, Xinyin Ma, and Xinchao Wang. Structural pruning for diffusion models. In Advances in Neural Information Processing Systems (NeurIPS), 2023.
[12] Yefei He, Jing Liu, Weijia Wu, Hong Zhou, and Bohan Zhuang. EfficientDM: Efficient quantization-wware fine-tuning of low-bit diffusion models. In International Conference on Learning Representations (ICLR), 2024.
[13] Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. CLIPScore: reference-free evaluation metric for image captioning. In Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7514-7528, 2021.
[14] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
[15] Bo-Kyeong Kim, Myoung-Kyu Song, Thibault Castells, and Shinkook Choi. BK-SDM: Archi- tecturally compressed stable diffusion for efficient text-to-image generation. In ICML Workshop on Efficient Systems for Foundation Models, 2023.
[16] Dongjun Kim, Chieh-Hsin Lai, Wei-Hsiang Liao, Naoki Murata, Yuhta Takida, Toshimitsu Uesaka, Yutong He, Yuki Mitsufuji, and Stefano Ermon. Consistency trajectory models: Learning probability flow ODE trajectory of diffusion. In International Conference on Learning Representations (ICLR), 2024.
[17] Tuomas Kynkäänniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Improved precision and recall metric for assessing generative models. In Advances in Neural Information Processing Systems (NeurIPS), 2019.
[18] Senmao Li, Taihang Hu, Joost van de Weijer. Fahad Khan, Tao Liu, Linxuan Li, Shiqi Yang, Yaxing Wang, Ming-Ming Cheng, and Jian Yang. Faster diffusion: Rethinking the role of the encoder for diffusion model inference. In Advances in Neural Information Processing Systems (NeurIPS), 2024.
[19] Xiuyu Li, Yijiang Liu, Long Lian, Huanrui Yang. Zhen Dong, Daniel Kang, Shanghang Zhang, and Kurt Keutzer. Q-Diffusion: Quantizing diffusion models. In Proceedings of the IEEE/CVE International Conference on Computer Vision (ICCV), 2023.
[20] Yanyu Li, Huan Wang, Qing Jin, Ju Hu, Pavlo Chemerys. Yun Fu, Yanzhi Wang, Sergey Tulyakov, and Jian Ren. SnapFusion: Text-to-image diffusion model on mobile devices within two seconds. In Advances in Neural Information Processing Systems (NeurIPS), 2023.
[21] Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft COCO: Common objects in context. In European Conference Computer Vision (ECCV), 2014.
[22] Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling. In International Conference on Learning Representations (ICLR), 2023.
[23) Haozhe Liu, Wentian Zhang, Jinheng Xie, Francesco Faccio, Mengmeng Xu, Tao Xiang, Mike Zheng Shou, Juan-Manuel Perez-Rua, and Jürgen Schmidhuber. Faster diffusion through temporal attention decomposition. Transactions on Machine Learning Research (TMLR), 2025.
URL http:ttt////peeeeeeeee http://wweeettoormmm//ddxxxx/o..
[24] Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference Learning Representations (ICLR), 2019.

---

[25] Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. DPM-Solver: A fast ODE solver for diffusion probabiliittic model sampling in around 10 steps. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
[26] Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. DPM-Solver++: Fast solver for guided sampling of diffusion probabilistic models. arXiv, abs/2211.01095, 2023,
URL https::s:://rxx
org/ hbs///b//// 2211 01095
[27] Haoyu Lu, Guoxing Yang, Nanyi Fei, Yuqi Huo, Zhiwu Lu, Ping Luo, and Mingyu Ding. VDT: General-purpose video diffusion transformers via mask modeling. In International Conference on Learning Representations (ICLR), 2024.
[28] Simian Luo, Yiqin Tan, Longbo Huang, Jian Li, and Hang Zhao. Latent consistency models: Synthesizing high-resolution images with ftw-step inference. arXiv, abs/2310.04378, 2023.
URL httthhtt/a/xxiii/ org/ abs/2310.....///////.. 2310. 04378
[29] Zhengyao Lv, Chenyang Si, Junhao Song, Zhenyu Yang. Yu Qiao, Ziwei Liu, and Kwan- Yee K. Wong. FasterCache: Training-free video diffusion model acceleration with high quality. In International Conference on Learning Representations (ICLR), 2025.
[30] Xinyin Ma, Gongfan Fang. Michael Bi Mi, and Xinchao Wang. Learning- to-ccche Accelerating diffusion transformer via layer caching. In Advances in Neural Information Processing Systems (NeurIPS), 2024.
[31] Xinyin Ma, Gongfan Fang, and Xinchao Wang. DeepCache: Accelerating diffusion models for free. In Proceddings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
[32] Shentong Mo, Enze Xie, Ruihang Chu, Lanqing Hong, Matthias Nießner, and Zhenguo Li. DiT-3D: Exploring plain diffusion transformers for 3d shape generation. In Advances in Neural Information Processing Systems (NeurIPS), 2023.
[33] Charlie Nash, Jacob Menick, Sander Dieleman, and Peter W. Battaglia. Generating images with sparse representations. In International Conference on Machine Learning (ICML), 2021.
[34] Zizheng Pan, Bohan Zhuang, De-An Huang, Weili Nie. Zhiding Yu, Chaowei Xiao, Jianfei Cai, and Anima Anandkumar. T-Stitch: Accelerating sampling in pre-trained diffusion models with trajectory stitching. In International Conference on Learning Representations (ICLR), 2025.
[35] William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEF/CVF International Conference on Computer Vision (ICCV), 2023.
[36] Yiran Qin, Zhelun Shi, Jiwen Yu, Xijun Wang, Enshen Zhou, Lijun Li, Zhenfei Yin, Xihui Liu, Lu Sheng, Jing Shao, Lei Bai, Wanli Ouyang, and Ruimao Zhang. WorcdSimBench: Towards video generation models as world simulators. arXiv, abs/2410.18072. 2024. URL
https://arxiv.. org/ abs/ 18072
[37] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 10684-10695 2022.
[38] Olaf Romneberger, Philipp Fischer, and Thomas Brox. U-Net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Inter- vention (MICCAI), pages 234-241, 2015.
[39] Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. In International Conference on Learning Representations (ICLR), 2022.
[40] Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, Xi Chen, and Xi Chen. Improved techniques for training GANs. In Advances in Neural Information Processing Systems (NeurIPS), 2016.
[41] Pratheba Selvaraju, Tianyu Ding, Tianyi Chen, Ilya Zharkov, and Luming Liang. FORA: Fast-forward caching diffusion transformer acceleration. arXiv, 2407.01425. 2024. URL
https://arze org/ abs/2407 01425
12

---

[42] Xuan Shen, Zhao Song, Yufa Zhou, Bo Chen, Yanyu Li, Yifan Gong, Kai Zhang, Hao Tan, Jason Kuen, Henghui Ding, Zhihao Shu. Wei Niu, Pu Zhao, Yanzhi Wang, and Jiuxiang Gu. LazyDiiT: Lazy learning for the acceleration of diffusion transformers. In Proceedings of the AAAI Conference on Artificial Intelligence, 2025.
[43] Junhyuk So, Jungwon Lee, and Eunhyeok Park. FRDiff: Feature reuse for universal training-fin free acceleration of diffusion models. In Proceedings of the European Conference Computer Vision (ECCV), page 328-344, 2024.
[44] Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations (ICLR), 2021.
[45] Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. In Advances in Neural Information Processing Systems (NeurIPS), 2019.
[46] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations (ICLR), 2021.
[47] Yang Song, Prafulla Dhariwal, Mark Chen, and llya Sutskever. Consistency models. In International Conference on Machine Learning (ICML), 2023.
[48] Yang Sui, Yanyu Li, Anil Kag, Yerlan Idelbayev, Junli Cao, Ju Hu, Dhritiman Sagar, Bo Yuan, Sergey Tulyakov, and Jian Ren. BitsFusion: 1.99 bits weight quantization of diffusion model. In Advances in Neural Information Processing Systems (NeurIPS). 2024.
[49] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems (NeurIPS), 2017.
[50] Fu-Yun Wang, Zhaoyang Huang, Alexander William Bergman, Dazhong Shen, Peng Gao, Michael Lingelbach, Keqiang Sun, Weikang Bian, Guanglu Song, Yu Liu, Xiaogang Wang, and Hongsheng Li. Phased consistency models. In Advances in Neural Information Processing Systems (NeurIPS), 2024.
[51] Felix Wimbauer, Bichen Wu, Edgar Schoenfeld, Xiaoliang Dai, Ji Hou, Zijian He, Artsiom Sanakoyeu, Peizhao Zhang, Sam Tsai, Jonas Kohler, Christian Rupprecht, Daniel Cremers, Peter Vajda, and Jialiang Wang. Cache me if you can: Accelerating diffusion models through block caching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
[52] Enze Xie, Junsong Chen, Junyu Chen, Han Cai, Haotian Tang, Yujun Lin, Zhekai Zhang, Muyang Li, Ligeng Zhu, Yao Lu, and Song Han. SANA: Efficient high-resolution text-to- image synthesis with linear diffusion transformers. In International Conference on Learning Representations (ICLR), 2025.
[53] Shuai Yang, Yukang Chen, Luozhou Wang, Shu Liu, and Ying-Cong Chen. Denoising diffusion step-aware models. In International Conference on Learning Representations (ICLR), 2024.
[54] Xiulong Yang, Sheng-Min Shih, Yinlin Fu, Xiaoting Zhao, and Shihao Ji. Your ViT is secretly a hybrid wwwpiiiniviieeeeeaatiieeee diffusion model. arXiv, 2022. URL hattps://arxiv org/
abs 0ba/2208.0777
[55] Ligeng Zhu. thop: Pytorch-opcounter, 2018 URL Cytorch-UUCCunteer .git
[56] Chang Zou, Evelyn Zhang, Runlin Guo, Haohang Xu, Conghui He, Xuming Hu, and Linfeng Zhang. Accelerating diffusion transformers with dual feature caching. arXiv, abs/2412. 18911, 2024. URL URIphttps: //aaxiv org/ abs/ 2412. 18911
[57] Chang Zou, Xuyang Liu, Ting Liu, Siteng Huang, and Linfeng Zhang. Accelerating diffu- sion transformers with token-wing wise feature caching. In International Conference on Learning Representations (ICIR), 2025.
13

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No]. or [NA].
[NA] means either that the question Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will published with the paper.
The reviewers of your paper will be asked use the checklist as one of the factors their evaluation. While "[Yes] is generally preferable to "[No]" ", it is perfectly acceptable to answer "[No] providedd proper justification is given (e.g.. "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering binary way, we
"[No] or "[NA]" is not grounds for rejection. While the questions are phrased acknowledge that the true answer is often more nuanced, please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Yes, the claims are well supported by the experimental results.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Yes, please refer to "Limitations" section.
14

---

Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: NA, the paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: Yes, pseudocode for both training and sampling is provided to support repro- ducibility.
Guidelines:
The answer NA means that the paper does not include experiments.
15

---

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g.. with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome describe the particular way they provide for reproducibilityy In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: No, the code is not open-sourced, but the paper includes pseudocode, model links, and implementation details in the supplementary material to support accessibility,
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to to reproduce the results. See the NeurIPS code and data submission guidelines (https
//nips.cc/public/eee.c wttp://ddd//ooetutiiieeeeeeeeeee ssi ssionPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
16

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification:
Yes, detailed experimental settings are included the supplementary material.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to a level detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Yes, error bars with random seeds are reported in the supplementary material for selected experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split. initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (c.g. negative еттог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Yes, the supplementary material provides details such as training iterations and the type of GPU used.
Guidelines:
The answer NA means that the paper does not include experiments.
17

---

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: http:/ss....pbllee/hccc.....eeee
Answer:
[Yes]
Justification: Yes, the work adheres to the NeurlPS Code of Ethics in all respects.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [No]
Justification: No, This paper presents work whose goal is to advance the field of machine learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
18

---

Answer: [No]
Justification:
No, safeguards regarding potential misuse are not discussed in the paper.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging. and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: Yes, all external assets used the paper are properly credited, and their licenses and terms of use respected.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the bataasesssppprsstttt paperswi theode. com/datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is available online, the authors are encouraged reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
Answer: [NA]
Justification: NA, the paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
19

---

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: NA, the paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: NA, the paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the L.M is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: NA, the core method development in this research does not involve LLMs as any important, original, non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non non-standard components.
Please refer to our LLM policy for what should or should not be described.
20

---

A Implementation Details
A.1 Training Details
Models. In T we provide a summary that lists checkpoint URLs and license information for each model-resolution pair used in our experiments. For simplicity, we denote image resolutions of 256 x 256, 512 512, and 1024 x 1024 as 256, 512, and 1024, respectively.
Training settings. The optimization is performed only on the modulation gate and modulator: We use the training set ImageNet 77] for DiT-XL/2 [35] and COC02014 (11 for text-to-image generation models. We train the modulation gate scores and modulator parameters on 4 H100 GPUs with A optimizer [24] and learning rate 10-3 for 200K iterations. The global batch size for 256 x 256 experiments is set for 64, and the corresponding value for 512 x 512 and 1024 x 1024 experiments is 8.
Table 8: Checkpoint URLs and license information for model--reooltting pairs used in our experiments.
Model
Resolution
Checkpoint URL
License
DIT-XL/2
35
256
htt.pppa....ciia...cooo////aaoad com/DiT/models///
CC-BY-NC
PixArt-o
256
GNU AGPLv3
pth
1024
GNU AGPLv3
PixArt hiarraappaa////////m//xxxxxx----
pth
SD3-medium
256
https: https://hngiiggccco.o//taaai1i1p
CreativeML Open RAIL++MM
10
medium.inc1_clips.nnntt
Sana-0.6B
256, 512
httpa://hugllngface.ccom
NSCL v2-custom
52
main/ http:/cccccoiint//Saaecc. (ttp:/aaaan__0________000_ MattipiMMliiiiinnn.....
pth
1024
https//huugiagtacc.coo
NSCL v2-custom
main/ http///cchaatttaaaa.ooooo..oe/ep
pth
A.2
Evaluation Details
We describe the methods used for measuring FLOPs and latency, as well as the selection of the COC02014 validation
FLOPs. FLOPs were calculated by multiplying the MAC' (Multily-AAccumulats) measured with Pytorch-OpCounter [55] by 2.
Latency. We evaluated the latency of generating a batch of images using classifier guidance by conducting 10 test runs and computing the average execution time. To ensure accurate measurement, we performed 3 warm-up iterations before recording the execution time for the subsequent 10 runs. The speed metric is defined as the ratio of the baseline model's 's latency to the latency of each method. The hardware setup consists a single NVIDIA H100 GPU (80GB HBM3) and an Intel Xeon Gold 6442Y CPU. The evaluation
performed using CUDA 12.1, Python 3.10, and PoTchhh 2.2.2.
COCO validation set. COC02014 21 dataset contains sufficient captions in both train and validation sets (40K each), so we used COC02014 instead of COCO2017. For validation, we randomly select 5K/30K captions from COC02014 validation set, following the methodology outlined in the prior work [3]. Some methods such as ToCa [57] provide test results on custom 30K COCO validation set, where some prompts are extracted from the COCO2014 trainset. Since we used COCO2014 trainset for training our framework, we evaluated all methods on COC02014 validation set for fair comparison. We uploaded the list files containing 5K/30K validation prompts to anonymous Github page, and the URLs are provided in Table 9
21

---

Table COCO2014 SK/30K validation sets.
Set
Prompts
URL
COCO-5K
va12014_5k.. 5k.
COCO-30K
30,000
30k.
A.3
Algorithms for Flow Models
now provide the training and sampling algorithms of the proposed framework for flow-based
models 22
Flow models. We define ia linear interpolation between data point from the data distribution
To ~ ((z))) and noise point T1 ~ [0 I). The velocity Ue is the direction from the noise to the data point:
(1 t + tzo and
U2
(8)
Flow models learn neural network to predict the conditional expectation 6, Elu, xxy.. To sample from a flow model, a random noise Iı is iteratively updated to zy using a denoising ODE with the learned flow model (0 t). This iterative step is often approximated using Euler sampling small discrete time intervals.
Training and sampling. Algorithms | and 2 apply the proposed method to diffusion models with DDIM 44 and DPM++ (26) samplers. The corresponding versions adapted for flow-based models are presented in Algorithms and While the samplers differ, the core idea, caching and selective modulation, is the same. the algorithms, we denote by üg the approximation of the original flow model (6g) obtained from our method.
Algorithm 3 Training
Algorithm 4 Sampling
I: Input: Data distribution g(-), flow model ()(-), optimizer. ODE solver ()).. total steps T,, the
1: Input:
Flow model (-(-), modulation gate score B. modulation parameters a, 7. ODE solver (r()). total steps T, the step schedule {t, ( (-),, activation cycle / and threshold
step schedule {t, 1 (-) and activation ey-
cle NV
2: Randomly initialize and
2: Compute r (T 1)%%
3: Compute (T' 1)%,
3: Ør-1 [0,0 I)
4: repeat
4: for == T -- to do
3o g(20) and ør-1 (O.
for / in (S.A, CA, MLP)
do
6:
+ N n + +r, n 4/0, (T)//T
6:
if 8%N = r then Calculate f.(; and cache A.(st.y's Eq (
# Step L. for calculating states for caching
7:
x, = (1 t. -T + tg
8:
v. - c0(2., t.) and cache (. in
# Inference and cache features
8:
else
9:
d~u
9:
gl. - Sigmoid(
10:
<- 8
10:
B.. - (0)
# Step fm for using cached states
11:
Calculate P. (() by Eq
11:
im + (VV,, to, fm)
12:
end if
12:
Jm+il (1 bm ) + 4m00
13:
13:
Um+1 +- (2 lm+1)
14:
+ (1m++1, fm+11 tm)
14:
if
8%NV = then U. +- (o(x., (.) Original inference
3 + Sigmoid(33)
15:
# Optimize
16:
Caleulate (a tmi a, by Eq with
16:
else
Eq C
17:
Us + ūo(ās, Ls; B, 7) # Use cached features
17:
15 tm.) To(xms tm 11 +
18: end
18:
Compute gradients for B, a, y with respect to loss C and update them with optimizer until converged
19: Ts-1 <- tassttatto-
20: end for
19:
21: return 15o
22

---

B Additional Analysis
B.1 Effect of Random Seed
Table D presents quantitative results for class-conditional 50K image generation on ImageNet-IK
using DiT-XL/2 (35] with DDIM +4 (50 steps). To assess statistical significance, we report mean + standard deviation over multiple runs with different random seeds {2, 55, 777} for our method, and the results are compared with those of the naive caching baseline (FORA). Table TO shows that the proposed framework consistently achieves higher generation performance than the nave caching baseline across most metrics.
Table 10: Qualitative performance for class-conditional 50K image generation with DiT-XL/2 and DDIM 50 steps on ImageNet IK classes. Our results, reported mean + standard deviation over multiple random seeds, show that the proposed framework consistently achieves higher generation performance than the naive caching baseline across most metrics.
Method
FID
sFID
IS f
Precision f
Recal
FORA (W=3) 41
3.28
5.72
228.47
0.80
0.56
Ours (N=3)
2.9020.0677
4.60=0.093
231.54#1.444
0.80=0.0039
0.58#0.0045
B.2 Effectiveness of Modulation
We now compare the performance the proposed modulator (threshold 0.9) against the cache-only baseline, using PixArt-a 3 and 20-step DPM++ [26] sampler with activation cycles / E {2. 4). This comparison extends the experiment shown in Figure 3 to a different architecture. Table 11 shows that the of modulation consistently improves the generation performance, with more pronounced effect observed in the case larger N. This highlights the effectiveness of modulation, since the model with larger A has a greater time spacing between features for the same sampling steps. Consequently, feature similarity decreases, making adjustments through the modulation more important. Moreover, since the modulator is applied to fewer layers ($2.02%) this results in minimal increase computation.
Table 11: Effectiveness of modulation. Modulation improves generation quality, especially at larger activation cycles M, with minimal increase FLOPs. The second column (Mod) shows the percentage of layers with modulation applied.
N
Mod
FLOPs 1
FID
0.00%
5.75T
26.17
2
2.02%
5.75T+40.11M,
24.03
0.00%
4.04T
27.03
3
1.28%
4.04T+33.03M
24.20
0.00%
2.90T
31.72
4
0.79%
2.90T+23.59M
25.63
23

---

B.3 Various Activation Cycles
We explore the relationship between different activation eycles (N) and the modulation gate threshold (c) with DPM++ [26] 20-step PixArt-o: [3]. This comparison extends the experiment shown in F to different architecture. Figure g shows that the optimal threshold appears around 0.9 across all combinations similar to the results observed in Figure 5 Also, the modulation is more effective as the feature gap between timesteps increases (i.e., with larger A). For instance, with W=4, FID decreases by more than 6 compared to caching-only baseline, whereas with N=2, the improvement around
35
N'=3
30
25
20
Gate Score Threshold
Figure 8: Various activation cycles (NV and gate score thresholds. The optimal threshold appears around 0.9 for all activation cycles. The proposed modulation becomes more effective with larger A compared to naive caching (hressold=1.000)
B.4 Robustness Across Different Sampling Methods
We assess the robustness of the default threshold under various sampling methods using PixArt-a ( Recall that we utilize the default sampling method DPM++ (26) for PixArt-a (256 x 256) and the threshold for modulation gate is
0.9. To demonstrate the robustness, sampler to Art-ra. We use the 20-step model with three activation cycles (A) and report results on COC02014 21 30K validation set. Table 12 shows that with the threshold fixed, modulation consistently improves FID performance, and the improvement becomes more pronounced as V increases. This trend is observed both DPM++ and DDIM.
also apply DDIM [44
Table 12: Robustness of the default threshold across different sampling methods. With the threshold fixed, modulation consistently improves FID performance across different samplers, with larger gains observed for larger activation cycles (N).
N
Mod
FLOPs
FID (DPM++)
FID (DDIM) +
0.00%
5.75T
26.17
28,30
2.02%
5.75T+40.11M
24.03
27.13
0.00%
4.04T
27.03
29.37
1.28%
4.04T+33.03M
24.20
27.37
0.00%
2.90T
31.72
34.65
0.79%
2.955M
25.63
25.48
24

---

B.5 Nonlinear Modulation Functions
We further explore lightweight nonlinear modulators based on common activation functions such as SiLU [9], sigmoid. hyperbolic tangent (tanh) and softplus, Details of the modulator formulations are found in Table [3]] where denotes the cached feature and and b are trainable scalar parameters. We trained a and b under similar setups to the proposed linear modulator (Appendix A). For evaluation, we used the configuration from Table T DiT-X1.22 [5 with a 50-step DDIM sampler 44 and an activation eycle N = 3, comparing performance of linear and nonlinear modulators. Table 13 reports 50K image generation performance on ImageNet- K 7 classes. All linear and nonlinear modulators outperformed the naive caching baseline FORA [1I]. which applies modulation. However, observed some variation across different nonlinear function types. Among them, the SiLU-based modulator achieved performance comparable to the proposed linear variant. These results indicate that the effectiveness of nonlinear modulation depends on the choice of function, supporting our use of simple yet effective linear design with lower computational overhead. Although experimented with diverse combinations (c.g., initialization, learning rate, threshold, and training iteration) for each nonlinear modulator and reported the best result, further hyperparameter tuning could potentially yield better performance.
Table 13: Results of class-conditional 50K image generation ImageNet-IK classes using DiT-XL/2 with 50 steps using different nonlinear modulation functions. Table (a) shows pseudo code of modulation functions. Table (b) demonstrates that all modulators outperformed the naive caching baseline, with SiLU performing on par with the linear variant, supporting the use of our linear design with lower computational overhead.
(a) Formulations modulation functions.
Method
Pseudo Code
Linear
SiLU [9]
silu(x)
SoftGate
sigmoid(a
b)
Tanh
tanh
Softplus & Log
log (clampp (pootpllus() (a)
1e-6)
x, min-1e-6, max-1e4))
Results of classtconditional generation.
Method
FID
sFID +
IS
Precision
Recall
DT2-XL/2 (50 steps, baseline)
2.29
4.32
239.54
0.81
0.60
FORA (no modulators) [1
3.28
5.72
228.47
0.80
0.56
Ours (linear)
2.90
4.60
231.54
0.80
0.58
SiLU [9]
2.99
4.74
230.22
0.79
0.58
SoftGate
3.00
4.78
231.93
0.80
0.57
Tanh
4.66
226.21
0.79
Softplus & Log
3.25
5.73
229.53
0.80
0.56
25

---

B.6 Validation under Low-Precision Settings
We validate the effectiveness of our framework under low-looeeee low-precision setting (8-bit weights and 16-bit activations), using DiT-XL/2 [35] with 50-step DDIM sampler [44], activation cycle N = 3 and modulation gate threshold 0.90. To obtain FP8 weights, we apply basic channel-wise symmetric quantization. Following the setup in Table [ class generation task. shown Table 14 compared to DDIM 50%-steps with similar generation quality, our method requires 1.46 fewer FL.OPs. Moreover, compared to FORA [1 and DDIM with 36% of the steps (which incur similar FL.OPs), our method achieves better image quality. These results demonstrate that our method remains effective even under low- ww-precision settings, not just in full-precision (32-bit) scenarios.
we evaluate performance the 50K ImageNet 7 1K-
Table 14: Validation under low-precision settings (FP8 weights and FP16 activations) for class- conditional 50K image generation with DiT-XL/2 and 50-step DDIM sampler on ImageNet-IK classes. Our method reduces FLOPs compared to fewer-step sampler at similar quality, achieves better image quality than naive caching and fewer-step samplers under similar FLOPs.
Method
FLOPs
FID |
sFID
IS
Precision
Recall f
DiT-XL/2 (50 steps, baseline)
11.44
2.46
4.70
237.76
0.80
0.60
50% steps
5.72T
3.18
5.30
231.12
0.79
0.58
36% steps
4.12T
4.37
6.43
214.37
0.77
0.56
FORA (W=3) 41
3.91
3.66
7.09
224.34
0.79
0.55
Ours (N=3)
3.91
3.22
5.13
224.87
0.79
0.57
26

---

B.7 Feature Similarity Analysis under Consistency Models
To explore the potential of combining our method with consisteney models, we analyze feature similarity in PixArt-a [3] with LCM sampler [28]. Specifically, we compute cosine similarity between feature pairs sampled at timestep interval of A 2 for 4-step generation, and intervals of W = 2 and N = 3 for 8-step generation. The analysis is conducted on self-attention, cross- attention, and MI.P layers, where our method is applied, and we focus on low-step settings and 8 steps), as LCM is a step distillation method that reduces the number of inference steps through full-network finetuning. The resulting distributions, shown as histograms similarity across timesteps. This observation suggests that when using LCM sampler, features vary significantly between steps, which may limit the effectiveness of our caching-bas method. In line with our observation and conjecture, applying caching-based method, D-DiT [5], to PixArt-cx with LCM resulted minimal effectiveness (Table 3 in [
layers,
Figure | reveal low
Self-Attention
Cross-Attention
MLP
step 4, = 2
Parcentage (%)
40
40
40
Percentage
Percentagg
20
20
20
1.25
0.5
0.75
Coeine Similrily
Cosine Similarity
Cosine Similarity
step 8, N = 2
Perceniage (%)
40
Percontage (%)
40
Percentage
30
20
20
10
10
10
0.20
0.5
0.75
0.25
0.5
0.75
0.
25
0.5
0.76
Cceine Simiartty
Cosine Similartty
Cosine Similarity
50
50
step 8, N 3
Percentage (%)
Percontage (%)
Percentage (%))
30
80
80
1.0
10
10
0.25
0.5
0.76
0.25
0.5
0.75
0.
25
0.5
0.75
Simiarity
Cosine Similarity
Cosine Similarity
Figure 9: Visualization of self-attention, cross-attention, and MLP features in PixArt-a with LCM sampler. The top three histograms show the cosine similarity distribution between feature pairs sampled at two-timestep interval in the 4-step model. The middle three histograms depict the distributions for the 8-step model with two-timestep interval. The bottom three plots present the distributions for the 8-step model with a three-timester interval. Across all cases, the distributions indicate low cosine similarity between features across timesteps.
27

---

C Further Comparison with Prior Methods
We report additional evaluations of prior eaching-based methods under their primary experimental settings, which could not be included in Section due to differences in configurations.
C.1 Comparison with Various Methods under N
We compared the proposed framework with existing efficiency improvement methods when the activation eycle is = meaning the network performs full network inference and save intermediate features every two steps. Table [5 shows results for class-conditional 50K image generation using DiT- XL/2 [35] with DDIM [44] 50-step sampler on ImageNet IK classes [7]. Our method demonstrates superior generation performance compared other techniques with similar FLOPs, such 52% steps, 50% steps, L2C (35 steps) 30 LazyDiT 42 and FORA 41). Additionally, our framework achieves similar image quality with approximately 50% of the FLOPs and 1.68x inference speed required for the original DDIM 50-step inference. Latency for all methods are measured on an H100 GPU to evaluate the speed-up. However, since LazyDiT's pretrained model not publicly available, its latency is omitted in the table. LazyDiT is expected to have similar inference latency to I.2C, it similarly alternates between full inference and caching.
Table 15: Results for activation cycle N Our method outperforms other efficiency improvement techniques in generation performance at comparable FLOPs. The proposed framework requires almost 50% computation to achieve similar image quality compared to the baseline DiT-XL/2 50-step sampling.
Method
F1.OPs
Speed f
FID
sFID
IS
Precision
Recall f
DiT-XL/2 (50 steps, baseline)
11.44
1.00x
2.26
4.29
238.60
0.80
0.60
52% steps
5.95T
1.92x
2.82
4.49
232.22
0.80
0.59
50% steps
5.72T
1.99x
2.85
4.54
231.20
0.80
0.58
T,2C (50 steps) 30
8.72T
1.24x
2.27
4.23
244.10
0.81
0.59
L2C (35 steps)
5.92T
1.81x
2.92
4.60
229.52
0.79
0.59
LazyDiT 42
5.74T
2.70
4.47
237.03
0.80
0.59
FORA A1
5.74T
1.68x
2.61
4.69
236.14
0.81
0.58
ToCa (R=93%) 57
6.77T
1.25x
2.92
4.51
225.77
0.79
0.58
Ours
5.74T
1.68
2.43
4.57
236.63
0.80
0.59
28

---

C.2 Comparison with DuCa
In Table T we compared our method with various existing caching strategies under the class- conditional 50K image generation setting using DiT-XL/2 [35] with DDIM [44] (50 steps) on ImageNet- IK 77. Additionally, we include comparison with DuCa [56]. a recently proposed token- wise caching method in Table [6] DuCa reports results under multiple configurations. DuCa(a) and DuCa(b) differ in the placement of caching steps: DuCa(a) inserts aggressive caching immediately after the fresh step, while DuCa(b) places conservative caching at the earliest position. S-A score denotes token selection based on self-attention scores, and V-norm uses vector norms for caching. Among the DuCa variants, DuCa(b) (S-A) and DuCa(b) (V-Caching) achieve FID scores comparable to ours, but both require x.23x more FLOPs than our method. While results for several configurations reported in DuCa, the current implementation" lacks sufficient documentation clarification of specific experimental settings, making difficult to measure certain metrics such as runtime Inception Score (IS) at the time of submission. Accordingly, we omit these entries from comparison table.
Table 16: Qualitative results for class-conditional 50K image generation with DiT-X1/2 and DDIM 50 steps ImageNet 1K classes. Our method requires significantly less computation than the baseline to achieve comparable image generation quality and outperforms other approaches with similar FLOPs in terms of generation performance.
Method
FLOPs
+
Speed
FID
sFID
IS 1
Precision t
Recall
D (50 steps, baseline)
11.44
1.00
2.29
4.32
239.54
0.81
0.60
50% steps
5.72T
x.99
2.92
4.51
228.15
0.80
0.58
36% steps
4.12T
2.76x
3,94
5.16
216.29
0.78
0.58
FORA (N=3) 41
3.91 T
2.16x
3.28
5.72
228.47
0.80
0.56
ToCa (N=3, R=93%)
57
5.12T
1.40
3.04
4.74
0.80
0.57
ToCa (W==, R=93%)
57
4.37 T
x.49
3.60
5.11
0.79
0.56
DuCa(a) (S-A score)
[56
4.38T T
3.19
4.66
0.79
0.57
DuCa(b) (S-A score)
[56
4.79T
3.05
4.66
0.80
0.57
DuCa(a) (V-Caching)
[6
4.38T
3.19
4.68
0.80
0.57
DuCa(b) (V-Caching)
[56]
4.79T
3.06
4.69
0.80
0.57
Ours (N=3)
3.91T
2.15x
2.90
4.60
231.54
0.80
0.58
C.3
Comparison with T-GATE
To ensure fair comparison with T-GATE (23), we evaluated our model using similar number of MACs to those reported for their PixArt-a 25-step model in Table of [23]. This comparison
performed on COCO-10K validation prompts Table 17 shows that model achieves significantly better generation performance and faster inference speed.
Table 17: Qualitative results for eext-to-image generation using PixArt-a with DPM++ (25 steps, 1024 1024) on COCO-IOK validation prompts. Our method outperforms T-GATE in image quality and latency under comparable MAC settings.
Method
Latency (
FPS
MACs
FID
PixArt-a (25 steps, baseline)
3
1.43
0.70
107.031
38.67
T-GATE (3
0.62
1.63
64.14 T
38.42
Ours (W=4)
0.53
1.91
60.03 T
32.77
29

---

D Comparison with Additional Metrics
This section presents the complete results of several experiments for which performance on various metrics could not be reported in Section due to space constraints.
D.1 Results on SD3 and Sana
In Table evaluated our method using SD3-medium [10] and Sana-0.6B 52 with Flow-Fuler
[2 20-step sampler. We additionally report CLIP score [33 and inference speed in Table [8] We
that the proposed method with activation cycle N generally yields better generation quality than the naive caching method (FORA), with negligible additional FLOPs and inference latency.
Table 18: Text-to-image generation performance of SD3-medium and Sana-0.6B with Flow-Eule sampler (20 steps, 256 x 256) on COC02014 30K validation prompts. Our method generally outperforms the naive caching method in image quality, while incurring only marginal increase FLOPs and inference latency. The second column (Mod) indicates the percentage of layers with modulation applied.
Model
Mod
FLOPs
Speed
FID
CLIP
SD3 (baseline)
10
11.08T
1.00x
23.58
15.81
FORA (W=3)
0.00%
3.92T
1.56x
47.27
15.60
Ours (N=3)
1.50%
1.900
1.54x
33.60
15.55
Sana (baseline)
4
1.00x
20.40
16.19
FORA (A=3)
41
0.00%
1.62
1.74x
23.19
16.15
Ours (N=3)
1.83%
1.62 + 1L80M
1.73x
22.25
16.18
D.2 Results on High-resolution
Table reports results high-resolution text-to-imaae generation, where we apply framework
Sana-0.6B [52) for generating 512 x 512 and 1024 x 1024 images on the COC02014 5K validation
(21). We also provide sFID, IS [40], Precision, Recall [17 and CLIP score [3 Table 19 Our results show that applying the proposed modulation to fewer than 2% of the layers generally improves generation quality across most evaluation metrics, while incurring negligible computational overhead.
Table 19: High-resolutinn performance of Sana-0.6B with Flow-Euler sampler (20 steps, 512 x 512 and 1024 1024) on COCO2014 5K validation prompts. Our method generally shows better image quality than the naive caching method with negligible additional FLOPs. The second column (Mod) indicates the percentage of layers with modulation applied.
Model
Mod
FL.PPS
FID.,
sFID
1S
Precision
Recall
CLIPT
Sana-512 (baseline) [52]
12.19T
31.21
72.28
39.62
0.64
0.34
16.11
FORA (V=3) 4
0.00%
4.33
29.66
80.73
39.52
0.59
0.35
16.20
Ours (A'=3)
1.10%
4.33 28.31 M
29.16
79.48
40.19
0.60
0.36
16.22
Sana- 1024 (baseline)
4
29.91
70.00
42.38
0.61
0.39
16.14
FORA (N=3)
0.00%
15.16T
41.05
0.55
0.39
Ours (N=3)
0.55%
15.16 T 66.06.666
29.17
81.54
41.33
0.55
0.40
16.28
30

---

E
Additional Figures
E.1
Scatter Plots
In addition to Figure ! we show more scatter plots to visualize feature values of attention layers at two-timestep intervals Figure I0 In the early denoising stage (larger timestep and smaller block index), features tend to exhibit high linearity between two-timestep intervals, whereas the linearity decreases as the process progresses to the later stage (smaller timestep and larger block index).
1st block
9th block
18th block
27th block
step 2 VS
-00
step 13 vs. 11
-000
1202
2ixek A5
Aivionnde Ao 33
-200
-201 value Exp.
step 31 VS. 39
step 49
-0
Figure 10: Scatter plots for visualizing feature values of attention layers at two-timestep intervals.
31

---

E.2 Learning Pattern of Modulation Gate
Due space constraints, only the final heatmap of modulation gate scores for attention is included in Figure 6 We plot how the modulation gate scores 5 change as the training progresses in Figure IT As gate scores are randomly initialized, they exhibit no distinct structure in the early phase of training. However, as the training progresses, the scores are separated. In both attention and MIP,, modulation gate scores are higher later denoising steps and deeper layers, which tend to need more modulation. Additionally, for MLP features, gate scores are large in the shallow layers.
Train Inerations 50,000
Train Ieeations 100,000
Train Iterations 150.000 000
1.0
12
12
Timezee (4)
18
18
24
0.5
30
30
3
42
43
13
17
21
25
13
17
21
25
13
21
25
Block Depth
Black Depth
Block Depth
Train Iterations 50,000
Train Iterations 100,000
Train Iterations 150 000
1.0
12
18
18
Timestep
24
24
30
10
36
42
42
463
46
0.0
13
25
13
17
21
Depth
Depth
Depth
Figure 11: Learning pattern of modulation gate scores for attention (top) and MLP (bottom). There is no clear pattern during the initial phase of training. As training progresses, gate scores for both attention and MLP show increase in values for later denoising steps and deeper layers, indicating need for modulation in those parts. Additionally, for MLP features, gate scores tend to be larger in the shallow layers.
32

---

E.3
Generated Images
To demonstrate qualitative results of our method, we apply both our approach and caching-on only method (FORA [4T]) to DiT-XL/2 [35] with DDIM 44 (50 steps, baseline) and an activation cycle of Figure 12 show 256 256 resolution generations for five ImageNet 7 classes. Overall, FORA tends to produce blurrier regions, while our method preserves higher sharpness. In the keeshond example, FORA distorts the appearance of the eyes. the car wheel image, the orange paint appears faded. For bullffro,, the fine texture details of the rock are lost. In the case of nail, part of the head missing, and in green mamba, the entire image appears noticeably more blurry.
class
DiT-XL/2 (50steps, baseline)
Ours LV 3, modulation)
FORA (N 3, cache only)
keeshond
wheel
bullfrog, Rana catesbeiana
nail
green mamba
Figure 12: Qualitative comparison of baseline DIT-XL/2, FORA, and our method on ImageNet.
33

---

We further qualitatively compare the generation performance of our method and baseline method in Figure [3] The images in the top row are generated by Sana-0.6B [52] (COC02014 [2T]. 512x5 12. 20-step Flow-Fuler (22). while the bottom shows results from our method (AV = 3). Generation quality is almost identical, despite our method reduces FL.OPs by 2.82x.
A person standing with broken umbrella.
A man sitting on his surfboard
A man wearing glasses and blue sweater.
Prompt
looking out into the
Baseline
Ours
Figure 13: Qualitative comparison of baseline Sana-0.6B and our method on COC02014.
34


Reviews:
[{"review_id": "m6BflFNBVw", "reviewer": "Reviewer_aP9D", "summary": "The paper introduces a selective caching framework that speeds up diffusion transformers by recycling intermediate features that look alike across neighboring timesteps. When a cached activation is sufficiently close to what the network would recompute, the model skips the full block and, at most, applies a learned scale-and-shift modulation. A tiny gate, trained with the rest of the transformer frozen, decides when this lightweight adjustment is needed. Because fewer than two percent of blocks receive modulation, the extra operations are almost free, and the approach can be inserted into existing samplers without architectural changes.\n\nTests on several diffusion transformers confirm the efficiency gains. With DiT-XL/2 the method cuts FLOP by about 3× and wall clock time by 2×, yet keeps FID, sFID, and IS comparable. Similar savings are observed on PixArt-α, SD3-medium, and Sana-0.6B, including high-resolution 1024 × 1024 synthesis. Across these models it consistently surpasses prior caching techniques such as FORA and ToCa, and it outperforms simple step-reduction baselines at matched compute budgets.", "questions": "N/A.", "limitations": "N/A.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength:\n1. Across four backbones the cache scheme cuts floating point operations and wall clock time, yet FID, sFID, and IS stay comparable of the original samplers at all tested resolutions.\n2.  This method is easy to integrate. Only a light gating head is trained while the diffusion transformer stays frozen, so existing checkpoints can adopt the method without any retraining of the large backbone.\n3. The experiments are comprehensive and solid. The authors evaluate DiT-XL/2, PixArt-α, SD3-medium, and Sana-0.6B at 256, 512, and 1024 pixel outputs, report both FLOPs and wall clock time, and run ablations on cache depth, gate threshold, and modulation cost. Results are matched to the compute budget of ToCa, FORA, and step-reduction baselines, which makes the gains in speed and quality easy to verify.\n\nWeakness:\n1. The method section could be made more accessible.\n2. The ablation study uncovers a counter-intuitive trend: when the gate threshold is set low and many blocks are modulated, FID actually increases and image quality drops. This unexpected behaviour shows that the method is sensitive to the threshold and that excessive modulation can harm, rather than help, performance.", "was_revised": true, "final_justification": "This is an interesting method and is effective among several backbone models. I don't give higher scores because that it can only be applied to a small fraction of the blocks, making this method not generally superior. Nevertheless, I believe the quality of this paper is good and it is suitable for NeurIPS.", "rebuttal": "We sincerely appreciate the thoughtful feedback. We address the comments below.\n\n**W1&W2.** Our framework aims for efficient inference by approximating full diffusion transformer computation through caching and lightweight modulation, leveraging the strong linear relationship between features at consecutive timesteps. As noted in Line 136, the naive caching mechanism alone provides decent generative performance, so we introduce a modulation gate to selectively apply modulation. We adopt a linear modulator based on the observed linear relationship, and deliberately keep it lightweight to maintain efficiency, which naturally limits its expressiveness. As a result, excessive application has been observed to degrade generative performance (Line 229). However, as shown in Figure 5, in the range $[0.90, 0.93]$ close to the reported threshold of 0.9, the FID metric varies by no more than 0.1 across activation cycles of 2, 3 and 5, suggesting that the performance remains reasonably consistent under different threshold settings. We also thank the reviewer for pointing out the clarity issue in the method section. In the final version, we plan to improve accessibility by adding a clearer high-level description and enhancing the illustrations."}, {"review_id": "E4tHOKLcEt", "reviewer": "Reviewer_3Fgr", "summary": "The paper proposes a cache-based framework for diffusion transformers that improves inference efficiency by reusing features and selectively modulating them through learned linear transformations, achieving comparable generation quality with significantly reduced computational cost.", "questions": "As an acceleration method based on the caching approach, FasterCache[1] also explores the estimation of feature evolution between adjacent time steps. What are the relationships and differences between our method and theirs?\n\n[1] FasterCache: Training-Free Video Diffusion Model Acceleration with High Quality", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The manuscript is well-written, exhibiting a clear and coherent structure that facilitates comprehension.\n2. Comprehensive and effective experiments: The method is validated across multiple models and datasets, outperforming SOTA caching methods like FORA and ToCa with comparable or lower computational cost.\n3. Building on existing feature caching methods, the paper introduces a novel linear modulation scheme that goes beyond the traditional choice between caching and full inference, enabling more flexible feature reuse.\nWeakness:\n1. Increased pipeline complexity: Although the training overhead is small, the introduction of modulators and gating mechanisms adds extra training and threshold tuning compared to the heuristic caching method.\n2. Limited modulation capacity? Applying modulation to all layers degrades quality, possibly due to the limited expressiveness of linear transformations. Could stronger modulators like MLPs or attention improve performance in more complex scenarios", "was_revised": true, "final_justification": "The authors have addressed my concerns. I keep my initial rating.", "rebuttal": "We truly appreciate the constructive suggestions. We provide responses to each comment below. \n\n\n**Q1**. While FasterCache [1] also utilizes modulation mechanism, it combines two recently cached features using a deterministic weighting function that is adaptively adjusted by timestep. In contrast, our method relies on only a single cached feature for modulation. Also, FasterCache differs in its application of classifier-free guidance, where it caches the high- and low-frequency biases between conditional and unconditional outputs using Fourier transformation—an approach not adopted in our framework. Overall, while our method requires training a lightweight modulator, it remains more efficient than FasterCache in terms of memory at runtime inference. In the final version, we will properly cite [1] in the related work section, and clearly articulate the differences in motivation and design between our method and theirs. \n\n---\n\n**W4**. We agree that modulators introduce some overhead. However, heuristic approaches like ToCa [2] also require careful hyperparameter tuning, as they rely on multiple criteria to decide which feature to cache. \n\n---\n\n**W5**. The proposed framework aims to approximate full diffusion transformer computation through caching and selective modulation for efficient inference. To minimize inference overhead introduced by the modulators, we adopt lightweight linear transformations. Since the modulation is applied to features that are already the outputs of attention and MLP layers, we chose not to use additional MLP- or attention-based modulators. \n\n---\n\n**References**\n\n[1] Lv, Z., et al. FasterCache: Training-free video diffusion model acceleration with high quality. In ICLR, 2025\n\n[2] Zou, C., et al. Accelerating diffusion transformers with token-wise feature caching. In ICLR, 2025"}, {"review_id": "GBFKXC0kKv", "reviewer": "Reviewer_T1S7", "summary": "The study presents an approach that accelerates diffusion transformer inference by storing intermediate features and conditionally adjusting them using lightweight, learned linear mappings, all while maintaining generation quality and significantly cutting computational demands.", "questions": "See Weakness", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. The method achieves a substantial reduction in FLOPs and inference latency (up to ~2.9×), while preserving generation quality across different diffusion transformers (e.g., DiT-XL/2, PixArt-α, SD3-medium), which demonstrates its strong generalizability and practical utility.\n\n2. The authors provide detailed studies on modulation thresholds, activation cycles, and learned gate patterns, offering useful insights into how and where modulation is most effective (e.g., later timesteps, deeper layers)\n\n3. The paper is clearly structured and effectively organized, making its content easy to understand and follow.\n\n\nWeakness&Question:\n\n1. While linear modulation is efficient, the paper does not explore or compare stronger alternatives like lightweight nonlinear modules, which might offer better accuracy-efficiency trade-offs.\n\n2. Can this method be integrated with other diffusion model acceleration techniques, such as model distillation and low-precision arithmetic?", "was_revised": false, "final_justification": "", "rebuttal": "We greatly appreciate the insightful comments. Please find our responses to the comments below.\n\n**Q1**. We further explored lightweight nonlinear modulators based on common activation functions such as SiLU [1], sigmoid, hyperbolic tangent (tanh) and softplus. Details of the modulator formulations are found in Table A, where x denotes the cached feature and a and b are trainable scalar parameters. We trained a and b under similar setups to the proposed linear modulator (Appendix A.1). For evaluation, we used the configuration from Table 1: DiT-XL/2 [2] with a 50-step DDIM sampler [3] and an activation cycle $\\mathcal{N}=3$, comparing performance of linear and nonlinear modulators. Table B reports 50K image generation performance on ImageNet-1K classes. All linear and nonlinear modulators outperformed the naive caching baseline (FORA [4]), which applies no modulation. However, we observed some variation across different nonlinear function types. Among them, the SiLU-based modulator achieved performance comparable to the proposed linear variant. These results indicate that the effectiveness of nonlinear modulation depends on the choice of function, supporting our use of a simple yet effective linear design with lower computational overhead. Although we experimented with diverse combinations (e.g., initialization, learning rate, threshold, and training iteration) for each nonlinear modulator and reported the best result, further hyperparameter tuning could potentially yield better performance. We thank the reviewer for the insightful suggestion and will incorporate these experiments into the final version of the paper.\n\n[Table A]\n| Method      | Pseudo Code                                                 |\n| --------  | ---------------------------------------------------- | \n| linear    | a * x + b                                            |\n| SiLU  | a * silu(x) + b                                    | \n| softgate  | sigmoid(a*x+b) * x                                   |\n| tanh      | a * tanh(x) + b                                      |\n| softplus&log   | log(clamp((softplus(a) + eps) * x, min=eps, max=1e4)) + b | \n\n※ eps = 1e-6\n\n---\n[Table B]\n\n\n| Method                        | FID↓   | sFID↓  | IS↑      | Precision↑ | Recall↑ |\n|-------------------------------|-------:|-------:|---------:|-----------:|--------:|\n| DiT-XL/2 (50 steps, baseline) | 2.29  | 4.32  | 239.54  | 0.81      | 0.60   |\n| FORA (no modulatiors)          | 3.28  | 5.72  | 228.47  | 0.80      | 0.56   |\n| Ours (linear)                 | 2.90  | 4.60  | 231.54  | 0.80      | 0.58   |\n| SiLU                      | 2.99  | 4.74  | 230.22  | 0.79      | 0.58   |\n| softgate                      | 3.00  | 4.78  | 231.93  | 0.80      | 0.57   |\n| tanh                          | 3.13  | 4.66  | 226.21  | 0.79      | 0.58   |\n| softplus&log                       | 3.25  | 5.73  | 229.53  | 0.80      | 0.56   |\n\n\n---\n\n**Q2. (Model distillation)** The proposed framework does not require updating network weights; instead, only the modulators and routers are trained, making it generally applicable to model distillation. However, as discussed in Appendix B.5, its effectiveness may be limited in scenarios where feature similarity between adjacent timesteps is low—such as the latent consistency model [5] we tested.\n\n**(Low-precision)** We validate the effectiveness of our framework under a low-precision setting (8-bit weights, 16-bit activations), using DiT-XL/2 [2] with a 50-step DDIM sampler [3], activation cycle $\\mathcal{N}=3$ and a modulation gate threshold of 0.90. To obtain FP8 weights, we apply basic channel-wise symmetric quantization. Following the setup in Table 1, we evaluate performance on the 50K ImageNet 1K-class generation task. As shown in the table below, compared to DDIM 50%-steps with similar generation quality, our method requires 1.46× fewer FLOPs. Moreover, compared to FORA [4] and DDIM with 36% of the steps (which incur similar FLOPs), our method achieves better image quality. These results demonstrate that our method remains effective even under low-precision settings, not just in full-precision (32-bit) scenarios. We appreciate the reviewer’s valuable suggestion and will include the corresponding experiments in the final version of the paper.\n\n\n| Method                         | FLOPs (T) | FID↓   | sFID↓  | IS↑      | Precision↑ | Recall↑ |\n|--------------------------------|-----------:|-------:|-------:|---------:|-----------:|--------:|\n| DiT-XL/2   (50steps, baseline) | 11.44     | 2.46  | 4.70  | 237.76  | 0.80      | 0.60   |\n| 50% steps                      | 5.72      | 3.18  | 5.30  | 231.12  | 0.79      | 0.58   |\n| 36% steps                      | 4.12      | 4.37  | 6.43  | 214.37  | 0.77      | 0.56   |\n| FORA   ($\\mathcal{N}=3$)        | 3.91      | 3.66  | 7.09  | 224.34  | 0.79      | 0.55   |\n| Ours   ($\\mathcal{N}=3$)        | 3.91      | 3.22  | 5.13  | 224.87  | 0.79      | 0.57   |\n\n---\n\n**References**\n\n[1] Elfwing, S., et al. Sigmoid-weighted linear units for neural network function approximation in reinforcement learning. Neural networks, 107, pp3-11, 2018 \n\n[2] Peebles, W. & Xie., S. Scalable diffusion models with transformers. In ICCV, 2023\n\n[3] Song, Ji., et al. Denoising diffusion implicit models. In ICLR, 2021 \n\n[4] Selvaraju, P., et al. FORA: Fast-forward caching in diffusion transformer acceleration. arXiv, 2024\n\n[5] Luo, S., et al. Latent consistency models: Synthesizing high-resolution images with few-step inference. arXiv, 2024"}]

Author rebuttal:

Meta-review: The paper proposes a cache-based acceleration framework for diffusion transformers that reuses intermediate features and applies lightweight learned linear modulation only when necessary, achieving up to 3× FLOP reduction and 2× latency drop on DiT-XL/2, PixArt-α, SD3-medium, and Sana-0.6B while preserving generation quality. Reviewers agree the method is technically solid, easy to plug in, and backed by extensive experiments, but note the linear modulator’s limited expressiveness, sensitivity to the gating threshold, and extra training/tuning burden. Authors adequately addressed these concerns in rebuttal and committed to clarifying the method section and adding ablations. Based on the above reason, this paper is recommended for acceptance.
Decision: Accept (poster)