MagCache: Fast Video Generation with Magnitude-Aware Cache
Authors: Zehong Ma
Longhui Wei
Feng Wang
Shiliang Zhang
Qi Tian
Venue: NeurIPS 2025 poster ()
Keywords: Fast Video Generation
Magnitude-Aware Cache
Training-Free Acceleration
Primary area: applications

Abstract: Existing acceleration techniques for video diffusion models often rely on uniform heuristics or time-embedding variants to skip timesteps and reuse cached features. These approaches typically require extensive calibration with curated prompts and risk inconsistent outputs due to prompt-specific overfitting.  In this paper, we introduce a novel and robust discovery: a unified magnitude law observed across different models and prompts. Specifically, the magnitude ratio of successive residual outputs decreases monotonically, steadily in most timesteps while rapidly in the last several steps. Leveraging this insight, we introduce a Magnitude-aware Cache (MagCache) that adaptively skips unimportant timesteps using an error modeling mechanism and adaptive caching strategy. Unlike existing methods requiring dozens of curated samples for calibration, MagCache only requires a single sample for calibration. Experimental results show that MagCache achieves 2.10×-2.68× speedups on Open-Sora, CogVideoX, Wan 2.1, and HunyuanVideo, while preserving superior visual fidelity. It significantly outperforms existing methods in LPIPS, SSIM, and PSNR, under similar computational budgets.

MagCache: Fast Video Generation with Magnitude-Aware Cache
Zehong Ma Ma
Longhui Wei Feng Wang', Shiliang Zhang'
Qi Tian
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University
Huawei Inc.
zehongma@stu. pku. edu. cn, weilh2568@gaal. com
Swangeve@foxmail.. slzhang.jdl jel@jdlll.... edu en, tian. waa.a..i1ll0 g11@huavei.com
Project page: http::///abbog-llaaittthh..1.1/
Codes: http:://gtaab.ccm//aaooglllllll
Abstract
Existing acceleration techniques for video diffusion models often rely on uniform heuristics time-embedding variants skip timesteps and reuse cached features. These approaches typically require extensive calibration with curated prompts and risk inconsistent outputs due to prompt-specific overfitting. this paper, we introduce a novel and robust discovery: a unified magnitude law observed across different models and prompts. Specifically, the magnitude ratio of successive residual outputs decreases monotonically, steadily in most timesteps while rapidly in the last several steps. Leveraging this insight, we introduce Magnitude-aware Cache (MagCache) that adaptively skips unimportant timesteps using error modeling mechanism and adaptive caching strategy. Unlike existing methods requiring dozens of curated samples for calibration, MagCache only requires a single sample for calibration. Experimental results show that MagCache achieves 10x-2.68x speedups on Open-Sora, CogVid, Wan 1. and Hunyuan Video, while preserving superior visual fidelity. significantly outperforms existing methods in LPIPS, SSIM, and PSNR, under similar computational budgets.
Introduction
In recent years, diffusion models 2 4
have achieved remarkable success in visual generation and understanding 5 6) tasks. These models have evolved from U-Net 7 9 to more sophisticated
diffusion transformers [0], significantly enhancing both model capacity and generation quality.
Leveraging these advancements, state-of-the-art video generation frameworks [T] 12 13
15 16
have demonstrated impressive fidelity and temporal coherence in generated videos.
Despite these achievements, the slow inference speed of diffusion models remains critical bottleneck The primary reason is the inherently sequential nature of the denoising process (18), which becomes increasingly problematic as models scale to higher resolutions and longer video durations
[7].
19
14 While recent approaches such as distillation 220 21 22 and post-training quantization
23
24 offer potential acceleration, they often require costly retraining and additional data, making
them less practical for widespread adoption.
27 28 present a lightweight alternative by reusing intermediate outputs in multiple steps without the need for retraining. However, conventional uniform caching strategies fail to fully exploit the dynamic nature of output similarities during inference, leading
Caching-based methods 25
"Corresponding authors. Project leader. f Work done during internship Huawei Inc.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

(a) Avearge Magnitude Ratio
(b) Magnitude Ratio Variability
Residual Cosine Distance
0.06
0.025
promptl
prompt2
promp12
0.95
0.020
prompu3
prompi3
0.015
0,90
0.02
procspei
0.85
procmpd2
0.005
procupe]
0.000
20
40
50
80
100
20
40
50
100
20
40
60
80
100
diffusion process
diclusion process (16)
diffusion process
Wan 2.1
Avearge Magnitude
Magnitude Ratio Variability
Residual Cosine Distance
pronge] prompe2
0.03
peonge] prompt2
0.015
prompt?
pcompts
0.02
0.010
0.95
prompee
0.01
0.94
procsp22
prompc3
0.00
20
40
60
80
100
20
40
60
80
100
20
40
60
50
100
diffusion process (%6
ditfusion process
dicfusion
(%6)
OpenSora
Figure I: Relationships between residuals across diffusion timesteps. Differences between adjacent residuals mainly due magnitude rather than direction during the first 80% of steps. In the final 20%, both magnitude ratio and cosine distance change sharply in opposite trends. but the magnitude ratio still reflects residual differences. (a) Average magniuude ratio decreases gradually, then drops sharply near the end; ratios close to 1 indicate stable transitions suitable for cache reuse. (b) Standard deviation of the magnitude ratio remains near zero in early steps, indicating stable magnitudes. (c) Token-wise cosine distance stays near zero early on, showing consistent residual directions.
to redundant computations and suboptimal cache utilization. Recent work such as AdaCache [29 dynamically adjusts caching strategies based on content complexity, while FasterCache [30 identifies redundancy in classifier-fre free guidance (CFG) outputs. Additionally, TeaCache [31] builds step- skipping functions through output residual modeling with time embedding difference or modulated input difference, which requires extensive calibration for different models and may overfit to the calibration set.
In this paper, we uncover new law for the magnitude ratio of successive residual outputs across different video diffusion models and prompts. The residual output is the differennee between the model's predicted output and its input. The magnitude ratio shows the change of residuals from previous timestep the current one. Figure aa) shows that the magnitude ratio steadily decreases during most of the diffusion process and drops sharply in the final steps. This trend tells us that many early and min-range timesteps behave very similarly, which suggests that there is lot of redundancy that can be used to speed up the process.
Figure demonstrates that the change across adjacent timesteps comes mainly from differences in magnitude rather than from their direction in the first 80% timesteps. In Figure(]cc) the token-wise cosine distances are very small in the first 80% process. This indicates that the direction vectors remain very similar. In addition, Figure bb) shows that the standard deviation of the magnitude ratio close zero in the early stages. Together, these observations confirm that the change between residual outputs at adjacent timesteps is primarily due to differences in average magnitude in the early timesteps. For the last 20% timesteps, both average magnitude ratio and residual cosine distance change dramatically. It indicates that the residual change of final 20% steps can also be measured by average magnitude ratios. Therefore, we can utilize the unified law of average magnitude ratio to indicate the difference between adjacent timesteps. Besides, the magnitude ratio is robust across various random inputs. As seen Figure aa) for Wan 2.1, 1, the ratio stays consistent and stable when different prompts used.

---

By relying on the stable magnitude ratio. we can avoid the complexity and the risk of overfitting found in polynomial fitting methods like TeaCache BT]. With this robust magnitude law, we can more accurately estimate and control the error introduced by skipping timesteps. This allows us to achieve a significant acceleration in inference without compromising visual quality.
Motivated by this insight, we introduce MagCache, magnitude-aware cache designed to adaptively skip unimportant timesteps based on the observed magnitude ratios. The method consists of two main components:
Accurate Error Modeling: Building the observations in Figure 1 we model the potential error in- troduced by skipping timesteps by quantifying changes in residual magnitude. Unlike TeaCache [1], method accurately estimates the enror even when multiple consecutive steps are skipped. In contrast, TeaCache performs poorly in this scenario due to the inherent inaccuracies of its polynomial fitting and prediction approach. By leveraging magnitude variation to estimate error, MagCache ensures that timestep skipping does not significantly compromise the quality of the generated video.
Adaptive Caching Strategy: With the accurate error modeling, timesteps until its accumulated error exceeds the predefined threshold or maximum skip length. This ensures that the total approximation error remains within an acceptable threshold, maintaining high visual fidelity while achieving significant speedup.
adaptively skips consccutive
In contrast to the TeaCache, which using 70 curated prompts to fitting coefficients. our MagCache requires only random sample forward once for calibration, avoids extensive fitting time. Besides, the calibrated magnitude curve is more stable and robust than the polynomial curve. It seamlessly integrates into existing diffusion model pipelines, providing a -ad--lay solution for efficient video generation. Our contributions are summarized as follows:
We identify stable, monotonically decreasing ratio of residual mag- nitudes, which robust across different prompts, providing a principled criterion for skipping redundant diffusion steps during inference.
Unified Magnitude Law:
MagCache: We introduce MagCache, a magnitude aware cache that adaptively skips timesteps with an error modeling mechanism and adaptive caching strategy.
Superior Performance: MagCache consistently achieves over 2 inference speedup video diffusion models such as Open-Sora, CogVideoX: Wan 2.1, and Hunyuan Video, as well as on image diffusion model Flux, while maintaining superior visual fidelity. Under similar computational budgets, MagCache significantly outperforms existing caching- based methods across LPIPS, SSIM, and PSNR metrics.
Related Work
2.1 Diffusion Models for Video Synthesis
Diffusion models have become foundational in generative modeling due to their ability to produce Initially, diffusion models for video synthesis followed the U-Net-based architecture and extended image diffusion models for temporally coherent video generation [4] 7 (35 36 37 38 39]. These methods generated short to medium-length videos by conditioning spatial diffusion models temporal signals [1)
high-quality and diverse outputs 2 32 33
However, the scalability of U-Net-based architectures poses limitations in modeling complex spa- tiotemporal dependencies. Inspired from the success of LLM [4] 44 45 46, 47). transformer- based diffusion models (DiT) have been increasingly adopted due to their greater modeling capacity and flexibility (88 [9
49 14]. Notably, Open-Sora (50) demonstrates the scalability and realism achievable through large-scale training diffusion transformers for video generation. Recently, the newly spen-sourced Wan 2.1 [16] has demonstrated impressive video generation performance, but generating a five-second video still takes several minutes on a single A800 GPU.
13
2.2 Efficiency Improvements in Diffuusion Models
Despite their impressive generation quality, diffusion models suffer from high inference costs, which limit their deployment in real-time resource-constrained settings. Efforts improve efficiency can
3

---

be broadly categorized into reducing the number of sampling steps and lowering the computational cost per step.
For sampling step reduction, methods based on improved SDE or ODE solvers [51
S2 53
and
model distillation [55
22
20
21 have been proposed. Consistency models [56
57 and pseudo-
numerical solvers 4 58 offer further improvements for fast sampling. Caching-based methods
improve inference efficiency by reusing features select timesteps.
To reduce per-step cost, approaches such as quantization [59
60
61 62], pruning 63 sparse
attention 65 66. 67]. and neural architecture search 68
69
have been explored.
In these methods, cache-based acceleration 77
has gained attention due its simplicity and portability. DeepCache [25], Faster Diffusion 73 and PAB [27] improve inference efficiency by reusing features at select timesteps. A-DiT
71
72
28 adapts this idea to transformer-based models by
caching attention-layer residuals. Recent work such as AdaCache [29] dynamically adjusts caching
strategies based on content complexity, while FasterCache (30] identifies redundancy in classifier-free
guidance (CFG) outputs to enable efficient reuse.
While these cache-based approaches demonstrate promising results, they often rely on heuristic or data-driven patterns that may not generalize across prompts or model variants. For example, TeaCache (31] builds step skipping functions through prompt-specific residual modeling with 70 curated prompts. It may overfit the calibration prompts and require extensive resources for calibration.
Differences with Previous Methods: Our method leverages a newly discovered unified law in residual magnitudes to accurately control the error when skipping timesteps. Unlike TeaCache, which requires extensive prompt-specific polynomial fitting and calibration, single random sample for calibration. This simpler, magnitude-awarr strategy offers a more robust and generalizable caching mechanism. It achieves significant speedup without compromising visual fidelity, and it reliably performs across different models and scenarios.
approach only needs
3 Method
3.1 Preliminary
Flow Matching. Flow matching 74 75 is a continuous-time generative modeling framework that learns velocity field (x, transport samples from a data distribution Pdata Paam(Xy) to a simple prior distribution (e.g., Gaussian). Given a forward trajectory x; defined by stochastic or deterministic interpolation between X9 and noise sample x1, the training objective is to match the model-predicted velocity (x(x., L) to the target velocity v* (x),():
LFM = Ex [Vx(xx - v vx,,, (x,,1))² [)]*]
This formulation naturally encompasses diffusion models and score-based models as special cases. For example, linear interpolants with time-dependent velocity targets can recover DDPM and score matching objectives.
Trajectory and Velocity. The forward trajectory x; is constructed via prescribed interpolant between the data sample Xy and a noise sample X1 ~ ~N, I), such as:
x, = (1- X + p(t) X1.
(2)
where p(t) is monotonically increasing interpolation schedule with p(0) 0 and p(1) 1. The ground-truth velocity v" (x,, t) can be derived from the time derivative of x, or computed via an optimal transport formulation depending on the specific design.
Residual. this work, we define the residual as the difference between the model's predicted velocity and its input at each timestep:
Vo(X,, x;
This residual captures the effective "update signal" generated by the model at each step, which reflects the model's 's internal belief about how the input should evolve. By analyzing these residuals across different timesteps, we uncover the magnitude correlation with timesteps, which form the basis for MagCache introduced in Section 3.3

---

Diffusion Timesteps
(Last Cached)
++1
(Current)
&, = E1-1 Coai (f,1)
& 5ō
Pass
Reuse Old Cache
Error Modeling
s-is
Fail
Compute New Cache
Eakp (f,/): emor of slep
Condition Check
total accumulated error
total error threshold
Adaptive Caching Strategy
maximum skip length
MagCache
Figure 2: Overview of the MagCache. The MagCache consists of error modeling mechanism and adaptive caching strategy. With the estimated total accumulated error &, MagCache adaptively reuses the old cache computes new cache by validating the two conditions in See 333
3.2 Magnitude Analysis
In this section, empirically demonstrate that unified magnitude law serves both accurate and stable criterion for measuring the difference between residuals. Concretely, we show (1) that the average magnitude ratio faithfully captures the change in residual outputs, and (2) that this ratio exhibits remarkable consistency across different models and prompts.
We define the per-step magnitude ratio
|rella )
Y = mean
(4)
| 1
where r, Vg (x+,t) X, denotes the residual at timestep
Accurate Criterion for Residual Difference. First, we verify that changes in residual outputs between adjacent timesteps are driven almost entirely by differences in magnitude rather than direction. As shown in Figure [a), during the first 80% of the diffusion trajectory, "/e decreases slowly and smoothly from 1, and the standard deviation of % remains token-wise cosine distance between r, and r;-1 also stays extremely small (Fīgure[]c). indicating that their directional patterns virtually unchanged. Together, these observations confirm that
zero (Figure [[b)). Meanwhile, the
Fr r R |||r |||
(5)
that "/ alone accurately quantifies the residual difference. In practice, when 7 is close to the two residuals are nearly identical; when 7r. drops, their difference grows accordingly.
Robustness Across Models and Prompts. Next, we assess the robustness of under varying models and textual prompts. Figure overlays the average magnitude-ratio curves for both Wan 2.1 and Open-Sora models: in each case, 7/ follows the same monotonically decreasing trajectory, with only a sharp fall in the final few steps. This consistency demonstrates model-agnostic behaviour. We further sample a diverse set of text prompts (see Appendix) and compute " for each; the resulting curves almost coincide on Wan 2.1, showing that " is prompt-invariant. Such stability implies that a single calibration suffices characterize residual scaling for any prompt model variant.
By combining these two findings, we establish that the average magnitude ratio not only captures the true per-step change in residual outputs but does so in highly stable and prompt-agnostic manner. This insight underpins our adaptive caching mechanism Section 3.3] allowing MagCache to skip redundant timesteps reliably without risking undue approximation error.
5

---

3.3 MagCache
Building on the unified magnitude law analyzed in Section MagCache leverages the stable, monotonically decreasing behavior of the per-step magnitude ratio to drive both its error modeling and adaptive caching decisions. An overview shown in Figure
3.3.1 Error Modeling
Let é be the last timestep at which we refreshed the cache. If we skip steps f 1..... t, the skip emror step given by
Eskip (f,
II %
(6)
i=ł+1
where
||rell
7V = mean
(7)
|ri11112
Since Section 3.2 demonstrated that (i) the residual difference dominated by these magnitude
changes (Equation 5 ) and (ii) % is highly stable (low variance) across models and prompts, this multiplicative estimate closely matches the practical deviation between the cached residual ry and the ground-truth residual To account for accumulated error over multiple skips, we maintain running total error Eg:
& E + Eskip t),
initialized with & O. Thanks to the near-zero standard deviation of 7 in early and mid timesteps (Figure [b)), this accumulation remains predictable, ensuring our bound on approximation error is both tight and reliable.
3.3.2 Adaptive Caching Strategy
Armed with an accurate error estimate, MagCache only skips step f. if both the total accumulated and the number of consecutive skips remain within bounds. Specifically, requires
&, S 6,
(9)
where 5 a user-specified threshold on total accumulated error, and
I -- K, where K is the maximum number of steps that could be skipped using a single cached residual. either condition is violated, we reset:
(10)
i+t, E, + 0,
(11)
recompute the true residual rt, and update the cache. Otherwise, reuse r, for step f, incurring new computation.
The introduction of maximum skip length K is crucial. Although our magnitude-bant based error modeling is highly accurate, it still an approximation. Over long sequences of steps, small modeling errors can accumulate. By bounding the skip length, ensure that such drift is regularly corrected, preventing the model from deviating too from the true residual trajectory.
In summary, by tightly integrating the empirical magnitude law from Section 3.2 into both error modeling and adaptive caching strategy, MagCache provides a principled, training- free acceleration framework that dynamically balances efficiency and quality in video diffusion inference.
4 Experiment
4.1
Settings
Base Models and Compared Methods. To demonstrate the effectiveness of our method, we quanti- tatively evaluate our MagCache on video diffusion models like Open-Sora 1.2 [TT, Cog Videox [4]
Wan 2.1 [16). and Hunyuan Video (76) and image diffusion model Flux [77]. Following TeaCache 31

---

Original(w/o Speed up)
TeaCache
MagCache (Ours)
Open- Sora
Lateney: 44.5s
latency
Latency: 21:2: Speed up 2.10x)
Latency: 44.55
Latency 21
Latency 21.25 (Speed up 2.10x)
Wan 2.1
Latency 187-25
Late Latency: 87 59
Latenny Speett up 2:68x)
Latency: 187:25
Latency:
Latency (Speed 2.68x
Comparison of visual quality and efficiency (denoted by latency) with the competing method. MagCache outperforms TeaCache [I] in both visual quality and efficiency, Latency evaluated on single A800 GPU. Video generation specifications: Open-Sora [1] (51 frames, 480p), Wan 2.1 1.3B [16 (81 frames, 480p). Best-viewed with zoom-in.
Figure 3:
we compare base models with recent efficient video synthesis techniques, including PAB 27 T-GATE (72), A-DiT (28), FasterCache [30]. TeaCache (31] DuCa (78), and TaylorSeer (79). to
highlight the advantages of our approach. Notably, our MagCache also support recent visual genera- tion or editing models, such as FramePack [80]. Wan2.2 [16], Flux-Konteds Kontext [81), OmniGen2 [82).
Owen-Image {3 and
[83 in the official code repository.
Evaluation Metrics. To assess the performance of video synthesis acceleration methods, we focus on two primary aspects: inference efficiency and visual quality. For evaluating inference efficiency, we use Floating Point Operations (FLOPs) and inference latency as metrics. Following PAB [2] and TeaCache (31), employ LPIPS
84 PSNR, and SSIM for visual quality evaluation.
by default for all experiments. Latency measured single A800 GPU. As shown in Figure the magnitude ratio remains stable and robust across different prompts. Therefore, we select the prompt in Appendix [A.] to compute the magnitude ratio. For all models, following prior works [86] 87.. we keep the first 20% of diffusion steps unchanged, as these initial steps are critical the overall generation process. is consistent with observation that the magnitude ratio has relatively larger variation in the first 20% steps. For Open-Sora we set K and 5 = 0.12 for MagCache- fast, and = 1, 5 = 0.06 for MagCache- slow. For Wan 2.1. MagCache- fast uses K 4 and 8 0.12, while MagCache-slue and 8 = 0.12. In the ablation study, we randomly sample 100 prompts from VBench to conduct our experiments. More details can refer official repository.
Implementation Detail. We enable FlashAttention
uses K
4.2 Main Results
Quantitative Comparison. Table provides a comprehensive evaluation our proposed MagCache method, highlighting its superiority over TeaCache and other cache-based methods in both inference efficiency and visual quality across diverse scenarios. We evaluated both the slow and fast variants of MagCache multiple baselines, including Open-Sora 1.2 (51 frames, 480P), Wan 2.1 1.3B (81
7

---

Table 1: Quantitative evaluation of inference efficiency and visual quality in video generation models. MagCache consistently achieves superior efficiency and better visual quality across different base models. It surpasses existing methods in visual quality by large margin under the similar computation budget. f denotes that these methods are not memory-efficient, which yeild tens of
additional memory cost.
Efficiency
Visual Quality
Method
FLOPs (P)
Speedup T Latency (8) J
LPIPS
SSIM T PSNR
Open-Sora 1.2 (5T frames, 480P)
Open-Sora 1.2 (T'
30)
3.15
1x
44.56
A-DiT 28
3.09
1.03x
0,5692
0.4811
11.91
T-GATE 72
2.75
1.19x
0.3495
0.6760
15,50
PAB-slow 27
2.55
1.33x
33.40
0.1471
0.8405
24.50
PAB-fast
2.50
1.40x
31.85
0.1743
0.8220
23.58
FasterCache f
1.91
1.72x
25.90
0.1511
0.8255
23.23
DuCa | 78
2.08x
21.42
0,2316
0.7652
19.96
TeaCache-slow
31
2.40
1.40X
31.69
0,1303
0.8405
23.67
TeaCache-fast
1.64
2.05x
21.67
0.2527
0.7435
18.98
MagCache-slow
1.41X
31.48
0.0827
0.8859
26.93
MagCache-fast
1.64
2.10X
21.21
0,1522
0.8266
23.37
Wan 2.1 1.3B (81 frames, 480P)
Wan 2.1 (T' 50)
8.21
1x
187.21
TeaCache-slow 31
5.25
1.59x
117.20
0.1258
0.8033
23.35
TeaCache-fast 33
3.94
2.14x
87.55
0,2412
0.6571
18.14
TaylorSecr(N=2,, 0=1)
79
2.07
90.15
0,3792
0.5220
15.06
MagCache-slow
3.94
2.14X
87.27
0.1206
0.8133
23,42
fagCache-fast
3.11
2.68x
69.75
0.1748
0.7490
21.54
Hunyuan Video (129 frames, 540P)
Hunyuan Video (T
50)
45.93
1x
1163
TeaCache-slow 31
27.56
1.63x
712
0.1832
0.7876
23.87
TcaCache-fass 31
20.21
2.26x
514
0.1971
0.7744
23.38
MagCache-slow
20.21
2.25
516
0.0377
0,9459
34,51
-ag
18.37
2.63x
441
0.0626
0.9206
31.77
CogVideoX 2B (49 frames, 480P)
Videi (T'= 50)
2.36
1x
74.10
TeaCache [31]
L.3
2.30x
32.20
0.1221
0.8815
27.08
MagCache
0.99
2.37 x
31.15
0.0787
0.9210
30.44
Flux (Text-to- Image 1024
1024)
Flux (T 28)
1x
14.26
TeaCache-slow 31
0.77
2.00x
7.11
0,2687
0.7746
20.14
TeaCache-fast [1
0.59
2.52x
5.65
0,3456
0.7021
18.17
MagCache-slow
0.59
2.57x
5.53
0.2043
0.8883
24.46
MagCache-fass
0.53
2.82x
5.05
0.2635
0.8093
21.35
frames, 480P), Flux (Text-to-Image 1024x 1024) and Hunyuan Video (129 frames, 540p) provide robust comparison.
On the Open-Sora 1.2 benchmark, compared to TeaCache-slow, with an LPIPS of 0. 1303, MagCache- slow significantly improves visual quality with an LPIPS of 0.0827, an SSIM 0.8859, PSNR of 26.93 de3mosstttg clear advantage over TeaCache. Notably, our MagCache-fan variant achieves a remarkable 2. 2.10x speedup with a latency of 21.21 seconds. This performance is comparable to the TcaCache-slow variant, which operates with higher latency of 3.69 seconds, while both methods deliver similar visual quality. other words, MagCache-fast successfully combines high acceleration effect with performance that rivals TeaCache-slow, demonstrating that is possible to achieve both rapid inference and competitive visual fidelity simultaneously.
For the Wan 2.1 1.3B benchmark, the benefits of MagCache become even more apparent. MagCache- slow reduces FLOPs from 8.21 to 3.94, resulting in 2.14x 14x speedup and a latency of 87.27 seconds, compared to TeaCache-slow's 5.25 FLOPPs with 1.59x speedup and 117.22 17.20s latency. In addition, MagCache-slow achieves better visual quality (LPIIS 1206, SSIM 0.8133, PSNR 23.42 (3.42) than its TeaCache counterpart. Meanwhile, MagCache-Fie further improves performance by reducing

---

FLOPs to 3,11, leading to an impressive 2.68x speedup and latency as low as 69.75 seconds, clearly outperforming TeaCache-fass fast in the speed-aceuracy trade-off.
Across other benchmarks, MagCache consistently provides better visual quality under comparable computational budgets. It is worth noting that other methods, such FasterCache [30], DuCa (78), and TaylorSeer [79]. require significantly larger memory for caching, which limits their applicability video diffusion models. For instance, TaylorSeer requires 40 GB of additional memory generate a 480P video with Wan 2.1 1.3B, whereas MagCache requires only 0.5 GB of extra memory.
Overall, these results demonstrate that MagCache achieve better visual quality than other cache- based methods under similar computational cost.
Compatibility with other acceleration methods. As shown in Appendix A.3 MagCache is compat- ible with other acceleration techniques, including model distillation and low-precision arithmetic.
Visualization, Figure 3 compares videos generated by MagCache-fast, the original model, and TeaCache-fas((3]] For Open-Sora, TeaCache performs poorly-the overall color and style of the video shift significantly. As shown in Table TeaCache-fast yields very low PSNR scores, indicating poor video quality. When PSNR falls below 20, visual distortions typically become quite noticeable. In the case of Wan 2.1, TeaCache alters key details such as the object held by the panda and the background wall, whereas our method preserves these fine details effectively. In human-centric scenarios, our approach maintains the identity and structure of the person, while TeaCache often modifies the person's identity entirely. Finally, our method achieves 2.68 speedup on Wan 2.1 without noticeable quality degradation. These results demonstrate that MagCache delivers superior visual quality with reduced latency compared TeaCache. Please refer to Appendix A.. for more visualization.
4.3
Ablation Studies
Table 2: Ablation study of maximum skip length K and total error threshold slow and fast inference mode. K controls the acceleration mode, while ö fine-tunes the qulity-speed trade-off within that mode.
Mode
Speedup
LPIPS +
SSIM
PSNR 1
0.06
2.0x
0.0940
0.8383
24.57
MagCache-slow (Wan2.1 1.3B)
0.12
2.
0.1053
0.8275
24.32
0.03
1.9x
0.0888
0.8427
24.68
0.06
2.4x
0.1375
0.7749
22.34
MagCacheffast
0.12
2.7x
0.1625
0.7571
22.25
(Wan2.1 1.3B)
0.03
2.0x
0.1263
0.7828
22.51
0.06
1.8x
0.1414
0.8142
23,91
MagCache-slow
0.12
1.9x
0.1432
0,8130
23.81
(OpenSora)
0.03
1.7x
0.1389
0.8162
24.05
0.06
2.1x
0.2040
0.7632
21.93
MagCache-fan
0.12
2.4x
0.2065
0.7542
21.77
(OpenSora) 0.03
2.0x
0.2000
0.7668
22.00
We conduct an ablation study to analyze sensitivity MagCache to its two primary hyperparam- eters: the maximum skip length K and the total accumulated error threshold 8. The parameter K primarily controls the acceleration mode (i.c., slow mode or fast mode), while å serves to fine-tune the trade-off between generation quality and inference speed within that mode. The results in Table 2 demonstrate that a desired speedup can be achieved with only a few adjustments. We provide robust default parameters for slow mode (K = 2, 0.06) and fast mode (K 4, d 0.06). Note that in the implementation details of Section 4.1 the parameters were specifically set to achieve similar acceleration speeds to TeaCache [11 for fair comparison.
The impact of maximum skip length K. The parameter primarily governs the acceleration mode of MagCache, setting broad trade-off between inference speed and generation quality. As shown in Table switching from K to K consistently yields a significant increase in speedup across different models. For instance, with the 1 model, increasing K from 2 to 4 elevates the speedup

---

PSNR from 2.0x to 2.4x under the default 5 = 0.06. This is accompanied by predictable trade-off in quality metries, with LPIPS increasing from 0.0940 to 0.1375. This clear relationship allows users to first select an acceleration range by choosing K before detailedly adjusting the qulity-speed trade-off.
The impact of threshold 8. Within selected mode, the threshold ō offers fine-grained control over the quality-speed trade-off. For example, in the slow mode (K' 2) for OpenSora, decreasing ó from default 0.06 to 0.03 slightly lowers the speedup from 1.8x to 1.7x but improves generation quality, with LPIPS decreasing from 0. 0.1414 to 0. 189 and PSNR increasing from 23.91 to 24.05. Conversely, increasing 8 from 0.06 0.12 boosts the speedup to 1.9x minor cost quality. This consistent and monotonic behavior confirms that only 1-2 adjustments to 5 are typically sufficient to achieve a desired balance. This tuning process is efficient and -riendiyndly, particularly with our interactive ComfyUl UI integration, where users can observe the effects of parameter changes in minutes.
The influence of the calibration prompt. In Table calibrate with a random prompt (Prompt from Figure 1), calibrate with all 944 prompts (average magnitude ratios), calibrate with the most distant outlier prompt (farthest from the average curve). The results in Table 3 show that all three configurations yield nearly identical visual quality and speedup, with negligible differences in LPIPS, SSIM, and PSNR. These findings confirm that MagCache does not rely on carefully selected calibration prompts, and is robust even in the presence of outliers or complex inputs.
compare three calibration strategies:
Table 3:
The influence of the calibration prompt. The calibration of MagCache is robust to the random prompt, even outliers.
Calibration Prompt
Speedup
LPIPS +
SSIM f
Random Prompt 1 (Ours)
2.14x
0.1206
0.8133
23.42
944 Prompts
2.14x
0.1162
0,8163
23.52
Outlier Prompt
2.21x
0.1209
0.8103
23.36
Please refer to Appendix A.4 for more ablation studies.
5 Conclusion and Future Work
In this paper, we introduce MagCache, novel magnitude-aware cache designed accelerate video diffusion models by adaptively skipping unimportant timesteps. Our approach leverages a newly discovered unified law governing the magnitude ratio of successive residual outputs, which remains robust across different video samples and prompts. This insight allows us to model skipping errors accurately, ensuring high visual fidelity even during rapid inference. Through extensive evaluations on benchmarks such as Open-Sora and Wan 2.1, we demonstrated that MagCache consistently achieves significant speedups while improving visual quality compared to existing methods. Our results indicate that MagCache is versatile solution, effectively balancing computation efficiency with output quality, making it applicable in various real-time or resource-constrained video generation scenarios. We have only verified the effectiveness of the magnitude law and MagCache video generation models. It is necessary to further validate and extend them to more tasks and models. In future work, we will validate the MagCache more tasks and models.
Acknowledgements
This work is supported in part by Grant No. 2023-ICIO-LA-001-088, in part by the Natural Science Foundation of China under Grant No. U20B2052, 61936011 1, 62236006, part by the Okawa Foundation Research Award, in part by the Ant Group Research Fund, and in part by the Kunpeng& Center of Excellence, Peking University.
References
[1] Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780-8794, 2021.
10

---

[2] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances neural information processing systems, 33:6840-6851, 2020.
[3] Jascha Sohl-Dicke Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsuper- vised learning using nonequilibrium thermodynamics. In International conference on machine learning, pages 2256-2265 PMLR, 2015.
[4] Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019.
[5] Dongkai Wang, Jiang Duan, Liangjian Wen, Shiyu Xuan, Hao Chen, and Shiliang Zhang. Generalizable object keypoint localization from generative priors. In Proceedings of the Computer Vision and Puttern Recognition Conference, pages 20265-20274, 2025.
[6] Shiyu Xuan, Zechao Li, and Jinhui Tang. Diff-mm: Exploring pre-trained text-to-image generation model for unified multi-modal object tracking. arXiv preprint arXiv:2505. 12606, 2025.
[7] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint 27X 1(2):, 2022.
[8] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Bureu Karagol Ayan, Tim Salimans, al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information processing systems, 35:36479 -64944, 2022.
[9] Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Macicj Kilian, Do- minik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:211..15177 2023.
[10] William Peebles and Saining Xic. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4195-4205, 2023.
[11] Zangwei Zheng, Xiangyu Peng, Tianji Yang, Chenhui Shen, Shenggui Li. Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. Open-sora: Democratizing efficient video production for all, 2024. http:///hhubboo////paccccccccc/e
[12] PKU-Yuar Lab and Tuzhan Al etc. Open-sora: Democratizing efficient video production for all, 2024. https//ddo..org/10. 528 1/oeeooo...00081
[13] Xin Ma, Yaohui Wang, Gengyun Jia, Xinyuan Chen, Ziwei Liu, Yuan-Fang Li, Cunjian Chen, and Yu Qiao. Latte: Latent diffusion transformer for video generation. arXiv preprint arXiv:2401.0334 2024.
[14] Zhuoyi Yang. Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with expert transformer. arXiv preprint arXiv:2408. 06072, 2024.
[15] Vehitect. vchitect-2.0: Parallel transformer for scaling up video diffusion models, 2024. https//github..om//ciiie wwtpa///ccooooccceeee/e/tttee chiitectt//ciitet-22...
[16] Ang Wang. Baole Ai, Bin Wen, Chaojie Mao, Chen-Wer Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, Jianyuan Zeng, Jiayu Wang, Jingfeng Zhang, Jingren Zhou, Jinkai Wang, Jixuan Chen, Kai Zhu, Kang Zhao, Keyu Yan, Lianghua Huang, Mengyang Feng, Ningyi Zhang, Pandeng Li, Pingyu Wu, Ruihang Chu, Ruili Feng, Shiwei Zhang, Siyang Sun, Tao Fang, Tianxing Wang. Tianyi Gui, Tingyu Weng. Tong Shen, Wei Lin, Wei Wang, Wei Wang. Wenmeng Zhou, Wente Wang, Wenting Shen, Wenyuan Yu, Xianzhong Shi, Xiaoming Huang, Xin Xu, Yan Kou, Yangyu Lv, Yifei Li, Yijing Liu, Yiming Wang, Yingya Zhang, Yitong Huang, Yong Li, You Wu, Yu Liu, Yulin Pan, Yun Zheng, Yuntao Hong, Yupeng Shi, Yutong Feng, Zeyinzi Jiang, Zhen Han, Zhi-Fan Wu, and Ziyu Liu. Wan: Open and advanced large-scale video generative models. arXiv preprint ar arXhv:2003.23314 2025.

---

[17] Yanyu Li, Huan Wang, Qing Jin, Ju Hu, Pavlo Chemerys. Yun Fu. Yanzhi Wang, Sergey Tulyakov, and Jian Ren. Snapfusion: Text-to-image diffusion model on mobile devices within two seconds. Advances in Neural Information Processing Systems, 36, 2024.
[18] Andy Shih, Suneel Belkhale, Stefano Ermon, Dorsa Sadigh, and Nima Anari. Parallel sampling of diffusion models. Advances in Neural Information Processing Systems, 36, 2024.
[19] Junsong Chen, Chongjian Ge, Enze Xie. Yue Wu, Lewei Yao, Xiaozhe Ren, Zhongdao Wang. Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart-sigma: Weak-to-strong training of diffusion transformer for 4k text-to-image generation. arXiv preprint arXiv:2403.0400 04692, 2024.
[20] Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach. Adversarial diffusion distillation. arXiv preprint 1. 17044, 2023.
[21] Xiang Wang. Shiwei Zhang, Han Zhang, Yu Liu, Yingya Zhang, Changxin Gao, and Nong Sang. Videolem: Video latent consistency model. arXiv preprint arXiv:2312.09109 2023.
[22] Chenlin Meng, Robin Rombach, Ruiqi Gao, Diederik Kingma, Stefano Ermon, Jonathan Ho. and Tim Salimans. On distillation of guided diffusion models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 14297-14306, 2023.
[23] Lei Chen, Yuan Meng. Chen Tang, Xinzhu Ma, Jingyan Jiang, Xin Wang, Zhi Wang, and Wenwu Zhu. Q-dit: Accurate post-training quantization for diffusion transformers. arXiv preprint 2xXiv:2406.17343 2024.
[24] Xinyin Ma, Gongfan Fang, Michael Bi Mi, and Xinchao Wang. Learning-to-cache: Accelerating diffusion transformer via layer caching. arXiv preprint crXiv:2406.01733, 2024.
[25] Mengwei Xu, Mengze Zhu, Yunxin Liu, Felix Xiaozhu Lin, and Xuanzhe Liu. Deepcache: Principled cache for mobile deep vision. In Proceedings of the 24th annual international conference on mobile computing and networking, pages 129-144, 2018.
[26] Pratheba Selvaraju, Tianyu Ding, Tianyi Chen, Ilya Zharkov, and Luming Liang. Fora: Fast- forward caching in diffusion transformer acceleration. arXiv preprint arXiv:2407.01425,, 2024.
[27] Xuanlei Zhao, Xiaolong Jin, Kai Wang, and Yang You. Real- sal-timme video generation with pyramid attention broadcast. arXiv preprint arXiv:2408. 12588, 2024.
[28] Pengtao Chen, Mingzhu Shen, Peng Ye, Jianjian Cao, Chongjun Tu, Christos-Savvas Bouganis, Yiren Zhao, and Tao Chen. 8-dit: A training-free acceleration method tailored for diffusion transformers. arXiv preprint arXiv:2406.00 01 125, 2024.
[29] Kumara Kahatapitiya, Haozhe Liu, Sen He, Ding Liu, Menglin Jia, Chenyang Zhang, Michael $ Ryoo, and Tian Xie. Adaptive caching for faster video generation with diffusion transformers. arXiv preprint arXiv:2411.02.39 2024.
[30] Zhengyao Lv, Chenyang Si, Junhao Song, Zhenyu Yang, Yu Qiao, Ziwei Liu, and Kwan- Yee K Wong. Fastercache: Training- free video diffusion model acceleration with high quality. arXiv preprint arXiv:2410. 19355, 2024.
[31] Feng Liu, Shiwei Zhang, Xiaofeng Wang. Yujie Wei, Haonan Qiu, Yuzhong Zhao, Yingya Zhang, Qixiang Ye, and Fang Wan. Timestep embedding tells: It's time to cache for video diffusion model. arXiv preprint arXiv:2411.19100 2024.
[32] Tao Wu, Xuewei Li, Zhongang Qi, Di Hu, Xintao Wang, Ying Shan, and Li. Spherediffusion: Spherical gcometry-aware distortion resilient diffuusion model. In Proceedings of the AAAI Conference Artificial Intelligence, volume 38, pages 6126-61 6634613, 2024.
[33] Tao Wu, Yibo Jiang, Yehao Lu, Zhizhong Wang, Zeyi Huang, Zequn Qin, and Xi Li. Multicrafter: High-fidelity multi-subject generation via spatially disentangled attention and identity-aware reinforcement learning, 2025. URL https: / /tt.iv//rxvvvvvvv org/obg
.29533
[34] Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David Fleet. Video diffusion models. Advances in Neural Information Processing Systems, 35:8633-888 -646, 2022.
12

---

[35] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684-10695, 2022.
[36] Yujie Wei, Shiwei Zhang, Zhiwu Qing. Hangjie Yuan, Zhiheng Liu, Yu Liu, Yingya Zhang, Jingren Zhou, and Hongming Shan. Dreamvideo: Composing your dream videos with cus- tomized subject and motion. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition, pages 6537-6549, 2024.
[37] Haoxin Chen, Menghan Xia, Yingqing He, Yong Zhang, Xiaodong Cun, Shaoshu Yang, Jinbo Xing, Yaofang Liu, Qifeng Chen, Xintao Wang, et al. Videocrafterl: Open diffusion models for high-quality video generation. arXiv preprint arXiv:2310. 195122, 2023.
[38] Tao Wu, Yong Zhang, Xintao Wang, Xianpan Zhou, Guangcong Zheng, Zhongang Qi, Ying Shan, and Xi Li. Customcrafter: Customized video generation with preserving motion and concept composition abilities. In Proceedings of the AAAI Conference on Artificial Intelligence. volume 39, pages 8469-8477, 2025.
[39] Tao Wu, Yong Zhang. Xiaodong Cun, Zhongang Qi, Junfu Pu, Huanzhang Dou, Guangcong Zheng, Ying Shan, and Xi Li. Videomaker: sero-shot customized video generation with the inherent force of video diffusion models. arXiv preprint arXiv:2412.19645 2024.
[40] Jiuniu Wang, Hangjie Yuan, Dayou Chen, Yingya Zhang, Xiang Wang. and Shiwei Zhang. Modelscope text-to-video technical report. arXiv preprint 2rXiv:2308.06571, 2023.
[41] Yujie Wei, Shiwei Zhang, Hangjie Yuan, Xiang Wang. Haonan Qiu, Rui Zhao, Yutong Feng, Feng Liu, Zhizhong Huang, Jiaxin Ye, et al. Dreamvideo-2: 2: Zero-shot subject-driven video customization with precise motion control. arXiv preprint arXiv:2410. 13830, 2024.
[42] Mengwei Xu, Dongqi Cai, Wangsong Yin, Shangguang Wang, Xin Jin, and Xuanzhe Liu. Resource-efficiet algorithms and systems of foundation models: A survey, ACM Comput. Surv., 57(5), January 2025. ISSN 0360-0300. doi: 10. 10.1145/3706418, URL /ttps:////// 1145/3706418
[43] Peng-Yuan Wang, Tian-Shuo Liu, Chenyang Wang, Yi-Di Wang, Shu Yan, Cheng-Xing Jia, Xu-Hui Liu, Xin-Wei Chen, Jia-Cheng Xu, Ziniu Li, et al. A survey on large language models for mathematical reasoning. arXiv preprint arXiv:2506 08446, 2025.
[44] Jing-Cheng Pang, Pengyuan Wang, Kaiyuan Li, Xiong-Hui Chen, Jiacheng Xu, Zongzhang Zhang, and Yang Yu. Language model self eef-imprrveemnt by reinforcement learning contempla- tion. arXiv preprint AXXi::330514443 2023.
[45] Haiyang SHEN, Yue Li, Desong Meng, Dongqi Cai, Sheng Qi. Li Zhang, Mengwei Xu, and Yun Ma. Shortcutsbench: A large-scale real-world benchmark for API-based agents. In The Thirteenth International Conference on Learning Representations, 2025. URL https
//openreview. //openteview.ne//ffrrm wtts/foorumiddkKIfff..
[46] Shiyu Xuan, Qingpei Guo. Ming Yang, and Shiliang Zhang. Pink:
Unveiling the power of referential comprehension for multi-modal 11ms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 13888- June 2024.
[47] Dongkai Wang. Shiyu Xuan, and Shiliang Zhang. Locllm: Exploiting generalizable human keypoint localization via large language model. Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 614-623, June 2024.
[48] Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, et al. Pixart-alpha: Fast training of diffusion transformer for photorealistic text-to-image synthesis. arXiv preprint 2rXiv:2310.00420 2023.
[49] Zangwei Zheng, Xiangyu Peng, Tianji Yang, Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. Open-sora: Democratizing efficient video production for all, March 2024. URL
[50] OpenAL. Sora, 2024. https//oppemi.cco///aeee//oo///
13

---

[51] Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020.
[52] Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in Neural Information Processing Systems, 35:5775-57877, 2022.
[53] Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm- solver++: Fast solver for guided sampling diffusion probabilistic models. arXiv preprint arXiv:2211.01095, 2022.
[54] Tero Karras, Miia Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. Advances in neural information processing systems, 35: 26565-26577, 2022.
[55] Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. arXiv preprint arXiv:2202.00511 2022.
[56] Simian Luo, Yiqin Tan, Longbo Huang, Jian Li, and Hang Zhao. Latent consistency models: Synthesizing high-resolution images with few-step inference. arXiv preprint arXiv:2310.04378, 2023.
[57] Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models, arXiv preprint arXiv:2303.01440 2023.
[58] Alexia Gotta go fast when generating data with score-based models. arXiv preprint arXiv:2105.14080, 2021.
Ke Li, Rémi Piché-Taillefer Tal Kachman, and loannis Mitliagkas.
[59] Yanjing Li, Sheng Xu, Xianbin Cao, Xiao Sun, and Baochang Zhang. Q-dm: An efficient low-bit quantized diffusion model. Advances in Neural Information Processing Systems, 36, 2024.
[60] Yuzhang Shang, Zhihang Yuan, Bin Xie, Bingzhe Wu, and Yan Yan. Post-training quantization on diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1972-1981, 2023.
[61] Yefei He, Luping Liu, Jing Liu, Weijia Wu, Hong Zhou, and Bohan Zhuang. Ptqd: Accurate post-training quantization for diffusion models. Advances in Neural Information Processing Systems, 36, 2024.
[62] Junhyuk So. Jungwon Lee, Dachyun Ahn, Hyungjun Kim, and Eunhyeok Park. Temporal dynamic quantization for diffusion models. Advances Neural Information Processing Systems, 36, 2024.
[63] Zehong Ma, Shiliang Zhang, Longhui Wei, and Qi Tian. Efficient multi-modal long context learning for training-fred free adaptation. In Forty-second International Conference on Machine Learning, 2025. URL
[64] Dingkun Zhang, Sijia Li, Chen Chen, Qingsong Xie, and Haonan Lu. Laptop-diff: Layer pruning and normalized distillation for compressing diffusion models. arXiv preprint arXiv:2404.11098 2024.
[65] Daniel Bolya and Judy Hoffman. Token merging for fast stable diffusion. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4599-4603, 2023.
[66] Hongjie Wang, Difan Liu, Yan Kang, Yijun Li, Zhe Lin, Niraj K Jha, and Yuchen Liu. Attention-driven training-free efficiency enhancement of diffusion models. In Proceedings of the IEEE/VE Conference on Computer Vision and Pattern Recognition, pages 16080-16089, 2024.
[67] Wangsong Yin, Daliang Xu, Mengwei Xu, Gang Huang, and Xuanzhe Liu. Dynamic sparse attention on mobile socs. arXiv preprint arXiv:2508. 16703, 2025.
14

---

[68] Lijiang Li, Huixia Li, Xiawu Zheng, Jie Wu, Xuefeng Xiao, Rui Wang, Min Zheng, Xin Pan, Fei Chao, and Rongrong Ji. Autodiffusion: Training-free optimization of time steps and architectures for automated diffusion model acceleration. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pages 7105-7114, 2023.
[69] Shuai Yang, Yukang Chen, Luozhou Wang. Shu Liu, and Yingeong Chen. Denoising diffusion step-aware models. arXiv preprint arXiv:2310003337, 20233
[70] Felix Wimbauer, Bichen Wu, Edgar Schoenfeld, Xiaoliang Dai, Ji Hou, Zijian He, Artsiom Sanakoyeu, Peizhao Zhang. Sam Tsai, Jonas Kohler, al. Cache if you can: Accelerating diffusion models through block caching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6211 6220, 2024.
[71] Junhyuk So, Jungwon Lee, and Eunhyeok Park. Frdiff: Feature reuse for universal training-free free acceleration of diffusion models. arXiv preprint wrXiv:2312.035//1 2023.
[72] Wentian Zhang, Haozhe Liu, Jinheng Xie, Francesco Faccio, Mike Zheng Shou, and Jürgen Schmidhuber. Cross-attention makes inference cumbersome text-to-image diffusion models. arXiv preprint arXiv:2404.02747, 2024.
[73] Senmao Li, Taihang Hu, Fahad Shahbaz Khan, Linxuan Li, Shiqi Yang, Yaxing Wang, Ming- Ming Cheng, and Jian Yang. Faster diffusion: Rethinking the role of unet encoder in diffusion models. arXiv preprini arXiv:2312.00 2.09608, 2023.
[74] Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling. In The Eleventh International Conference Learning Representations, 2023. URL htttps://pperreiieee http:://ppppeeeeeeeeeeetttraraaa
[75] Simian Luo, Yiqin Tan, Longbo Huang, Jian Li, and Hang Zhao. Latent consistency models: Synthesizing high-resolution images with few-step inference, 2024. URL https //openreviev net nett/forummiddddduBorjBa33
[76] Weijie Kong. Qi Tian, Zijian Zhang. Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03603, 2024.
[77] Black Forest Labs. Flux.
wow.m//aaa-oooesttt........ -abesstt-ab/ffuuss 2024.
[78] Chang Zou, Evelyn Zhang, Runlin Guo, Haohang Xu, Conghui He, Xuming Hu, and Lin- feng Zhang. Accelerating diffusion transformers with dual feature caching. arXiv preprint arXiv:2412.18911, 2024.
[79] Jiacheng Liu, Chang Zou, Yuanhuiyi Lyu, Junjie Chen, and Linfeng Zhang. From reusing to forecasting: Accelerating diffusion models with taylorseers. arXiv preprint arXiv:2503.06923 2025.
[80] Lvmin Zhang and Maneesh Agrawala. Packing input frame contexts in next-frame prediction models for video generation. Arxiv, 2025.
[81] Black Forest Labs, Stephen Batifol, Andreas Blautaann, Frederic Boesel, Saksham Consul, Cyril Diagne, Tim Dockhorn, Jack English, Zion English, Patrick Esser, Sumith Kulal, Kyle Lacey, Yam Levi, Cheng Li, Dominik Lorenz, Jonas Müller, Dustin Podell, Robin Rombach, Harry Saini, Axel Sauer, and Luke Smith. Flux. kontext: Flow matching for in-context image generation and editing in latent space, 2025. https://s://
htgg/bbs/2500 15742
[82] Chenyuan Wu, Pengfei Zheng, Ruiran Yan, Shitao Xiao, Xin Luo, Yueze Wang, Wanli Li, Xiyan Jiang, Yexin Liu, Junjie Zhou, Ze Liu, Ziyi Xia, Chaofan Li, Haoge Deng, Jiahao Wang, Kun Luo, Bo Zhang, Defu Lian, Xinlong Wang, Zhongyuan Wang, Tiejun Huang, and Zheng Liu. Omnigen2: Exploration to advanced multimodal generation. arXiv preprint arx:v:2500 / 18871, 2025.
15

---

[83] Chenfei Wu, Jiahao Li, Jingren Zhou, Junyang Lin, Kaiyuan Gao, Kun Yan, Sheng ming Yin. Shuai Bai, Xiao Xu, Yilei Chen, Yuxiang Chen, Zecheng Tang, Zekai Zhang, Zhengyi Wang, An Yang, Bowen Yu, Chen Cheng, Dayiheng Liu, Deqing Li, Hang Zhang, Hao Meng, Hu Wei, Jingyuan Ni, Kai Chen, Kuan Cao, Liang Peng, Lin Qu, Minggang Wu, Peng Wang, Shuting Yu, Tingkun Wen, Wensen Feng, Xiaoxiao Xu, Yi Wang, Yichang Zhang, Yongqiang Zhu, Yujia Wu, Yuxuan Cai, and Zenan Liu. Qwen-image technical report, 2025. URL
https: https://arxiv //arxiv org/ ab2/ 02324
[84] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unrea- sonable effectiveness of deep features as perceptual metric. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 586-595, 2018.
[85] Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. Flashattention: Fast and memory-efficient exact attention with io-awareness. Advances in Neural Information Processing Systems, 35:16344-16359 2022.
[86] Haocheng Xi, Shuo Yang, Yilong Zhao, Chenfeng Xu, Muyang Li, Xiuyu Li, Yujun Lin, Han Cai, Jintao Zhang, Dacheng Li, et al. Sparse videogen: Accelerating video diffusion transformers with spatial-temporal sparsity. arXiv preprint arXiv:2502.01776 2025.
[87] Yifei Xia, Suhan Ling, Fangcheng Fu, Yujie Wang, Huixia Li, Xuefeng Xiao, and Bin Cui. Training- frree and adaptive sparse attention for efficient long video generation, arXiv preprint arXiv:2502.21079, 2025.
16

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction accurately reflect the contributions and scope of the paper. The introduction outlines the limitations of existing methods and presents the novel magnitude-awar caching strategy (MagCache) as a solution, which is further detailed in the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: In Section the paper discusses limitations related to the scope of the experiments and the focus on video diffusion models, suggesting areas for future work.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged
the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
17

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
The paper provides a theoretical framework for the magnitude ratio and its application in caching strategies, including assumptions and derivations in the method section.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proof' can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: Section 4.1 outlines the experimental setup, including baseline models, eval- uation metrics, and implementation details, which are sufficient for reproducing the main results.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often good way to accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
18

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: We utilize the Vbench benchmark for evaluation, which is open access. The prompts used in the experiments are provided in the Appendix. We are organizing our code and plan to open-source both soon.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please the NeurlPS code and data submission guidelines (https://nips.co
Segllicguides/Codeeee
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
htapss.cc/publicguuidees setpeedeeee/ooeSeeeeeeeeeeeeeeee
ssinnooiccoyyy for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Section 4.1 provides detailed settings, including hyperparameters and experi- mental conditions, to understand the results.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented the core the paper to a level detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: Due to the resource limitation, we do not report error bars. Our method is stable in Open-SoRA and Wan 2.1 shown in Table [
19

---

Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the eπror bar is the standard deviation or the standard error of the mean.
It is OK to report I-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (c.g. negative error rates).
show in tables
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
Question:
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification:
We use single A800 GPU for latency measurement and experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics ccctta///eeerripp..cppppl//////e
Answer: [Yes]
Justification:
We followed the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [Yes]
20

---

Justification: As discussed in Section 1 and Section 2 our MagCache accelerates video generation by up 2x. This improvement facilitates the practical deployment of video diffusion models while significantly reducing computational costs and carbon footprint.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation. generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: We do not release any data or models. We do not believe that the algorithm for fast video generation has high risk for misuse.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All existing models and datasets used in this work are properly credited with appropriate references, and their licenses and terms of use have been fully respected.
Guidelines:
The answer NA means that the paper does not existing assets.
21

---

The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re- re-packaeed, both the original license and the license of the derived asset (if has changed) should provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/moodl as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsoureing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: We do not conduct any experiments with human subjects or any experiments that are otherwise subject to IRB review.
22

---

Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: The method development in this research does not involve LLMs as any important, original, or non-standard components. We only utilize it for editing and formatting pape.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non standard components.
Please refer to our LLM policy (https://neura) hatpr/rppp..ccoooffeeeeeee///222 for what should or should not be described.
23

---

A Technical Appendices and Supplementary Material
A.1 Prompts in Figure 1
We utilize the following three prompts to generate the average magnitude ratio, magnitude ratio variability, and residual cosine distance. In all experiments, we only utilize Prompt 1 calibrate the average magnitude ratio.
Prompt I: stylish woman walks down a Tokyo street filled with warm glowing neon
animated city signage. She wears black leather jacket, long red dress, and black hoots, and carries a black purse. She wears sunglasses and red lipstick. She walks confidently and casually. The street is damp and reflective, creating mirror effect of the colorful lights. Many pedestrians walk about.
Prompt 2: In a still frame, stop sign
Prompt a laptop. frozen time
Table 4: The list of prompts in Figure I In all experiments, we only utilize prompt to calibrate the magnitude ratio for MagCache.
A.2 Definition of Statistics in Figure
In Figure D we define three metrics: the average magnitude ratio, magnitude ratio variability, and residual cosine distance. The average magnitude ratio is defined in Equation 7 Specifically, Equation first computes the L2 norm of the residuals r, and r along the channel dimension, then takes the token-wise ratio, and finally averages the result across the sequence length dimension to obtain "t. The mean operation is omitted in Equation 7
The magnitude ratio variability a and residual cosine distance disl can be represented as follows: Magnitude Ratio Variability.
|
std
(12)
where E RR- denotes the residual at timestep and || represents the L2 norm computed along the channel dimension d. The standard deviation then calculated across the sequence length dimension N. Residual Cosine Distance.
N N (11-cosr) cos(r;, r(-1))-
disl
(13)
Here, the cosine distance is computed for each token between residuals at timesteps and 1, and the final residual cosine distance dist; is obtained by averaging across all tokens.
A.3 Compatibility with Other Acceleration Methods
To assess the versatility of MagCache, investigate its compatibility with other acceleration techniques for diffusion models, namely model distillation and low-precision arithmetic. These experiments demonstrate that MagCache can be seamlessly integrated plug-and-play module to achieve further speedups on already optimized models.
Compatibility with Model Distillation. Model distillation reduces the number of denoising steps by distillation post-training. We evaluated the performance MagCache on FusionX, few-step distilled variant of the Wan2.1 14B model. As presented in Table []] MagCache achieves 1.66x speedup on FusionX without a significant degradation in visual quality.
With the reduced hep-wise redundancy inherent in distilled models, MagCache still provides sub- stantial acceleration by caching and reusing intermediate computations. Notably, simple reduction of inference steps T = 6 to match MagCache's 's speedup results in severe decline in generation

---

quality, with an LPIPS score of 0.2982. In contrast, MagCache achieves a much better LPIPS of 0.1812, demonstrating its ability maintain quality while accelerating inference.
Table Evaluation of MagCache on the few-step distilled model Wan2.1 14B FusionX (33 frames, 480P). MagCache accelerates the distilled model while maintaining significantly better visual quality compared to a baseline with a reduced step count at the same speedup.
Method
Speedup
Latency (s)
LPIPS
SSIM
PSNR
FusionX (T=10) FusionX (T=6)
1x
30
1.66
18
0.2982
0.6471
20.35
MagCache
1.66x
18
0.1812
0.7868
24.23
(T=10, skip 4 steps)
Compatibility with Low-Precision Arithmetic. We also assessed the performance of MagCache in low-precision settings by applying it to a 4-bit quantized version of the Wan2.1 14B model. The results, detailed Table show that MagCache remains effective under quantization. Both the fast and slow variants of MagCache improve the quality- speed trade-off.
Specifically, MagCache- achieves 2x speedup with only a minor increase in memory footprint. While there is slight drop in the speedup ratio compared to the full-precision model due to the reduced numerical precision, acceleration benefit remains substantial.
Table 6: Evaluation of MagCache on the 4-bit quantized Wan2. 14B model (33 frames, 480P).
MagCache is compatible with oo-precision settings, maintaining its acceleration benefits.
Method
Memory
Speedup
Latency (s)
LPIPS
SSIM
PSNR
Wan2.1 14B 4bit (T'=0)
26.3G
1x
241
MagCache-fast
26.5G
2.0x
119
0.2223
0.6780
20.42
MagCache-slow
26.5G
1.4x
169
0.1184
0.7902
22.78
A.4
More Ablations
A.4.1 Robustness to Denoising Schedulers.
To evaluate the generalizability of the calibrated magnitude ratios across different denoising sched- ulers, we performed a cross-scheduler validation. Magnitude ratios were calibrated using one scheduler (e.g.. UniPC) and then directly applied during inference with different scheduler (e.g., DPM++). As shown Table 7 the performance remains remarkably consistent. For instance, ratios calibrated with UniPC and used with DPM++ achieve the same 2.1x speedup and nearly identical SSIM and PSNR scores as when both calibration and inference use DPM++ This demonstrates that the learned magnitude ratios generalized effectively.
specific to the dynamics of single scheduler and can be
Table 7: Robustness of magnitude ratios to different schedulers. The speedup and visual quality is stable when the scheduler used for inference differs from the one used for calibration
Calibrated Scheduler
Inference Scheduler
Speedup t
LPIPS
SSTM
PSNR t
UniPC
UniPC
2.1x
0.1053
0.8275
24.32
UniPC
DPM+
2.1x
0.0976
0.8412
24.37
DPM++
DPM++
2.1x
0.0976
0.8412
24.37
DPM++
UniPC
2.1x
0.1053
0.8275
24.32
A.4.2 Robustness to the Number of Steps.
We also tested the robustness of the magnitude ratios when the number of inference steps is changed. For these cases, we use nearest-neighhor interpolation align the calibrated ratio curve to the new number of steps. Table g shows that hyperparameters calibrated with 50 steps can be effectively
25

---

applied to 30-step inference process, and vice-versa, without requiring recalibration. The speedup remains constant, and the image quality metrics are determined by the number of inference steps rather than the calibration setting. This stability simplifies the deployment of MagCache, as the pre-calibrated magnitude ratios can be used across different inference steps.
Table 8: Robustness of magnitude ratios to different numbers of inference steps. The speedup is maintained, and quality metrics are consistent for a given number of inference steps, regardless the number of steps used for calibration.
Calibrated Steps
Inference Steps
Speedup t
LPIPS
SSIM
PSNR
50
50
2.1 x
0.1053
0.8275
24.32
30
50
2.1x
0.1053
0.8275
24.32
50
30
2.1x
0.1917
0.7116
21.06
30
2.1x
0.1917
0.7116
21.06
A.4.3
Computation of Skip Error in Equation 6
we adopt the multiplicative formulation in Equation 6 to compute the single-step skip error Eskip(f, f) between the cached residual ry at timestep f and the ground-truth residual r timestep t. The multiplicative formulation is reasonable according observation and ablation experiments.
In Section 3.3
following empirical
Empirical Observation. We first define the ground-truth magnitude ratio between the resiudal re and r2 as f). Accordding to our empirical observation in Figure the magnitude ratio [ f) can be approximated by the product IT
Ves i.e.:
Trelle 1ril2
|ral
[(I. i) mean
II 7
II 1=f1
mean
(14)
|r,-1 12
i=++1
A Average Magnitude Ratio between r, and r; (f = 3)
1.00
0.95
0.90
0.85
0.50
0.75
0.70
Mi %
0.65
T
20
80
100
diffusion process (%)
Figure 4: Average Magnitude Ratio between r, and r,. where = t- 3. The I[t, f) is the ground- truth magnitude ratio, while the [[ 7/4 is the predicted magnitude ratio using the multiplicative formulation Equation 6
Besides, the difference between [ f) and Ii N is less than 1e-5 in value. Therefore, the multiplicative formulation in Equation accurately captures the ground-truth magnitude ratio and thus serves as a reliable surrogate.
Ablation Experiments. As naive baseline, consider a simplified error modeling method that ignores the accumulated error from previously skipped timesteps and considers only the instantaneous magnitude ratio ? timestep t. The corresponding skip error is defined as:
26

---

Table 9: Different error modeling methods of single-step skip error Eskip on Wan 2.1. Our multiplica- tive formulation in Equation performs better than the naive baseline in Equation 15
Error Modeling
Latency (8)
LPIPS 4
SSIM f
PSNR
Wan
Multiplicative
0.1053
0.8275
24.32
Equation 6
Naive
84
0.1154
0.8137
24.06
Equation 15
Eskip (f,
%/-
(15)
As shown in Table 9 our multiplicative formulation (Equation 6 consistently outperforms the naive baseline (Equation I5) across all evaluation metries. This result aligns with our empirical observation that the multiplicative product [i "/t provides an accurate approximation of the ground-truth magnitude ratio [tt, i) between residuals and rj.
It is also worth noting that when the magnitude ratio exceeds 1.0, we take the absolute value the Video skip error, as is done in models like Hunyuan Video and Flux.
A.4.4 The Influence of the Initial Steps.
In this section, investigate the impact of preserving different numbers of initial steps during inference. As shown in Table I0] the first 10 steps are crucial to the overall quality of the generated video. Reducing the number of unchanged initial steps from 10 5 leads to significant degradation in video quality, with LPIPS increasing from 0.1053 to 0.2431, SSIM dropping from 0.8275 to 0.6423, and PSNR falling from 24.32 18.80.
While retaining more steps generally improves video quality, it also increases latency and computa- tional cost. To strike balance between visual fidelity and efficiency, we adopt default setting that preserves the first 20% of steps, corresponding to steps for Wan 2.1 and steps for Open-Sora.
Table 10: Ablation study the number of initial unchanged steps for Wan 2.1. The model Wan 2.1 has 50 inference steps in total. t: Default setting where the first 10 steps (20%) are preserved.
Initial Unchanged Steps
Ratio
Latency (s)
LPIPS +
SSIM
PSNR 1
Wan 2.17
50
187
5
10%
73
0.2431
0.6423
18.80
f
20%
0.1053
0.8275
24.32
15
30%
98
0.0664
0.8966
27.71
A.5 Theoretical Analysis of Equation 5
Equation 5 states that when the cosine distance between adjacent residuals is small, the residual difference Ir rt-1ll can be approximated by the magnitude difference | -|rr-llll::
||r, rz- ~ ||ra|| ||r:-Ill|-
(5)
This holds when r, and 7t-1 nearly colinear, i.e., 1 cos(r:, F1-1) 0. Empirically, this occurs the first 80% steps. Theoretically, we can analysize the cosine law:
|lr - r 1|2 (1]rell 11)=-111) 21|r:|l||mm- cos(re, r2-1)).
(16)
As - cos(r r1-1) + 0, the second term vanishes, yielding Equtation By the chain rule:
Let r(t) = vo( Jt.
dr di = (0. (0,vo 1)x' 00
(17)
Assuming |at||| S M, spatial lipschitz continuity || || S Teg, and temporal lipschitz continuity
||0rvel| 5 Le, we get
27

---

dr
S (L 1)M + =.
(18)
dt
where B aggregates the ODE's stiffness (M) and model smoothness (Te I.), which usually within small constant range. Integrating over step size h(small in early steps), can get || -t-1ll 1
Bh. Assuming | ||r-1 2 O, we get:
1 1-cos(ri, |Ire Tt- 2| (11rall - 111--111 I12
I 11- 2 rr
Ba 2c
O(h²). (19)
2 rr r-
Hence, Tr and r--1 are nearly aligned when h is small. In non-uniform schedules (e.g., shift=8), early steps use small (e.g.. 0.001), making ro- negligible, validating Equation 5
A.6 More Visualization Cases
In this section, we present additional qualitative results, including both videos and images, to further demonstrate the effectiveness of MagCache. Compared with TeaCache, MagCache consistently achieves superior visual quality while maintaining comparable or lower latency. Specifically, Mag- Cache consistently delivers better alignment with ground-truth content, improved preservation of fine visual details, and enhanced rendering of textual elements, such as clearer and more accurate text generation in both videos and images. The qualitative results span four widely used video generation models and one state-of-the-art image generation model: Wan 2.1 1.3B in Figure 5 Wan 2.1 14B in Figure Open-Sora in Figure 7 Hunyuan Video in Figure g and Flux(Image Generation Model) in Figure
28

---

Wan 2.1 1.38
Original Latency: 187s
TeaCache Latency:
MagCache Latency: 69s
Original Latency: 187s
TeaCache Latency: 87s
MagCache Latency: 69s
Original Latency: 187s
TeaCache Latency: 87s
MagCache Latency: 69s
Original Latency: 187s
TeaCache Latency: 87s
MagCache Latency: 69s
Figure 5: Videos generated by Wan 2.1 1.3B using original model, Teacache-Fas and our MagCache- Fast, Best-viewed with zoom-in.
29

---

Wan 2.1 1.38
Original Latency: 187s
TeaCache Latency: 87s
MagCache Latency: 69s
Original Latency: 187s
TeaCache Latency: 87s
MagCache Latency: 69s
Original Latency: 187s
TeaCache Latency:
MagCache Latency: 69s
Wan 2.1 14B
Original Latency: 3497s
TeaCache Latency: 1749s
MagCache Latency: 1470s
Figure 6: Videos generated by Wan 2.1 1.3B and Wan 2.1 14B using original model, Teacache-Fase and our MagCache-FFi Fast. Best-viewed with zoom-in.
30

---

Open-Sora
Original Latency: 44s
TeaCache Latency: 21s
MagCache Latency: 21s
Original Latency: 44s
TeaCache Latency: 215
MagCache Latency: 21s
Original Latency: 44s
TeaCache Latency: 21s
MagCache Latency: 215
Original Latency: 44s
TeaCache Latency: 21s
MagCache Latency: 21s
Figure 7: Videos generated by Open-Sora using original model, Teacache-Fas Fast, and our MagCache- Fast. Best-viewed with zoom-in.
31

---

HunyuanVideo
Original Latency: 1163s
STOP
STOP
STOP
STOP
STOP
TeaCache Latency: 514s
TvEOT
Tv.OT
TvFOT
TvEOT
TvFOT
MagCache Latency: 441s
STOP
STOP
STOP
STOP
STOP
ROO
Original Latency: 1163s
TeaCache Latency: 514s
MagCache Latency: 441s
Original Latency: 1163s
TeaCache Latency: 514s
MagCache Latency: 441s
Original Latency: 1163s
TeaCache Latency: 514s
MagCache Latency: 441s
Figure 8: Videos generated by Hunyuan Video using original model, Teacache- Fast, and our MagCache- Fast. Best-viewed with zoom-in.
32

---

Flux
STOP
Original Latency: 14.26s
TeaCache Latency: 5.65s
STOP
MagCache Latency: 5.05s
Original Latency: 14.26s
STOP
TeaCache Latency: 5.65s
MagCache Latency: 5.05s
STOP
STOP
Original Latency: 14.26s
9
STOP
TeaCache Latency: 5.65s
STOP
MagCache Latency: 5.05s
9
Figure 9: Images generated by Flux using original model, Teacache-Fas Fast, and MagCache-Fass Fast. Best- viewed with zoom-in.
33


Reviews:
[{"review_id": "hbp1GbY7Ga", "reviewer": "Reviewer_vpsp", "summary": "This paper introduces MagCache, a magnitude-based caching method to accelerate the inference of video diffusion models, after analyzing the relationships between magnitude residuals across diffusion timesteps. The authors propose accurate error modeling and adaptive caching strategy to adaptively skip consecutive diffusion timesteps until its accumulated error exceeds the predefined threshold or maximum skip length. Experiments demonstrate the over 2x acceleration of MagCache on Open-Sora and Wan 2.1.", "questions": "1. How sensitive is the proposed magnitude-aware caching method to the choice of the single calibration sample? Would the choice affect the generation quality? While Figure 1 shows consistency across three prompts, would an outlier or particularly complex prompt result in a less representative curve?\n\n2. Are the hyperparameters dependent on the denoising scheduler? Should these hyperparameters be recalibrated when altering the denoising scheduler or the number of denoising steps?", "limitations": "Yes.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths\nThe analysis in Figure 1 clearly demonstrates the motivation of this paper to use magnitude residual as the calibration information.\nThe proposed method is a plug-and-play solution requires only a random sample for calibration.\nThe experimental results demonstrate the acceleration (over 2x) of video diffusion models’ inference.\n\nWeaknesses\nThe method's performance relies on the careful tuning of two key hyperparameters $\\delta$ and $K$ for each model.\nThere are step reduction methods for accelerating diffusion models, which may have greater variations between steps. MagCache’s may not be useful for few-step diffusion models.\nComparison with other relevant caching-based video diffusion acceleration methods is missing, such as AdaCache and FasterCache.", "was_revised": true, "final_justification": "I would like to maintain my original positive score for this work.", "rebuttal": "**Q1:** The performance relies on the tuning of two key hyperparameters $\\delta$ and $K$ for each model.\n\n**A1:** Thanks for the question. The varying $\\delta$ and $K$ settings across models were chosen to ensure fair comparisons at similar speedups. In practice, **MagCache performs robustly with default parameters across different models**, requiring only minor $\\delta$ adjustments. We offer two predefined modes:\n\n- **Slow mode: $K = 2$, $\\delta = 0.06$ (default)**\n- **Fast mode: $K = 4$, $\\delta = 0.06$ (default)**\n\nHere, $K$ controls speedup mode, and $\\delta$ adjusts the quality–speed trade-off within the selected mode. In Table R1.1, varying $\\delta$ offers fine control with just 1–2 adjustments. **We recommend the default slow mode for general use, and our ComfyUI interface makes parameter tuning fast and user-friendly.**\n\n**Table R1.1 Parameter Configurations in Slow and Fast inference mode.**\n\n| **Mode**                     | $K$   | $\\delta$ | **Speedup ↑** | **SSIM ↑** | **PSNR ↑** |\n| :--------------------------- | ----- | -------- | ------------- | ---------- | ---------- |\n| **MagCache-slow (Wan2.1)**   | **2** | **0.06** | 2.0           | 0.838      | 24.5       |\n|                              |       | 0.12     | 2.1           | 0.827     | 24.3       |\n|                              |       | 0.03     | 1.9           | 0.842      | 24.6       |\n| **MagCache-fast (Wan2.1)**   | **4** | **0.06** | 2.4           | 0.774      | 22.3       |\n|                              |       | 0.12     | 2.7           | 0.757      | 22.2       |\n|                              |       | 0.03     | 2.0           | 0.782      | 22.5       |\n| **MagCache-slow (OpenSora)** | **2** | **0.06** | 1.8           | 0.814      | 23.9       |\n|                              |       | 0.12     | 1.9           | 0.813      | 23.8       |\n|                              |       | 0.03     | 1.7           | 0.816      | 24.0       |\n| **MagCache-fast (OpenSora)** | **4** | **0.06** | 2.1           | 0.763      | 21.9       |\n|                              |       | 0.12     | 2.4           | 0.754      | 21.7       |\n|                              |       | 0.03     | 2.0           | 0.766      | 22.0       |\n\n---\n\n**Q2:** MagCache’s may not be useful for few-step diffusion models. \n\n**A2:** Despite larger residual variations in few-step models, **MagCache still provides acceleration by reusing residuals with small differences**. On FusionX (a 10-step distilled Wan2.1 14B), MagCache achieves a 1.66× speedup with minimal quality loss(Table R1.2). In contrast, directly reducing the steps to 6 significantly degrades quality, highlighting the advantage of MagCache.\n\n**Table R1.2: Evaluation of MagCache on Few-Step Distilled Model (Wan 2.1 14B FusionX, 33 frames, 480P)**\n\n| **Method**                    | **Speedup ↑** | **SSIM ↑** | **PSNR ↑** |\n| ----------------------------- | ------------- | ---------- | ---------- |\n| FusionX (T = 10)              | 1             | -          | -          |\n| FusionX (T = 6)               | 1.66          | 0.647      | 20.3       |\n| MagCache (T=10, skip 4 steps) | 1.66          | **0.786**  | **24.2**   |\n\n---\n\n**Q3:** Comparison with other caching-based acceleration methods, such as AdaCache and FasterCache.\n\n**A3:** Thanks for the valuable suggestion. We compare MagCache with AdaCache, FasterCache, DuCa, and TaylorSeer across Open-Sora, Wan2.1, and Hunyuan in Table R1.3. On Open-Sora, MagCache matches AdaCache’s speedup but outperforms it in quality with significantly less extra memory (0.1G vs 11.3G). It also surpasses FasterCache in performance, speed, and memory efficiency **(0.1 G vs. 11.3 G)** on Open-Sora. **This is because MagCache caches only a single residual instead of layer-wise outputs like AdaCache and FasterCache.** Additionally, MagCache consistently achieves superior performance on both the Wan and Hunyuan models. **Across all models, MagCache offers a strong balance of speed, quality, and memory usage.**\n\n\n\n**Table R1.3: Comparison with Other Cache-based Methods across Various Models.**\n\n| **Method**                                  | Speedup ↑ | Extra Memory↓ | SSIM ↑    | PSNR ↑   |\n| ------------------------------------------- | --------- | ------------- | --------- | -------- |\n| **Open-Sora 1.2 (T = 30, 51 frames, 480P)** | 1         | -             | -         | -        |\n| FasterCache                                 | 1.7       | 11.3G         | 0.825     | 23.2     |\n| AdaCache                                    | 2.1       | 11.3G         | 0.786     | 21.0     |\n| DuCa                                        | 2.0       | 23.9G         | 0.765     | 19.9     |\n| TeaCache-slow                               | 1.4       | **0.1G**      | 0.840     | 23.6     |\n| TeaCache-fast                               | 2.0       | **0.1G**      | 0.743     | 18.9     |\n| MagCache-slow                               | 1.4       | **0.1G**      | **0.885** | **26.9** |\n| MagCache-fast                               | **2.1**   | **0.1G**      | 0.826     | 23.3     |\n| **Wan 2.1 (T = 50, 81 frames, 480P)**       | 1         |               | -         | -        |\n| TeaCache                                    | 2.1       | **0.5G**      | 0.657     | 18.1     |\n| TaylorSeer(N=2, O=1)                        | 2.0       | 40G           | 0.522     | 15.0     |\n| MagCache-fast                               | **2.6**   | **0.5G**      | **0.749** | **21.5** |\n| **Hunyuan Video (T=50, 65 frames, 480P)**   | 1         |               | -         | -        |\n| AdaCache                                    | 2.6       | -             | 0.695     | 17.0     |\n| TeaCache                                    | 2.9       | 0.1G          | 0.792     | 20.2     |\n| TaylorSeer (N=3, O=1)                       | 3.0       | 46.2G         | 0.833     | 21.4     |\n| MagCache                                    | **3.1**   | **0.1G**      | **0.837** | **24.3** |\n\n---\n\n**Q4:** Sensitivity to the calibration sample. Does prompt selection impact the quality? Would an outlier or particularly complex prompt result in a less representative curve?\n\n**A4: MagCache is robust to the selection of the calibration sample**, and generation quality remains consistent even with outliers. We first evaluate 944 prompts from VBench and observe very low variance in magnitude ratios and cosine distances(Table R1.4). The magnitude ratios exhibit a stable, monotonically decreasing trend, with sharper drops in the final 20% steps. The cosine distance trends are also stable across prompts.\n\nWe also test MagCache with three calibration prompt: a random prompt, all 944 prompts, and an outlier prompt (farthest from the average curve). **All three yield similar speedup and visual quality (Table R1.5)**, confirming MagCache's robustness.\n\n\n\n**Table R1.4 Statistics of Magnitude Ratios and Cosine Distance across 944 Vbench prompts.** `0.9965 (± 0.0004) denotes mean ± standard deviation`\n\n| Diffusion Process (%) | Magnitude Ratio (Wan 2.1) | Cosine Distance  (Wan 2.1) |\n| --------------------- | ------------------------- | -------------------------- |\n| 0–20                  | 0.9965 (±0.0004)          | 0.0008 (±0.0003)           |\n| 20–40                 | 0.9951 (±0.0002)          | 0.0003 (±0.0003)           |\n| 40–60                 | 0.9928 (±0.0002)          | 0.0001 (±0.0001)           |\n| 60–80                 | 0.9851 (±0.0002)          | 0.0002 (±0.0001)           |\n| 80–100                | 0.9285 (±0.0019)          | 0.0054 (±0.0009)           |\n\n**Table R1.5 Influence of Calibration Prompt.**\n\n| Calibration Prompt    | Speedup ↑ | SSIM ↑ | PSNR ↑ |\n| --------------------- | --------- | ------ | ------ |\n| Random Prompt (Ours)  | 2.1       | 0.813  | 23.4   |\n| 944 Prompts (Average) | 2.1       | 0.816  | 23.5   |\n| Outlier Prompt        | 2.1       | 0.810  | 23.3   |\n\n---\n\n**Q5:** Are the hyperparameters dependent on the denoising scheduler? Should hyperparameters be recalibrated when altering denoising scheduler or number of denoising steps?\n\n**A5: MagCache’s hyperparameters are robust across different schedulers and numbers of steps. As shown in Table R1.6 and Table R1.7, magnitude ratios calibrated with one scheduler or step setting can generalize well to others without recalibration.** For instance, magnitude ratios calibrated under UniPC can be applied to inference using DPM++, and vice versa, with minimal impact on quality or speedup. When steps change, we use nearest-neighbor interpolation to align the ratio curve. Note that using the same inference scheduler or number of steps tends to yield consistent results.\n\n\n\n**Table R1.6 Robustness of Magnitude Ratios to Different Schedulers.**\n\n| Calibrated Scheduler | Inference Scheduler | Speedup ↑ | SSIM ↑ | PSNR ↑ |\n| -------------------- | ------------------- | --------- | ------ | ------ |\n| UniPC                | UniPC               | 2.1       | 0.827  | 24.2   |\n| UniPC                | DPM++               | 2.1       | 0.841  | 24.3   |\n| DPM++                | DPM++               | 2.1       | 0.841  | 24.3   |\n| DPM++                | UniPC               | 2.1       | 0.827  | 24.2   |\n| Euler                | Euler               | 2.1       | 0.85   | 25.1   |\n| Euler                | UniPC               | 2.1       | 0.827  | 24.2   |\n| UniPC                | Euler               | 2.1       | 0.85   | 25.1   |\n\n**Table R1.7 Robustness of Magnitude Ratios to Different Numbers of Steps.**\n\n| Calibrated Steps | Inference Steps | Speedup ↑ | SSIM ↑ | PSNR ↑ |\n| ---------------- | --------------- | --------- | ------ | ------ |\n| 50               | 50              | 2.1       | 0.827  | 24.3   |\n| 30               | 50              | 2.1       | 0.827  | 24.3   |\n| 50               | 30              | 2.1       | 0.711  | 21.0   |\n| 30               | 30              | 2.1       | 0.711  | 21.0   |"}, {"review_id": "c9XO9hHOwn", "reviewer": "Reviewer_AVyF", "summary": "This paper proposes MagCache, a training-free method to accelerate video diffusion models by adaptively skipping redundant timesteps based on a newly observed magnitude decay law. The authors find that the L2 norm of residuals between timesteps decreases consistently, enabling a reliable and prompt-agnostic criterion for caching. MagCache models skip error using this magnitude ratio and reuses cached features when the error is within a threshold. Experiments on Open-Sora and Wan 2.1 show 2.1×–2.7× speedup with better or comparable visual quality than prior methods like TeaCache.", "questions": "See weaknesses.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths \n\n- This paper identifies a magnitude decay law that is consistent across models and prompts.\n- MagCache requires only a single calibration sample, no retraining, and integrates easily.\n- MagCache achieves 2×+ speedups while improving visual metrics (LPIPS, SSIM, PSNR).\n\nWeaknesses\n- A notable weakness of the paper is the incompleteness of baseline comparisons. In particular, the authors do not compare MagCache against two recent state-of-the-art diffusion acceleration methods: TaylorSeer[1] and DuCa[2].\n- The ~2× speedup is solid but not as high as some recent approaches (e.g., TaylorSeer ~5×).\n- The experiments are conducted on two large text-to-video diffusion models (Open-Sora and Wan), but broader validation is somewhat limited. The method’s generality would be better established by testing on more diverse setups (e.g., CogVideoX).\n\n\nReference:\n- [1] Accelerating Diffusion Transformers with Dual Feature Caching, ICCV 2025.\n- [2] DuCa: Accelerating Diffusion Transformers with Dual Feature Caching, arxiv 2024.", "was_revised": true, "final_justification": "The author has effectively addressed my concerns, and I have decided to increase my score.", "rebuttal": "**Q1:** Comparison with two recent state-of-the-art diffusion acceleration methods: TaylorSeer[1] and DuCa[2]\n\n**A1:** Thanks for the valuable suggestion. We have added detailed comparisons with TaylorSeer [1] and DuCa [2] across three representative models: Open-Sora, Wan2.1 1.3B, and HunyuanVideo in Table R2.1. A comparison will be included in the revised paper.\n\nCompared with DuCa on Open-Sora and HunyuanVideo, MagCache achieves better visual quality at similar speedups, while requiring only 0.1 GB of extra memory—significantly less than DuCa’s 23.9 GB (Open-Sora) and 14.7 GB (HunyuanVideo). Against TaylorSeer, MagCache delivers better visual quality on both Wan2.1 1.3B and HunyuanVideo at similar speedups, with much lower memory overhead (0.1GB-0.5 GB vs. 40–46 GB). \n\nIt is worth noting that TaylorSeer and DuCa cache outputs of all transformer layers, while MagCache only caches a single residual output, greatly reducing memory usage.\n**In summary, MagCache consistently outperforms TaylorSeer and DuCa in both visual quality and memory efficiency.**\n\n\n\n**Table R2.1: Comparison with Other Cache-based Methods across Various Models.**\n\n| **Method**                          | **Speedup ↑** | Extra Memory↓ | **Latency (s) ↓** | **LPIPS ↓** | **SSIM ↑** | **PSNR ↑** |\n| ----------------------------------- | ------------- | ------------- | ----------------- | ----------- | ---------- | ---------- |\n| **Open-Sora 1.2** (51 frames, 480P) |               |               |                   |             |            |            |\n| Open-Sora 1.2 (T = 30)              | 1×            | -             | 44.56             | -           | -          | -          |\n| DuCa                                | 2.08×         | 23.9G         | 21.42             | 0.2316      | 0.7652     | 19.96      |\n| TeaCache-slow                       | 1.40×         | **0.1G**      | 31.69             | 0.1303      | 0.8405     | 23.67      |\n| TeaCache-fast                       | 2.05×         | **0.1G**      | 21.67             | 0.2527      | 0.7435     | 18.98      |\n| MagCache-slow                       | 1.41×         | **0.1G**      | 31.48             | **0.0827**  | **0.8859** | **26.93**  |\n| MagCache-fast                       | 2.10×         | **0.1G**      | 21.21             | 0.1522      | 0.8266     | 23.37      |\n| **Wan 2.1 1.3B** (81 frames, 480P)  |               |               |                   |             |            |            |\n| Wan 2.1 (T = 50)                    | 1×            |               | 187.21            | -           | -          | -          |\n| TeaCache-slow                       | 1.59×         | **0.5G**      | 117.20            | 0.1258      | 0.8033     | 23.35      |\n| TeaCache-fast                       | 2.14×         | **0.5G**      | 87.55             | 0.2412      | 0.6571     | 18.14      |\n| TaylorSeer(N=2, O=1)                | 2.07×         | 40.0G         | 90.15             | 0.3792      | 0.5220     | 15.06      |\n| MagCache-slow                       | 2.14×         | **0.5G**      | 87.27             | **0.1206**  | **0.8133** | **23.42**  |\n| MagCache-fast                       | 2.68×         | **0.5G**      | 69.75             | 0.1748      | 0.7490     | 21.54      |\n| **Hunyuan Video** (65 frames, 480P) |               |               |                   |             |            |            |\n| Hunyuan Video (T=50)                | 1×            |               | 177.50            | -           | -          | -          |\n| DuCa                                | 4.62×         | 14.7G         | 38.41             | 0.3516      | 0.6443     | 16.16      |\n| TeaCache-slow                       | 2.97×         | **0.1G**      | 59.76             | 0.1759      | 0.7927     | 20.23      |\n| TeaCache-fast                       | 4.55×         | **0.1G**      | 39.01             | 0.4377      | 0.6216     | 16.07      |\n| TaylorSeer (N=3, O=1)               | 3.00×         | 46.2G         | 59.16             | 0.1353      | 0.8335     | 21.42      |\n| TaylorSeer (N=5, O=1)               | 5.00×         | 46.2G         | 35.50             | 0.2691      | 0.7039     | 16.79      |\n| MagCache-slow                       | 3.10×         | **0.1G**      | 57.20             | **0.1283**  | **0.8372** | **24.38**  |\n| MagCache-fast                       | 5.00×         | **0.1G**      | 35.45             | 0.2656      | 0.7166     | 20.32      |\n\n---\n\n**Q2:** The ~2× speedup is solid but not as high as some recent approaches (e.g., TaylorSeer ~5×).\n\n**A2:**  Thanks for the comment. **As shown in Table R2.1, at similar speedup levels (~5×), MagCache can achieve a superior performance than TaylorSeer on HunyuanVideo with significantly less extra memory cost.** **MagCache with ~2x speedup targets a balanced trade-off between speed and quality, allowing users to enjoy acceleration without sacrificing lots of quality.** If users desire higher speedups, MagCache can be configured with larger $K$ and $\\delta$ values to further accelerate inference.\n\n---\n\n**Q3:** The experiments are conducted on two large text-to-video diffusion models (Open-Sora and Wan), but broader validation is somewhat limited. The method’s generality would be better established by testing on more diverse setups (e.g., CogVideoX).\n\n**A3:** Thanks for valuable suggestions. In the Table 5 of supplementary material, we have included a comparison on HunyuanVideo (540P) and Flux. To strengthen the generality of our approach, we futher expand our evaluation to include HunyuanVideo (480P & 540P), CogVideoX, and Flux. **As shown in Table R2.2, MagCache consistently delivers strong performance across all various models and resolutions, demonstrating its robustness and generality.**\n\nMoreover, **we have also qualitatively validated MagCache on additional models such as FramePack, VACE, Flux Kontext, Chroma, and OmniGen2**, and will release demos and code in our open-source repository.\n\n\n\n**Table R2.2 Generalization of MagCache across Diverse Models and Resolutions.**\n\n| **Method**                           | **Speedup ↑** | **Latency (s) ↓** | **LPIPS ↓** | **SSIM ↑** | **PSNR ↑** |\n| ------------------------------------ | ------------- | ----------------- | ----------- | ---------- | ---------- |\n| **Hunyuan Video** (65 frames, 480P)  |               |                   |             |            |            |\n| Hunyuan Video (T=50)                 | 1×            | 177.50            | -           | -          | -          |\n| DuCa                                 | 4.62×         | 38.41             | 0.3516      | 0.6443     | 16.16      |\n| TeaCache-slow                        | 2.97×         | 59.76             | 0.1759      | 0.7927     | 20.23      |\n| TeaCache-fast                        | 4.55×         | 39.01             | 0.4377      | 0.6216     | 16.07      |\n| TaylorSeer (N=3, O=1)                | 3.00×         | 59.16             | 0.1353      | 0.8335     | 21.42      |\n| TaylorSeer (N=5, O=1)                | **5.00×**     | 35.50             | 0.2691      | 0.7039     | 16.79      |\n| MagCache-slow                        | 3.10×         | 57.20             | **0.1283**  | **0.8372** | **24.38**  |\n| MagCache-fast                        | **5.00×**     | **35.45**         | 0.2656      | 0.7166     | 20.32      |\n| **HunyuanVideo** (129 frames, 540P)  |               |                   |             |            |            |\n| HunyuanVideo (T = 50)                | 1×            | 1163              | -           | -          | -          |\n| TeaCache-slow                        | 1.63×         | 712               | 0.1832      | 0.7876     | 23.87      |\n| TeaCache-fast                        | 2.26×         | 514               | 0.1971      | 0.7744     | 23.38      |\n| MagCache-slow                        | 2.25×         | 516               | **0.0377**  | **0.9459** | **34.51**  |\n| MagCache-fast                        | **2.63×**     | **441**           | 0.0626      | 0.9206     | 31.77      |\n| **CogVideoX 2B** (49 frames, 480P)   |               |                   |             |            |            |\n| CogVideoX (T = 50)                   | 1×            | 74.10             | -           | -          | -          |\n| TeaCache                             | 2.30×         | 32.20             | 0.1221      | 0.8815     | 27.08      |\n| MagCache                             | **2.37×**     | **31.15**         | **0.0787**  | **0.9210** | **30.44**  |\n| **Flux** (Text-to-Image 1024 × 1024) |               |                   |             |            |            |\n| Flux (T = 28)                        | 1×            | 14.26             | -           | -          | -          |\n| TeaCache-slow                        | 2.00×         | 7.11              | 0.2687      | 0.7746     | 20.14      |\n| TeaCache-fast                        | 2.52×         | 5.65              | 0.3456      | 0.7021     | 18.17      |\n| MagCache-slow                        | 2.57×         | 5.53              | **0.2043**  | **0.8883** | **24.46**  |\n| MagCache-fast                        | **2.82×**     | **5.05**          | 0.2635      | 0.8093     | 21.35      |"}, {"review_id": "yS8u9lx5ft", "reviewer": "Reviewer_9jpp", "summary": "This paper presents MagCache, a novel magnitude-aware caching strategy that efficiently accelerates video generation in diffusion models with minimal calibration cost. Extensive experiments on models like Open-Sora and Wan 2.1 demonstrate its effectiveness, and the method shows potential as a plug-and-play solution for faster inference.", "questions": "1. How should δ and K be selected when applying MagCache to other video generation models? Are there any heuristics or data-driven methods that guide this choice? Is the cost high?\n2. How does the type of video content (e.g., human, natural scene, dynamic textures) influence the selection of γ and the effectiveness of MagCache?\n3. How sensitive is MagCache to different prompt types? Does prompt complexity or semantic category affect caching behavior or performance?", "limitations": "Yes. The authors have clearly acknowledged the limitations of their approach and briefly discussed potential negative societal impacts. Their transparency is appreciated.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. The paper proposes MagCache, a novel magnitude-aware caching mechanism designed to accelerate video generation in diffusion models.\n2. Extensive experiments demonstrate that MagCache achieves notable speedups on state-of-the-art video diffusion models such as Open-Sora (2.1×) and Wan 2.1 (2.68×), showcasing its practical effectiveness.\n3. The method requires only a single random sample for calibration, significantly reducing both computational overhead and calibration time.\n\nWeaknesses:\n1. Insufficient motivation and generality. While MagCache aims to accelerate inference by exploiting motion-aware magnitude patterns, the empirical motivation presented in Section 3.2 is not fully convincing. In video generation, various motion types, magnitudes, and semantic content (e.g., human vs. landscape) can significantly affect inference dynamics. However, the current analysis relies only on two video models (Open-Sora, Wan 2.1) and three prompts, which is limited in scope.\n\n1.1 The choice of models is too narrow to support the claim that MagCache is a \\textbf{unified}, plug-and-play solution.\n\n1.2 Figure 1 uses only three prompts, which is insufficient to substantiate the prompt-invariance claim made in Section 3.2.\n\n2. Lack of theoretical insight into Equation (5). While the empirical benefits of the magnitude ratio are shown, the underlying mechanism and theoretical justification remain under-explored. A deeper analysis would strengthen the motivation and reveal how MagCache influences the denoising dynamics at a more principled level.\n\n3. Parameter sensitivity and lack of consistency. In Section 3.3.2, critical parameters δ and K are introduced without adequate justification or guidance. Moreover, the settings for δ and K vary across experiments (e.g., K = 1, 2, 3, 4), indicating a lack of standardized configuration. The absence of a consistent selection rule raises concerns about reproducibility, robustness, and generalization.\n\n4. Figure 2 is difficult to interpret and lacks clarity in both visual design and explanation.", "was_revised": true, "final_justification": "The authors’ rebuttal has resolved some of my earlier concerns and provided helpful clarifications. Accordingly, I am revising my score from borderline reject to borderline accept.", "rebuttal": "**Q1:** Insufficient motivation and generality. While MagCache aims to accelerate inference by exploiting motion-aware magnitude patterns, the empirical motivation is not fully convincing. The current analysis on two video models and three prompts is limited in scope.\n\n**A1:** Thanks for the insightful feedback. **Our core motivation is to accelerate video generation by exploiting redundancy in residuals across adjacent steps.** Emperically, the residual difference is small in the first 80% steps across diverse models and prompts. Notably, MagCache is step-aware, not motion-aware. Both motion and semantics evolve minimally between early adjacent steps. **Besides, we extend experiments to include more models (Table R3.1) and a broader set of prompts (Table R3.2 & R3.3) to demonstrate the effectiveness and generality of MagCache.** \n\n\n\n**Q1.1:** Model diversity is limited to support the plug-and-play claim.\n\n**A1.1:** To strengthen the generality, we integrate MagCache to HunyuanVideo, CogVideoX, and Flux (Table R3.1), observing competitive or superior performance across these models. We also qualitatively validated MagCache on additional models (FramePack, VACE, Flux Kontext, Chroma, and OmniGen2), and will release supporting code and demos. **In summary, MagCache is a unified, plug-and-play acceleration method applicable to a wide range of diffusion models.**\n\n\n\n**Table R3.1 Generalization of MagCache across Diverse Diffusion Models.**\n\n| Method                                      | Speedup ↑ | SSIM ↑ | PSNR ↑ |\n| ------------------------------------------- | --------- | ------ | ------ |\n| **HunyuanVideo (T = 50, 129 frames, 540P)** | 1         | -      | -      |\n| TeaCache-slow                               | 1.6       | 0.787  | 23.8   |\n| TeaCache-fast                               | 2.2       | 0.774  | 23.3   |\n| MagCache-slow                               | 2.2       | 0.945  | 34.5   |\n| MagCache-fast                               | 2.6       | 0.920  | 31.7   |\n| **CogVideoX2B (T = 50, 49 frames, 480P)**   | 1         | -      | -      |\n| TeaCache                                    | 2.3       | 0.881  | 27.0   |\n| MagCache                                    | 2.3       | 0.921  | 30.4   |\n| **Flux (T = 28)**                           | 1         | -      | -      |\n| TeaCache-slow                               | 2.0       | 0.774  | 20.1   |\n| TeaCache-fast                               | 2.5       | 0.702  | 18.1   |\n| MagCache-slow                               | 2.5       | 0.888  | 24.4   |\n| MagCache-fast                               | 2.8       | 0.809  | 21.3   |\n\n---\n\n**Q1.2:** Figure 1 uses only three prompts,  insufficient to substantiate the prompt-invariance.\n\n**A1.2:** To assess prompt invariance, we analyzed magnitude ratios over **944 VBench prompts**, grouped into five 20% diffusion intervals. **As shown in Table R3.2, the magnitude ratios exhibit low standard deviation and a consistent decreasing trend,** with a sharp drop in the final 20%. Cosine distances follow a similar pattern, aligning with Figure 1.\n\nFurther, Table R3.3 demonstrates that the choice of calibration prompt has negligible effect. Whether using a random prompt, the full prompt set, or an outlier, performance remains stable. **In summary, magnitude ratios are consistent across prompts, and MagCache is robust to prompt choice.**\n\n\n\n**Table R3.2 Statistics of Magnitude Ratios and Cosine Distance across 944 Vbench prompts.** `0.9965 (± 0.0004) denotes mean ± standard deviation`\n\n| Diffusion Process (%) | Magnitude Ratio (Wan2.1 ) | Cosine Distance (Wan2.1) |\n| --------------------- | ------------------------- | ------------------------ |\n| 0–20                  | 0.9965 (±0.0004)          | 0.0008 (±0.0003)         |\n| 20–40                 | 0.9951 (±0.0002)          | 0.0003 (±0.0003)         |\n| 40–60                 | 0.9928 (±0.0002)          | 0.0001 (±0.0001)         |\n| 60–80                 | 0.9851 (±0.0002)          | 0.0002 (±0.0001)         |\n| 80–100                | 0.9285 (±0.0019)          | 0.0054 (±0.0009)         |\n\n**Table R3.3 Effect of Calibration Prompt.**\n\n| Calibration Prompt                     | **Speedup ↑** | **SSIM ↑** | **PSNR ↑** |\n| -------------------------------------- | ------------- | ---------- | ---------- |\n| Random Prompt (Ours)                   | 2.1           | 0.813      | 23.4       |\n| 944 Prompts (Average Magnitudes)       | 2.1           | 0.816      | 23.5       |\n| Outlier Prompt (Farthest from Average) | 2.1           | 0.810      | 23.3       |\n\n---\n\n**Q2:** Lack of theoretical insight into Equation (5).\n\n**A2:** Thanks for the valuable comment. Equation (5) states that when the cosine distance between adjacent residuals is small, the residual difference $||r\\_t - r\\_{t-1}||$ can be approximated by the magnitude difference $\\left|||r\\_t|| - ||r\\_{t-1}||\\right|$:\n$$\n||r_t - r_{t-1}|| \\approx \\left|||r_t|| - ||r_{t-1}||\\right|. \\tag{5}\n$$\n\nThis holds when $r\\_t$ and $r\\_{t-1}$ are nearly colinear (i.e., $1 - \\cos(r\\_t, r\\_{t-1}) \\approx 0$). Empirically, this occurs in the first 80% of steps. Theoretically, we can analysize the cosine law:\n\n$$\n||r_t - r_{t-1}||^2 = (||r_t|| - ||r_{t-1}||)^2 + 2||r_t||||r_{t-1}||(1 - \\cos(r_t, r_{t-1})).\n$$\n\nAs $1 - \\cos(r\\_t, r\\_{t-1}) \\to 0$, the second term vanishes, yielding Eq. (5). Let $r(t) = v\\_\\theta(x\\_t, t) - x\\_t$. By the chain rule: \n$$\n\\frac{dr}{dt} = (\\partial_x v_\\theta - I) x'(t) + \\partial_t v_\\theta.\n$$\nAssuming $||x'\\_t|| \\le M$, spatial lipschitz continuity $||\\partial\\_x v\\_\\theta|| \\le L\\_x$, and temporal lipschitz continuity $||\\partial\\_t v\\_\\theta|| \\le L\\_t$, we get\n\n\n\n$$\n\\left|\\left|\\frac{dr}{dt}\\right|\\right| \\le (L_x + 1)M + L_t = B,\n$$\n\nwhere $B$ aggregates the ODE’s stiffness ($M$) and model smoothness ($L_x, L_t$), which is usually within a small constant range. Integrating over step size $h$(small in early steps), we can get $||r_t - r_{t-1}|| \\le Bh.$ Assuming $||r\\_t||, ||r\\_{t-1}|| \\ge c > 0$, we get:\n\n$$\n1 - \\cos(r_t, r_{t-1}) = \\frac{||r_t - r_{t-1}||^2 -(||r_t|| - ||r_{t-1}||)^2}{2||r_t||||r_{t-1}||}\\le \\frac{||r_t - r_{t-1}||^2}{2||r_t||||r_{t-1}||} \\le \\frac{B^2 h^2}{2c^2} = O(h^2).\n$$\n\nHence, $r\\_t$ and $r\\_{t-1}$ are nearly aligned when $h$ is small. In non-uniform schedules (e.g., shift=8), early steps use small $h$ (e.g., 0.001), making $1 - \\cos(r\\_t, r\\_{t-1})$ negligible, validating Equation (5). \n\n---\n\n**Q3:** Parameter sensitivity and lack of consistency. \n\n**A3:** The varying $\\delta$ and $K$ settings across models were chosen to ensure fair comparisons at similar speedups. In practice, **MagCache performs robustly with default parameters across different models**, requiring only minor $\\delta$ adjustments. We offer two predefined modes:\n\n- **Slow mode: $K = 2$, $\\delta = 0.06$ (default)**\n- **Fast mode: $K = 4$, $\\delta = 0.06$ (default)**\n\nHere, $K$ controls speedup mode, and $\\delta$ adjusts the quality–speed trade-off within the selected mode. In Table R1.1, varying $\\delta$ offers fine control with just 1–2 adjustments (doubling or halving). **We recommend the slow mode ($K=2, \\delta=0.06$), which works well for various models.** **Our ComfyUI interface allows users to easily adjust these parameters in a few seconds.**\n\n\n\n**Table R3.4 Parameter Configurations in Slow and Fast inference mode.**\n\n| **Mode**                     | $K$   | $\\delta$ | **Speedup ↑** | **SSIM ↑** | **PSNR ↑** |\n| :--------------------------- | ----- | -------- | ------------- | ---------- | ---------- |\n| **MagCache-slow (Wan2.1)**   | **2** | **0.06** | 2.0           | 0.838      | 24.5       |\n|                              |       | 0.12     | 2.1           | 0.8275     | 24.3       |\n|                              |       | 0.03     | 1.9           | 0.842      | 24.6       |\n| **MagCache-fast (Wan2.1)**   | **4** | **0.06** | 2.4           | 0.774      | 22.3       |\n|                              |       | 0.12     | 2.7           | 0.757      | 22.2       |\n|                              |       | 0.03     | 2.0           | 0.782      | 22.5       |\n| **MagCache-slow (OpenSora)** | **2** | **0.06** | 1.8           | 0.814      | 23.9       |\n|                              |       | 0.12     | 1.9           | 0.813      | 23.8       |\n|                              |       | 0.03     | 1.7           | 0.816      | 24.0       |\n| **MagCache-fast (OpenSora)** | **4** | **0.06** | 2.1           | 0.763      | 21.9       |\n|                              |       | 0.12     | 2.4           | 0.754      | 21.7       |\n|                              |       | 0.03     | 2.0           | 0.766      | 22.0       |\n\n---\n\n**Q4:** Figure 2 is difficult to interpret and lacks clarity in both visual design and explanation.\n\n**A4:** We have revised Figure 2 for improved clarity. It now better illustrates how $K$ and $\\delta$ jointly govern the adaptive caching and reusing, with annotations and a step-wise flow to highlight the caching logic. Due to the rebuttal policy, we will update the figure in the revised paper.\n\n---\n\n**Q5:** How does the type of video content (e.g., human, natural scene, dynamic textures) influence the the effectiveness of MagCache? How sensitive is MagCache to different prompt types? \n\n**A5:** We group the 944 Vbench prompts into 5 types. Table R3.5 shows that prompt types have negligible effect on MagCache’s performance. Stable magnitude ratios across prompts (Table R3.2) and calibration robustness (Table R3.3) also support this conclusion.\n\n**Table R3.5 Robustness of MagCache across Different Prompt Types.**\n\n| Prompt Type     | **SSIM ↑** | **PSNR ↑** |\n| --------------- | ---------- | ---------- |\n| Human           | 0.824      | 23.4       |\n| Objects         | 0.812      | 23.5       |\n| Natural Scene   | 0.812      | 23.2       |\n| Dynamic Motion  | 0.805      | 23.0       |\n| Fictional Scene | 0.828      | 23.4       |\n| Average         | 0.813      | 23.4       |"}, {"review_id": "gmTEgMVBr3", "reviewer": "Reviewer_YTog", "summary": "This paper introduces MagCache, a novel caching strategy to accelerate video generation in diffusion models. MagCache leverages a newly discovered law: the magnitude ratio of successive residual outputs decreases monotonically across different models and prompts. By employing an error modeling mechanism and adaptive caching strategy, it adaptively skips unimportant timesteps, thereby significantly boosting inference speed while maintaining high visual fidelity.", "questions": "See weakness", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths：\n\n1. This paper proposed a novel unified magnitude decay law that reveals the stability and monotonicity of the residual magnitude ratios in video diffusion models, providing a theoretical basis for skipping timesteps.\n\n2. MagCache performs excellently on multiple benchmark models (such as Open-Sora and Wan 2.1), achieving significant acceleration (up to 2.68×) and superior visual quality compared to existing methods.\n\n3. MagCache can be seamlessly integrated into existing diffusion model pipelines without requiring additional training or large amounts of data.\n\n4. The writing of this paper is clear and easy to follow.\n\n\nWeaknesses&Question：\n\nCompatibility with other acceleration methods: Can MagCache be integrated with other diffusion model acceleration techniques, such as model distillation and low-precision arithmetic?", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the valuable feedback. **MagCache is a unified, plug-and-play acceleration method compatible with a wide range of video generation models**, including Wan 2.1, Open-Sora, HunyuanVideo, CogVideoX, few-step distilled models (FusionX), and low-precision models. It supports ComfyUI interface for ease of use.\n\n---\n\n**Q1:** Compatibility with other acceleration methods: Can MagCache be integrated with other diffusion model acceleration techniques, such as model distillation and low-precision arithmetic?\n\n**A1:** Thanks for the insightful question. **MagCache is compatible with other acceleration techniques, including model distillation and low-precision arithmetic.**\n\nAlthough distilled models tend to exhibit larger variations between denoising steps, **MagCache remains effective even for few-step distilled models by caching and reusing intermediate computations with minimal differences.** As shown in Table R4.1, we evaluated MagCache on FusionX (a 10-step distilled version of Wan2.1 14B) and achieved a 1.66× speedup without significant degradation in visual quality. Since distillation has reduced redundancy between steps, the speedup ratio of MagCache (1.66×) on the distilled model (FusionX) is lower than that on the non-distilled version (Wan 2.1). Notably, simply reducing the inference steps to T=6 yields much worse performance than using MagCache with T=10, even at the same speedup of 1.66×.\n\nWe also evaluated MagCache under low-precision settings. **As shown in Table R4.2, MagCache is compatible with 4-bit quantized models**, maintaining acceleration benefits with only a minor drop in speedup due to the reduced numerical precision. Both the “fast” and “slow” variants maintain strong visual quality when used with low-precision models.\n\n---\n\n**Table R4.1: Evaluation of MagCache on Few-Step Distilled Model (Wan 2.1 14B FusionX, 33 frames, 480P)**\n\n| **Method**                                         | **Speedup ↑** | **Latency (s) ↓** | **LPIPS ↓** | **SSIM ↑** | **PSNR ↑** |\n| -------------------------------------------------- | ------------- | ----------------- | ----------- | ---------- | ---------- |\n| Wan2.1 14B FusionX (T = 10)                        | 1×            | 30                | -           | -          | -          |\n| Wan2.1 14B FusionX (T = 6)                         | 1.66x         | 18                | 0.2982      | 0.6471     | 20.35      |\n| Wan2.1 14B FusionX + MagCache (T=10, skip 4 steps) | **1.66×**     | **18**            | **0.1812**  | **0.7868** | **24.23**  |\n\n---\n\n**Table R4.2: Evaluation of MagCache on Low-precision Arithmetic Model (Wan 2.1 14B, 33 frames, 480P)**\n\n| **Method**                             | Memory | **Speedup ↑** | **Latency (s) ↓** | **LPIPS ↓** | **SSIM ↑** | **PSNR ↑** |\n| -------------------------------------- | ------ | ------------- | ----------------- | ----------- | ---------- | ---------- |\n| Wan2.1 14B 4bit (T = 30)               | 26.3G  | 1×            | 241               | -           | -          | -          |\n| Wan2.1 14B 4bit (T = 30)+MagCache-fast | 26.5G  | 2.0x          | 119               | 0.2223      | 0.6780     | 20.42      |\n| Wan2.1 14B 4bit (T = 30)+MagCache-slow | 26.5G  | 1.4x          | 169               | 0.1184      | 0.7902     | 22.78      |"}]

Author rebuttal:

Meta-review: The paper was favorably reviewed by 4 experts, that recommended acceptance. The paper proposes a testing time speed up method for video diffusion models, that takes into consideration the magnitude residuals across time steps. The method gets 2x acceleration on most recent models, compared to prior works in this area (TeaCache). The AC agrees and recommends acceptance. Congrats!
Decision: Accept (poster)