LightCache: Memory-Efficient, Training-Free Acceleration for Video Generation

## Abstract

Abstract Training-free acceleration has emerged as an advanced research area in video generation based on diffusion models. The redundancy of latents in diffusion model inference provides a natural entry point for acceleration. In this paper, we decompose the inference process into the encoding, denoising, and decoding stages, and observe that cache-based acceleration methods often lead to substantial memory surges in the latter two stages. To address this problem, we analyze the characteristics of inference across different stages and propose stage-specific strategies for reducing memory consumption: 1) Asynchronous Cache Swapping. 2) Feature chunk. 3) Slicing latents to decode. At the same time, we ensure that the time overhead introduced by these three strategies remains lower than the acceleration gains themselves. Compared with the baseline, our approach achieves faster inference speed and lower memory usage, while maintaining quality degradation within an acceptable range. The Code is available at https://github.com/NKUShaw/LightCache .

## 1 Introduction

Diffusion models have recently received significant global attention due to their impressive generative capabilities Croitoru et al. (2023); Yang et al. (2023); Rombach et al. (2022). These models have demonstrated remarkable efficacy in various applications, like the generation of images Peebles & Xie (2023); Ho et al. (2020); Song et al. (2020), language Hoogeboom et al. (2021); Austin et al. (2021); Yang et al. (2025), and video Blattmann et al. (2023); Guo et al. (2023); Yang et al. (2024). However, due to the problems of high computational time cost and large memory usage, researchers are often seriously restricted to their implementation and deployment in the real world Ma et al. (2024a). Especially in the video generation task process, the model must simultaneously perform a denoising action on all frames, leading to a high GPU memory usage and a long inference latency Xing et al. (2024). To solve this problem, existing studies have attempted to achieve acceleration through training Luo et al. (2023); Lyu et al. (2022); Yin et al. (2024); Wang et al. (2023); Kim et al. (2023); Ma et al. (2024a); Wimbauer et al. (2024); Agarwal et al. (2024) and training-free methods Ma et al. (2024b); Zhan et al. (2024); Yu et al. (2023); Li et al. (2023b; a).

Figure: Figure 2: Highly similar feature map between all up-sampling and down-sampling layers
Refer to caption: plots/motivation/redunt.jpg

Figure: Figure 3: Memory Usage
Refer to caption: x1.png

Training methods require a lot of training resources and may have a negative impact on the generation performance of the original model Li et al. (2023c); Ma et al. (2024b); Yang et al. (2023); Ma et al. (2024a); Zhou et al. (2025). In contrast, training-free methods for acceleration have focused on two main strategies Ma et al. (2024a); Liu et al. (2025): 1) skip certain time steps in the inference stage Ma et al. (2024b), and 2) decrease the inference overhead per step through methods Zhan et al. (2024); Yu et al. (2023); Li et al. (2023b). The inherent redundancy in the feature maps during the denoising stage provides a natural entry point for skipping certain inference steps Li et al. (2023a). Based on our experimental observations and a review of existing literature Ma et al. (2024b); Li et al. (2023a); Tian et al. (2025); Li et al. (2023b; 2025), we found that adjacent denoising steps often produce highly similar intermediate representations (See in Figure [2](https://arxiv.org/html/2510.05367v1#S1.F2)), particularly in the later stages of inference when the signal becomes clearer and most of the noise has been eliminated. This temporal coherence suggests that many feature maps encode overlapping semantic information, thereby enabling the possibility of reducing the number of inference steps without significantly compromising the quality of the generated results.

Our research is based on a cache mechanism that skips the computation of certain feature maps at specific timesteps to accelerate video generation. However, the existence of caching directly leads to a surge in GPU memory usage (See in Figure [3](https://arxiv.org/html/2510.05367v1#S1.F3)). For example, when we applied DeepCache to EasyAnimate (a DiT architecture) using 4 L40 GPUs (45 GB each), we immediately encountered an out-of-memory issue. This happens because model inference differs from model training. During training, the common practice is either data parallelism or model parallelism (tensor/layer parallelism). In data parallelism, a full copy of the model is placed on each GPU, while the sample batch is evenly divided, leading to relatively balanced memory usage. In model parallelism, the model is split across GPUs, so memory is also distributed more evenly. However, during model inference/sampling, if we only generate a single sample (batch = 1), there is no natural way to distribute the workload as in training. Multi-GPU parallelism can only assign different modules to different GPUs, but the memory usage of each module is not the same. If a single module (e.g., U-Net or DiT) already exceeds the capacity of a single GPU, then even with multi-GPU parallelism, out-of-memory errors will occur.

Therefore, we also hope to reduce the GPU burden of the inference stage. To explore the sources of computation and memory cost, we divide the inference into three stages: encoding, denoising, and decoding stages, and measured the peak memory(See in Figure [3](https://arxiv.org/html/2510.05367v1#S1.F3)).
That is, the memory requirement for diffusion inference depends on the highest memory of the three stages.
For example, when we applied the method Zhan et al. (2024) to reduce memory usage, we found that the peak memory did not decrease (We use the default setting in the diffusers library). According to our experiments, this method focuses on the denoising stage (See in Tab. [2](https://arxiv.org/html/2510.05367v1#S2.T2)), while the peak memory of some models or some experimental settings is in the decoding stage(See in Figure [3](https://arxiv.org/html/2510.05367v1#S1.F3)). Empirically, regardless of whether DeepCache Ma et al. (2024b) is used for accelerating inference, the memory usage of the encoding stage remains consistently low and unchanged. Therefore, memory optimization efforts should focus on the denoising and decoding stages.

We propose the following optimization strategy (See in Figure [4](https://arxiv.org/html/2510.05367v1#S1.F4)):

- •
Chunk: The denoising stage primarily involves the computation of feature maps. We propose that selectively chunking the height and width of feature maps from different layers to varying degrees can effectively reduce memory usage while maintaining generation quality.
- •
Asynchronous Swapping: The model offloading function von Platen et al. (2022) in the Diffusers library moves inactive layers or models to the CPU to reduce GPU memory usage. Following a similar idea, swapping inactive cached feature maps to the CPU could also alleviate memory pressure. To further mitigate the latency overhead, asynchronous swapping can be employed: feature maps can be transferred between GPU and CPU in the background while the GPU continues computing other parts of the model, thereby overlapping communication and computation.
- •
Slicing: The decoding stage can be optimized by splitting large inputs into smaller batches and processing them sequentially. In our case, although we generate only a single video, the generative models we adopt are classifier-free guidance models Ho & Salimans (2022). This naturally doubles the number of feature maps (conditional and unconditional) and requires denoising across multiple frames simultaneously (a total of $2\times N$, where $N$ is the number of frames). Thus, VAE slicing remains highly applicable.

Figure: Figure 4: Denoising: The timestep is divided into a cache step and a normal step. The cache step reuses the cached feature maps stored in the CPU. The normal step chunk the size of feature maps and concatenate them. Decoding: We use VAE to decode the latent frame by frame.
Refer to caption: plots/overview.png

## 2 Methodology

### 2.1 Preliminary

#### Diffusion Forward Process.

The core idea of diffusion models Ho et al. (2020) is to remove noise from fully corrupted data in order to recover valid semantic information using a series of timesteps. Its essence is to learn the transformation of noise distribution to ground truth distribution. For the data $\mathbf{x}$, we can gradually add Gaussian noise to the data in T steps:

$$ $q(\mathbf{x}_{t}\mid\mathbf{x}_{t-1})=\mathcal{N}(\mathbf{x}_{t};\sqrt{1-\beta_{t}}\mathbf{x}_{t-1},\beta_{t}\mathbf{I})$ (1) $$

where $t$ belongs to $[0,T]$, and $[\beta_{0},...,\beta_{T}]$ schedule the noise.

#### Diffusion Reverse Process.

To recover the valid semantic information, we train the diffusion model to denoise the randomly sampled noise into the ground truth distribution. We use a network (For example, U-Net Ronneberger et al. (2015)) $\epsilon(\mathbf{x}_{t},t)$to predict noise:

$$ $p_{\theta}(\mathbf{x}_{t-1}\mid\mathbf{x}_{t})=\mathcal{N}(\mathbf{x}_{t-1};\frac{1}{\sqrt{\alpha_{t}}}(\mathbf{x}_{t}-\frac{1-\alpha_{t}}{\sqrt{1-\bar{\alpha}_{t}}}\epsilon_{\theta}(\mathbf{x}_{t},t)),\beta_{t}\mathbf{I})$ (2) $$

where $\alpha_{t}=1-\beta_{t}$ and $\bar{\alpha}_{t}=\prod_{i=1}^{T}\alpha_{i}$. Applied iteratively, it gradually removes the noise of the current $\mathbf{x}_{t}$, bringing it close to a real data point when we reach $\mathbf{x}_{0}$.

#### Feature Calculation

U-Net Ronneberger et al. (2015), originally proposed for medical image segmentation, uses skip connections to effectively combine low- and high-level features. Its architecture is built on successive down-sampling and up-sampling blocks that first encode the input into a compact representation and then decode it for downstream applications:

$$ $\{D_{i}\}^{m}_{i=0},\{U_{i}\}^{m}_{i=0}$ (3) $$

Down-sampling is the encoding stage, which extracts the information from the previous layer through the $\{D_{i}\}^{m}_{i=0}$. Up-sampling is the decoding stage, which converges the inversion from the previous layer and the skip-connection:

$$ $U_{i}=\text{Concat}(D_{i},U_{i+1})$ (4) $$

Therefore, at the heart of a U-Net model is a concatenation of low-level features from the skip connection and the high-level features from the previous layer.

### 2.2 Asynchronous Swapping

We introduce DeepCache Ma et al. (2024b), a simple yet effective method that exploits temporal redundancy between steps in the diffusion reverse process to accelerate inference, serving as the foundation of our approach. As the cacheing mechanism Smith (1982) in a computer system and KVCache in transformer Pope et al. (2023), DeepCache method caches the slowly evolving features $F^{t}$ to eliminate some unnecessary computations in some steps:

$$ $\displaystyle F^{t}_{cache}$ $\displaystyle\leftarrow D^{t}_{m},\text{Concat}(D^{t}_{m},U^{t}_{m+1})$ (5) $\displaystyle D^{t+1}_{m},U^{t+1}_{m}$ $\displaystyle\leftarrow F^{t}_{cache}$ $$

Where $U^{t}_{m}\text{ or }D^{t}_{m}$ means the $m$-th up-sampling layer or down-sampling layer at time step $t$.

For example, in the case of an up-sampling layer, we set a hyperparameter $n$, meaning that the feature map is explicitly recalculated once every $n$ steps, while in the remaining steps it is reused from the cache: $F^{t}_{cache}:U^{0}_{m},U^{n}_{m},U^{2n}_{m},\dots$. There is no additional computational cost to these time steps: $\{(1,\dots,n-1);(n+1,\dots,2n-1);\dots\}$, since it can be simply retrieved from the cached feature map. Although DeepCache enables feature map reuse for acceleration, the additional cached feature maps lead to a substantial increase in memory usage.

In the official diffusers library, CPU offloading von Platen et al. (2022) selectively transfers weights between the GPU and CPU. When a component is needed, it is moved to the GPU, and when it is not needed, it is offloaded back to the CPU. This method operates on submodules rather than entire models, thereby saving memory by avoiding storing the whole model on the GPU. Inspired by this idea, we store the cached features on the CPU and bring them back when required. This approach does not affect generation quality, though it slightly increases inference time, while significantly reducing GPU memory usage.

Figure: Figure 5: Violin Figure (AnimateDiff-Light) of Quality Metric: LPIPS(↓), PSNR(↑), SSIM(↑)
Refer to caption: x2.png

Figure: Figure 6: Violin Figure (SVD) of Quality Metric: LPIPS(↓), PSNR(↑), SSIM(↑)
Refer to caption: x3.png

### 2.3 Feature chunk and Sliced Decoding

So far, we have not modified the fundamental feature process of the diffusion model. The input $F^{t}_{m}\in\mathbb{R}^{B\times T\times C\times H\times W}$ of (m+1)-th layers is a 5-D feature map with dimensions {batch, frames, channels, height, width}, where the batch size is 2, corresponding to the conditional and unconditional feature maps in the classifier-free guidance model. Zhan et al. (2024) found that directly modifying the batch or frame dimensions would break the spatiotemporal continuity of video generation, leading to distorted temporal information extraction. On the other hand, directly modifying the channel dimension would cause misalignment with the model architecture. Therefore, they keep the first three dimensions unchanged and apply chunking only to the spatial dimensions (height and width):

$$ $Chunk(F^{t}_{m})\in\mathbb{R}^{B\times T\times C\times\frac{H}{\eta}\times\frac{W}{\omega}}$ (6) $$

where $\eta$ and $\omega$ are two hyperparameters. Larger values lead to smaller feature map sizes, which reduce memory consumption but may degrade generation quality. This presents a trade-off issue, requiring a balance between memory efficiency and output quality.

**Table 1: Result: DeepCache leads to a surge in memory usage, while FME decreases the speed-up, and both may cause quality degradation across different models. In contrast, our approach not only preserves quality but also accelerates inference and reduces memory consumption to below the baseline model. N means that the feature map is recalculated once every N steps.**
| Method | Time (s) | Speed Up | LPIPS (↓) | PSNR (↑) | SSIM (↑) | Peak Memory | Peak Reserved |
| --- | --- | --- | --- | --- | --- | --- | --- |
| AnimateDiff-Light (U-Net) 8 steps |  |  |  |  |  |  |  |
| Euler | 3.84 | 1.00× | - | - | - | 13562 | 23430 |
| DeepCache (N=2) | 2.87 | 1.34× | 0.164 | 28.64 | 0.7152 | 15200 | 24744 |
| FME | 4.07 | 0.95× | 0.609 | 27.81 | 0.4064 | 4868 | 5688 |
| Our (N=2) | 2.89 | 1.33× | 0.164 | 28.64 | 0.7152 | 5559 | 6238 |
| Our (N=3) | 2.41 | 1.59× | 0.205 | 28.42 | 0.6654 | 5559 | 6238 |
| SVDI2V-XT (U-Net) 25 steps |  |  |  |  |  |  |  |
| Euler | 47.39 | 1.00× | - | - | - | 15346 | 31084 |
| DeepCache (N=2) | 28.65 | 1.65× | 0.039 | 34.32 | 0.9262 | 21416 | 37570 |
| FME | 57.23 | 0.83× | 0.018 | 37.10 | 0.9649 | 10148 | 11552 |
| Our (N=2) | 30.92 | 1.53× | 0.031 | 34.33 | 0.9271 | 13937 | 15964 |
| Our (N=3) | 24.55 | 1.93× | 0.061 | 33.55 | 0.9189 | 13937 | 15964 |
| Our (N=4) | 21.27 | 2.23× | 0.074 | 32.22 | 0.8837 | 13937 | 15964 |
| Our (N=8) | 16.57 | 2.86× | 0.251 | 28.84 | 0.8345 | 13937 | 15964 |
| AnimateDiff (U-Net) 50 steps |  |  |  |  |  |  |  |
| Euler | 33.50 | 1.00× | - | - | - | 13564 | 23944 |
| DeepCache (N=2) | 20.67 | 1.62× | 0.127 | 31.75 | 0.9017 | 16843 | 27576 |
| FME | 35.86 | 0.93× | 0.556 | 28.46 | 0.5325 | 5898 | 6688 |
| Our (N=2) | 22.33 | 1.50× | 0.127 | 31.75 | 0.9017 | 8583 | 9908 |
| Our (N=3) | 18.09 | 1.85× | 0.172 | 30.55 | 0.8581 | 8583 | 9908 |
| Our (N=4) | 15.96 | 2.10× | 0.243 | 30.08 | 0.8417 | 8583 | 9908 |
| TextToVideo (U-Net) 25 steps |  |  |  |  |  |  |  |
| Euler | 5.03 | 1.00× | - | - | - | 7334 | 10814 |
| DeepCache (N=2) | 3.45 | 1.46× | 0.226 | 29.57 | 0.6856 | 8157 | 11592 |
| FME | 7.55 | 0.67× | 0.039 | 35.17 | 0.8956 | 4048 | 4724 |
| Our (N=2) | 4.82 | 1.04× | 0.225 | 29.57 | 0.6842 | 3939 | 4606 |
| EasyAnimate (DiT) 25 steps |  |  |  |  |  |  |  |
| Euler | 78.63 | 1.00× | - | - | - | 34884.07 | 36830.00 |
| DeepCache (N=2) | - | - | - | - | - | OOM | OOM |
| FME | - | - | - | - | - | - | - |
| Our (N=2) | 59.88 | 1.31× | 0.410 | 28.39 | 0.0743 | 29593.33 | 31596.00 |
| Our (N=3) | 41.14 | 1.91× | 0.548 | 28.43 | 0.0583 | 29593.33 | 31596.00 |
| Our (N=5) | 25.94 | 3.03× | 0.590 | 28.43 | 0.0555 | 29593.33 | 31596.00 |

**Table 2: Abalation Study: We remove each of the proposed modules and demonstrate the effectiveness of our approach through the memory consumption changes observed across the three stages.**
| Method | Time (s) | LPIPS (↓) | PSNR (↑) | SSIM(↑) | Encode Memory | Denoise Memory | Decode Memory |
| --- | --- | --- | --- | --- | --- | --- | --- |
| AnimateDiff-Light 8 steps |  |  |  |  |  |  |  |
| DeepCache | 2.88 | 0.164 | 28.64 | 0.7152 | 3829.13 | 6845.11 | 15200.83 |
| Our (N=2) | 2.89 | 0.164 | 28.64 | 0.7152 | 252.46 | 5559.15 | 4183.40 |
| - swapping | 2.57 | 0.164 | 28.64 | 0.7152 | 3829.13 | 6845.11 | 6115.49 |
| - slicing | 3.46 | 0.164 | 28.64 | 0.7152 | 252.46 | 5559.15 | 11547.01 |
| - chunk | 3.34 | 0.164 | 28.64 | 0.7152 | 252.46 | 5559.15 | 4183.40 |
| SVDI2V-XT 25 steps |  |  |  |  |  |  |  |
| DeepCache | 28.69 | 0.031 | 34.33 | 0.9262 | 5924.08 | 17202.37 | 21416.39 |
| Our (N=2) | 30.92 | 0.031 | 34.33 | 0.9271 | 2987.84 | 13936.91 | 9254.12 |
| - swapping | 30.97 | 0.031 | 34.33 | 0.9272 | 5924.08 | 16639.12 | 11863.77 |
| - slicing | 31.03 | 0.031 | 34.33 | 0.9272 | 2987.84 | 15439.1 | 17269.52 |
| - chunk | 28.77 | 0.031 | 34.33 | 0.9262 | 2987.84 | 16002.1 | 9211.18 |
| AnimateDiff 50 steps |  |  |  |  |  |  |  |
| DeepCache | 20.57 | 0.127 | 31.75 | 0.9017 | 3827.81 | 9864.46 | 16842.62 |
| Our (N=2) | 22.33 | 0.127 | 31.75 | 0.9017 | 255.45 | 8583.48 | 5835.14 |
| - swapping | 20.57 | 0.127 | 31.75 | 0.9017 | 3827.81 | 9864.46 | 7757.28 |
| - slicing | 22.32 | 0.127 | 31.75 | 0.9017 | 255.45 | 8583.48 | 13198.53 |
| - chunk | 22.21 | 0.127 | 31.75 | 0.9017 | 255.45 | 8583.48 | 5835.14 |
| TextToVideo 25 steps |  |  |  |  |  |  |  |
| DeepCache | 3.44 | 0.227 | 29.57 | 0.6842 | 3532.90 | 5394.76 | 8157.18 |
| Our (N=2) | 4.82 | 0.225 | 29.57 | 0.6842 | 659.88 | 4279.35 | 3545.69 |
| - swapping | 4.49 | 0.227 | 29.57 | 0.6842 | 3532.90 | 5090.40 | 4522.34 |
| - slicing | 4.08 | 0.227 | 29.57 | 0.6842 | 659.88 | 4279.35 | 4804.40 |
| - chunk | 3.55 | 0.227 | 29.57 | 0.6842 | 659.88 | 4582.85 | 3539.35 |

We found that the feature chunk only works at the denoising stage based on the experiments. If the video memory usage of the decoding stage is higher than that of the denoising stage, even if we reduce the memory usage of the denoising stage to 0, the overall peak memory will not change. Previously, we optimized the height and width dimensions of the five dimensions {batch, frames, channels, height, width}. While we can’t prune the channel dimension without changing the model architecture, we can consider the batch and frames dimensions:

$$ $F\in\mathbb{R}^{(B\times T)\times C\times H\times W}$ (7) $$

In the decoding stage, the decoding function will merge {batch, frames} to $batch\times frames$. At this time, the model has integrated conditional and unconditional information. For one sample, $batch=1$, so it is equivalent to processing all the frames at once:

$$ $F\in\mathbb{R}^{T\times C\times H\times W}$ (8) $$

Obviously, the amount of GPU memory at this time is extremely large, and the methods we introduced earlier do not involve calculations at this stage. We consider the video as a combination of multiple pictures and use VAE slicing von Platen et al. (2022) to decode the video frame by frame. It should be noted that this method turns a single computation into multiple computations, but under the acceleration of DeepCache, the time consumed by decoding is negligible.
Finally, we merge the all the images $\hat{I}_{n}$ to get the video:

$$ $\hat{I}_{0:T}=\{VAE(F_{n})\}^{T}_{n=0},F_{n}\in\mathbb{R}^{C\times H\times W}$ (9) $$

## 3 Experiments

### 3.1 Analysis

We adopted the default Euler scheduler and compared DeepCache, FME, and our method. For prompt, we use the default prompt (Image and Text) in the huggingface von Platen et al. (2022) to generate the video. All experiments were conducted on four NVIDIA L40S GPUs (45GB each) with a batch size of 1. The resolution was set to 512×512 for all methods, except for SVD where 1024×576 was used. The number of inference steps was fixed at 8 for AnimateDiff-Light, 25 for StableVideoDiffusion and TextToVideo, and 50 for AnimateDiff. In addition, the earliest version of DeepCache focused only on accelerating U-Net architecture models. Although Fora claims to have accelerated DiT architecture models, the open-source code is incomplete, and Fora is for image generation, so we extend DeepCache to DiT architecture models (EasyAnimate).
However, since the original model already had excessively high memory consumption, running DeepCache resulted in out-of-memory (we use 4 L40S, 45GB per GPU) issues. Moreover, FME is not applicable to DiT architectures, so no experimental results are available for either method.

In terms of acceleration, DeepCache (N=2) achieves a speed-up increase from 34% to 65% over the baseline, while Ours (N=2) achieves an increase from 4% to 53%. In contrast, FME does not provide acceleration and may even lead to a slowdown. Beyond acceleration, we further evaluate peak memory usage to understand the trade-offs introduced by different methods. The peak memory usage of DeepCache increases the most, by approximately 11% to 40%. Furthermore, as the number of generated frames increases, the peak memory consumption grows proportionally—for example, at 50 frames it reaches 30026MB, an increase of about 96%. FME markedly reduces peak memory by approximately 34% to 64%, with a change in speed-up from -33% to -5%, but leads to a substantial loss in generation quality. Our method provides a more balanced trade-off: maintaining quality (LPIPS, SSIM, and PSNR) close to DeepCache (More cases can be seen in [A](https://arxiv.org/html/2510.05367v1#A1)), while confirming that the acceleration does not come at the expense of perceptual quality and achieving faster speed—slightly lower than DeepCache—but with peak memory usage significantly lower than both the baseline and DeepCache. Due to the presence of cached feature maps, our peak memory is higher than that of FME. Overall, our approach consistently outperforms the baseline, DeepCache, and FME, offering the most favorable balance across speed, quality, and memory (see Table [1](https://arxiv.org/html/2510.05367v1#S2.T1)).

### 3.2 Ablation Studies

In this section, we conduct ablation studies to evaluate the effectiveness of our proposed method.
We first analyze the contribution of each module to peak memory reduction by selectively removing them and measuring their impact across different stages of the pipeline, then investigate whether our approach depends on the choice of sampling scheduler by comparing it with several alternatives.

#### Three Modules

To thoroughly investigate the impact of our proposed three modules on the overall peak memory usage, we designed a series of ablation experiments by selectively removing each module to isolate its contribution. Specifically, we first evaluated the baseline case using only DeepCache, and we observed that peak memory was mainly dominated by the denoising and decoding stages. Then we implemented our method and found a notable improvement, that the peak memory consumption decreased simultaneously across all three stages, demonstrating the collective effectiveness of the modules in reducing memory demand. To further compare the contribution of each module, we removed them individually and measured peak memory across the three stages. The results show that: 1) Removing swapping significantly increased peak memory in all three stages. Swapping, which is built upon the CPU offloading functionality of the diffusers library, involves transferring both model weights and cached feature maps. Its effect spans all stages. 2) Removing slicing significantly increased peak memory in the decoding stage. This is because VAE slicing processes frames sequentially rather than all at once during decoding. 3) Removing chunking significantly increased peak memory in the denoising stage. This is because only chunking is applied to feature maps specifically during the denoising process (See in Table [2](https://arxiv.org/html/2510.05367v1#S2.T2)).

#### Scheduler

We consistently adopted the official default sampling scheduler. In the ablation experiments, we also included DDIM and PNDM. The baseline runtime varied slightly across different schedulers, and after applying our method, the quality degradation also differed depending on the scheduler. Among them, the default Euler scheduler achieved the best performance on time cost, LPIPS, PSNR, and SSIM (See in Table [3](https://arxiv.org/html/2510.05367v1#S3.T3)). These experiments demonstrate that our method is not constrained by the choice of scheduler.

### 3.3 Visualization

We compared the frame-by-frame quality differences of videos generated by each method against the original model. Among the three metrics, PSNR and SSIM are more reflective of low-level pixel and structural similarity, while LPIPS better aligns with human perception. The results are visualized using violin plots, which show that LightCache maintains consistency with DeepCache, whereas FME produces unstable outcomes (See in Figure [5](https://arxiv.org/html/2510.05367v1#S2.F5) and [6](https://arxiv.org/html/2510.05367v1#S2.F6)).

**Table 3: Different sampler on SVDI2V-XT, * means this sampler is the official default sampler.**
| Method | Time (s) | Speed Up | LPIPS (↓) | PSNR (↑) | SSIM (↑) | Peak Memory | Peak Reserved |
| --- | --- | --- | --- | --- | --- | --- | --- |
| DDIM | 58.07 | 1.00× | - | - | - | 15789 | 27626 |
| + Ours(N=2) | 37.22 | 1.56× | 0.264 | 31.72 | 0.7843 | 14078 | 18452 |
| PNDM | 60.32 | 1.00× | - | - | - | 15352 | 27626 |
| + Ours(N=2) | 35.50 | 1.699× | 0.505 | 31.13 | 0.6344 | 14133 | 20146 |
| Euler* | 47.39 | 1.00× | - | - | - | 15346 | 31084 |
| + Ours(N=2) | 30.92 | 1.53× | 0.031 | 34.33 | 0.9271 | 13937 | 15964 |

## 4 Related Works

### 4.1 Training-free Acceleration of Diffusion Models.

Training-free acceleration methods aim to improve the inference efficiency of diffusion models without modifying their parameters or requiring additional training. These approaches typically exploit structural redundancies or adaptively skip computation steps during the sampling process Ma et al. (2024b); Yu et al. (2023); Zhan et al. (2024); Li et al. (2023a); Tian et al. (2025); Zhou et al. (2025). The reverse process of diffusion models inherently involves the step-by-step reconstruction of images or videos Ho et al. (2020). However, certain intermediate states, such as latent representations and attention maps, contain natural redundancy or sparsity, which provides the foundation for training-free acceleration strategies Ma et al. (2024b). These methods include: 1) Inference-skipping, where redundant steps with minimal change or intermediate activations with large zero regions are omitted to reduce unnecessary computations Ma et al. (2024b); Tian et al. (2025); Yu et al. (2023); Zhou et al. (2025). 2) Computation graph optimization, where inference engines (e.g., ONNX Runtime, TensorRT, TVM) restructure the execution graph to eliminate redundant operations and reduce memory access overhead while preserving mathematical equivalence. 3) Model quantization, where model weights and activations are quantized from high to low precision, reducing memory usage and accelerating computation without retraining Siddegowda et al. (2022); Wang et al. (2025); Zhao et al. (2025; 2024a); Shang et al. (2023).
In general, training-free approaches often require a careful trade-off between speed improvement, memory savings, and acceptable levels of accuracy degradation.

Figure: Figure 7: 4 cases for Animate-Light and SVD-XT
Refer to caption: iclr2026/plots/case.png

### 4.2 Cache in Diffusion Models.

Cache temporarily stores parts of main memory that are expected to be accessed again soon Smith (1982). Recently, a lot of work has applied the cache mechanism in diffusion models Ma et al. (2024b); Agarwal et al. (2024); Wimbauer et al. (2024); Ma et al. (2024a); Zhou et al. (2025); Liu et al. (2025). By carefully analyzing the model’s computation workflow, inference efficiency can be significantly improved. One effective strategy is to reuse intermediate or final representations generated during earlier timesteps of computation. This approach avoids redundant recalculations, which not only accelerates the overall inference process but also reduces computational resource consumption and latency.
Ma et al. (2024b); Wimbauer et al. (2024); Zhou et al. (2025) leverage the temporal redundancy in the reverse denoising process of diffusion models by caching and reusing features from specific layers, thereby avoiding redundant computations and accelerating the generation process. Agarwal et al. (2024) significantly reduces computational cost and latency by reusing intermediate noise states from the diffusion process, allowing new requests to skip a portion of the denoising steps and condition on cached intermediate states generated from similar previously processed text prompts.
FreeDoM leverages energy functions to provide guidance during the diffusion sampling process via energy gradients Yu et al. (2023).
EasyCache uses a dynamic threshold to determine when it is safe to reuse cached results Zhou et al. (2025).
PAB leverages the U-shaped variation pattern of attention outputs in DiT models during the diffusion process, and reuses the stable-phase attention features through a pyramid-style broadcasting strategy, thereby significantly reducing redundant computations Zhao et al. (2024b).
FORA reduces redundant computation by caching and reusing the intermediate features of the Attention and MLP layers at fixed intervals Selvaraju et al. (2024).
These advancements demonstrate the growing potential of cache-based strategies to improve the efficiency and scalability of diffusion model inference.

## 5 Conclusion

In this work, we proposed LightCache, a training-free framework that achieves low-memory and accelerated video generation without compromising output quality. By analyzing the inference process across encoding, denoising, and decoding stages, we introduced three complementary strategies: Asynchronous swapping, Feature chunk, and VAE slicing. Compared to baseline model and DeepCache, and FME, these stage-specific optimizations substantially reduce peak memory usage while keeping the additional time overhead well below the acceleration gains. In future work, we plan to explore combining LightCache with complementary training-free methods, as well as extending it to DiT models, longer video sequences and multi-modal generation tasks where memory efficiency and inference speed are critical.

## 6 Ethics statement

This work does not involve human subjects, personally identifiable data, or sensitive information. We believe it does not raise ethical concerns.

## 7 Reproducibility statement

We have made our code available in the anonymous project. Detailed hyperparameters, inference procedures, and data processing steps are provided in the code file. Random seeds are fixed for reproducibility, and all results have been verified across multiple independent runs.

## References

- Agarwal et al. (2024)
Shubham Agarwal, Subrata Mitra, Sarthak Chakraborty, Srikrishna Karanam, Koyel Mukherjee, and Shiv Kumar Saini.
Approximate caching for efficiently serving $\{$Text-to-Image$\}$ diffusion models.
In *21st USENIX Symposium on Networked Systems Design and Implementation (NSDI 24)*, pp. 1173–1189, 2024.
- Austin et al. (2021)
Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne Van Den Berg.
Structured denoising diffusion models in discrete state-spaces.
*Advances in neural information processing systems*, 34:17981–17993, 2021.
- Blattmann et al. (2023)
Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al.
Stable video diffusion: Scaling latent video diffusion models to large datasets.
*arXiv preprint arXiv:2311.15127*, 2023.
- Croitoru et al. (2023)
Florinel-Alin Croitoru, Vlad Hondru, Radu Tudor Ionescu, and Mubarak Shah.
Diffusion models in vision: A survey.
*IEEE Transactions on Pattern Analysis and Machine Intelligence*, 45(9):10850–10869, 2023.
- Guo et al. (2023)
Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai.
Animatediff: Animate your personalized text-to-image diffusion models without specific tuning.
*arXiv preprint arXiv:2307.04725*, 2023.
- Ho & Salimans (2022)
Jonathan Ho and Tim Salimans.
Classifier-free diffusion guidance.
*arXiv preprint arXiv:2207.12598*, 2022.
- Ho et al. (2020)
Jonathan Ho, Ajay Jain, and Pieter Abbeel.
Denoising diffusion probabilistic models.
*Advances in neural information processing systems*, 33:6840–6851, 2020.
- Hoogeboom et al. (2021)
Emiel Hoogeboom, Didrik Nielsen, Priyank Jaini, Patrick Forré, and Max Welling.
Argmax flows and multinomial diffusion: Learning categorical distributions.
*Advances in neural information processing systems*, 34:12454–12465, 2021.
- Kim et al. (2023)
Bo-Kyeong Kim, Hyoung-Kyu Song, Thibault Castells, and Shinkook Choi.
On architectural compression of text-to-image diffusion models.
2023.
- Li et al. (2023a)
Lijiang Li, Huixia Li, Xiawu Zheng, Jie Wu, Xuefeng Xiao, Rui Wang, Min Zheng, Xin Pan, Fei Chao, and Rongrong Ji.
Autodiffusion: Training-free optimization of time steps and architectures for automated diffusion model acceleration.
In *Proceedings of the IEEE/CVF International Conference on Computer Vision*, pp. 7105–7114, 2023a.
- Li et al. (2023b)
Senmao Li, Taihang Hu, Fahad Shahbaz Khan, Linxuan Li, Shiqi Yang, Yaxing Wang, Ming-Ming Cheng, and Jian Yang.
Faster diffusion: Rethinking the role of unet encoder in diffusion models.
*CoRR*, 2023b.
- Li et al. (2025)
Senmao Li, Lei Wang, Kai Wang, Tao Liu, Jiehang Xie, Joost van de Weijer, Fahad Shahbaz Khan, Shiqi Yang, Yaxing Wang, and Jian Yang.
One-way ticket: Time-independent unified encoder for distilling text-to-image diffusion models.
In *Proceedings of the Computer Vision and Pattern Recognition Conference*, pp. 23563–23574, 2025.
- Li et al. (2023c)
Yanyu Li, Huan Wang, Qing Jin, Ju Hu, Pavlo Chemerys, Yun Fu, Yanzhi Wang, Sergey Tulyakov, and Jian Ren.
Snapfusion: Text-to-image diffusion model on mobile devices within two seconds.
*Advances in Neural Information Processing Systems*, 36:20662–20678, 2023c.
- Liu et al. (2025)
Feng Liu, Shiwei Zhang, Xiaofeng Wang, Yujie Wei, Haonan Qiu, Yuzhong Zhao, Yingya Zhang, Qixiang Ye, and Fang Wan.
Timestep embedding tells: It’s time to cache for video diffusion model.
In *Proceedings of the Computer Vision and Pattern Recognition Conference*, pp. 7353–7363, 2025.
- Luo et al. (2023)
Simian Luo, Yiqin Tan, Suraj Patil, Daniel Gu, Patrick von Platen, Apolinário Passos, Longbo Huang, Jian Li, and Hang Zhao.
Lcm-lora: A universal stable-diffusion acceleration module.
*arXiv preprint arXiv:2311.05556*, 2023.
- Lyu et al. (2022)
Zhaoyang Lyu, Xudong Xu, Ceyuan Yang, Dahua Lin, and Bo Dai.
Accelerating diffusion models via early stop of the diffusion process.
*arXiv preprint arXiv:2205.12524*, 2022.
- Ma et al. (2024a)
Xinyin Ma, Gongfan Fang, Michael Bi Mi, and Xinchao Wang.
Learning-to-cache: Accelerating diffusion transformer via layer caching.
*Advances in Neural Information Processing Systems*, 37:133282–133304, 2024a.
- Ma et al. (2024b)
Xinyin Ma, Gongfan Fang, and Xinchao Wang.
Deepcache: Accelerating diffusion models for free.
In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pp. 15762–15772, 2024b.
- Peebles & Xie (2023)
William Peebles and Saining Xie.
Scalable diffusion models with transformers.
In *Proceedings of the IEEE/CVF international conference on computer vision*, pp. 4195–4205, 2023.
- Pope et al. (2023)
Reiner Pope, Sholto Douglas, Aakanksha Chowdhery, Jacob Devlin, James Bradbury, Jonathan Heek, Kefan Xiao, Shivani Agrawal, and Jeff Dean.
Efficiently scaling transformer inference.
*Proceedings of machine learning and systems*, 5:606–624, 2023.
- Rombach et al. (2022)
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer.
High-resolution image synthesis with latent diffusion models.
In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pp. 10684–10695, 2022.
- Ronneberger et al. (2015)
Olaf Ronneberger, Philipp Fischer, and Thomas Brox.
U-net: Convolutional networks for biomedical image segmentation.
In *International Conference on Medical image computing and computer-assisted intervention*, pp. 234–241. Springer, 2015.
- Selvaraju et al. (2024)
Pratheba Selvaraju, Tianyu Ding, Tianyi Chen, Ilya Zharkov, and Luming Liang.
Fora: Fast-forward caching in diffusion transformer acceleration.
*arXiv preprint arXiv:2407.01425*, 2024.
- Shang et al. (2023)
Yuzhang Shang, Zhihang Yuan, Bin Xie, Bingzhe Wu, and Yan Yan.
Post-training quantization on diffusion models.
In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pp. 1972–1981, 2023.
- Siddegowda et al. (2022)
Sangeetha Siddegowda, Marios Fournarakis, Markus Nagel, Tijmen Blankevoort, Chirag Patel, and Abhijit Khobare.
Neural network quantization with ai model efficiency toolkit (aimet).
*arXiv preprint arXiv:2201.08442*, 2022.
- Smith (1982)
Alan Jay Smith.
Cache memories.
*ACM Computing Surveys (CSUR)*, 14(3):473–530, 1982.
- Song et al. (2020)
Jiaming Song, Chenlin Meng, and Stefano Ermon.
Denoising diffusion implicit models.
*arXiv preprint arXiv:2010.02502*, 2020.
- Tian et al. (2025)
Ye Tian, Xin Xia, Yuxi Ren, Shanchuan Lin, Xing Wang, Xuefeng Xiao, Yunhai Tong, Ling Yang, and Bin Cui.
Training-free diffusion acceleration with bottleneck sampling.
*arXiv preprint arXiv:2503.18940*, 2025.
- von Platen et al. (2022)
Patrick von Platen, Suraj Patil, Anton Lozhkov, Pedro Cuenca, Nathan Lambert, Kashif Rasul, Mishig Davaadorj, Dhruv Nair, Sayak Paul, William Berman, Yiyi Xu, Steven Liu, and Thomas Wolf.
Diffusers: State-of-the-art diffusion models.
[https://github.com/huggingface/diffusers](https://github.com/huggingface/diffusers), 2022.
- Wang et al. (2025)
Juncheng Wang, Chao Xu, Cheng Yu, Lei Shang, Zhe Hu, Shujun Wang, and Liefeng Bo.
Synchronized video-to-audio generation via mel quantization-continuum decomposition.
In *Proceedings of the Computer Vision and Pattern Recognition Conference*, pp. 3111–3120, 2025.
- Wang et al. (2023)
Zhendong Wang, Yifan Jiang, Huangjie Zheng, Peihao Wang, Pengcheng He, Zhangyang Wang, Weizhu Chen, Mingyuan Zhou, et al.
Patch diffusion: Faster and more data-efficient training of diffusion models.
*Advances in neural information processing systems*, 36:72137–72154, 2023.
- Wimbauer et al. (2024)
Felix Wimbauer, Bichen Wu, Edgar Schoenfeld, Xiaoliang Dai, Ji Hou, Zijian He, Artsiom Sanakoyeu, Peizhao Zhang, Sam Tsai, Jonas Kohler, et al.
Cache me if you can: Accelerating diffusion models through block caching.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pp. 6211–6220, 2024.
- Xing et al. (2024)
Zhen Xing, Qijun Feng, Haoran Chen, Qi Dai, Han Hu, Hang Xu, Zuxuan Wu, and Yu-Gang Jiang.
A survey on video diffusion models.
*ACM Computing Surveys*, 57(2):1–42, 2024.
- Yang et al. (2023)
Ling Yang, Zhilong Zhang, Yang Song, Shenda Hong, Runsheng Xu, Yue Zhao, Wentao Zhang, Bin Cui, and Ming-Hsuan Yang.
Diffusion models: A comprehensive survey of methods and applications.
*ACM Computing Surveys*, 56(4):1–39, 2023.
- Yang et al. (2025)
Ling Yang, Ye Tian, Bowen Li, Xinchen Zhang, Ke Shen, Yunhai Tong, and Mengdi Wang.
Mmada: Multimodal large diffusion language models.
*arXiv preprint arXiv:2505.15809*, 2025.
- Yang et al. (2024)
Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al.
Cogvideox: Text-to-video diffusion models with an expert transformer.
*arXiv preprint arXiv:2408.06072*, 2024.
- Yin et al. (2024)
Tianwei Yin, Michaël Gharbi, Richard Zhang, Eli Shechtman, Fredo Durand, William T Freeman, and Taesung Park.
One-step diffusion with distribution matching distillation.
In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pp. 6613–6623, 2024.
- Yu et al. (2023)
Jiwen Yu, Yinhuai Wang, Chen Zhao, Bernard Ghanem, and Jian Zhang.
Freedom: Training-free energy-guided conditional diffusion model.
In *Proceedings of the IEEE/CVF International Conference on Computer Vision*, pp. 23174–23184, 2023.
- Zhan et al. (2024)
Zheng Zhan, Yushu Wu, Yifan Gong, Zichong Meng, Zhenglun Kong, Changdi Yang, Geng Yuan, Pu Zhao, Wei Niu, and Yanzhi Wang.
Fast and memory-efficient video diffusion using streamlined inference.
*arXiv preprint arXiv:2411.01171*, 2024.
- Zhao et al. (2025)
Maosen Zhao, Pengtao Chen, Chong Yu, Yan Wen, Xudong Tan, and Tao Chen.
Pioneering 4-bit fp quantization for diffusion models: Mixup-sign quantization and timestep-aware fine-tuning.
In *Proceedings of the Computer Vision and Pattern Recognition Conference*, pp. 18134–18143, 2025.
- Zhao et al. (2024a)
Tianchen Zhao, Xuefei Ning, Tongcheng Fang, Enshu Liu, Guyue Huang, Zinan Lin, Shengen Yan, Guohao Dai, and Yu Wang.
Mixdq: Memory-efficient few-step text-to-image diffusion models with metric-decoupled mixed precision quantization.
In *European Conference on Computer Vision*, pp. 285–302. Springer, 2024a.
- Zhao et al. (2024b)
Xuanlei Zhao, Xiaolong Jin, Kai Wang, and Yang You.
Real-time video generation with pyramid attention broadcast.
*arXiv preprint arXiv:2408.12588*, 2024b.
- Zhou et al. (2025)
Xin Zhou, Dingkang Liang, Kaijin Chen, Tianrui Feng, Xiwu Chen, Hongkai Lin, Yikang Ding, Feiyang Tan, Hengshuang Zhao, and Xiang Bai.
Less is enough: Training-free video diffusion acceleration via runtime-adaptive caching.
*arXiv preprint arXiv:2507.02860*, 2025.

## Appendix A Appendix

### A.1 LLM Usage Statement

Large language models (LLMs) were employed to assist in refining the clarity, grammar, and style of the manuscript. The use of LLMs was limited to language polishing and did not contribute to the generation of substantive ideas, analyses, or results. All scientific content, interpretations, and conclusions are the authors’ own.

## Peer Reviews

confidence: 5
contribution: 2
initial_rating: 2
presentation: 2
questions: How is the trade-off between memory reduction and generation quality quantitatively controlled in the proposed methods? Are there guidelines for choosing the chunking or swapping parameters?
soundness: 2
strengths: 1.The paper is easy to follow and conceptually clear.
2.The proposed methods are effective in reducing memory consumption.
summary: This paper presents LightCache, a training-free framework designed to improve memory efficiency during video diffusion model inference. The authors decompose the process into denoising and decoding stages and propose three corresponding strategies: feature chunking to reduce feature map size, asynchronous swapping to offload cached features to the CPU, and sliced VAE decoding to handle frames sequentially. Experiments across several video generation models demonstrate that these methods effectively lower peak GPU memory usage while maintaining generation quality.
weaknesses: 1.There is almost no novelty—chunking, swapping, and slicing are all standard memory optimization techniques already used in many diffusion frameworks.

2.The method shows no real advantage on multiple models (SVD, AnimateDiff, TextToVideo); speed-up at N=2 is minimal, and larger N values severely degrade generation quality.

3.Baseline comparison is too limited, focusing mainly on DeepCache, without strong evidence of improvement.

4.The approach largely repackages existing engineering tricks, trading memory for time rather than offering genuine acceleration.  

5.The ablation study is insufficient, with limited analysis of how each proposed component (chunking, swapping, slicing) quantitatively contributes to overall performance.

---

confidence: 3
contribution: 2
initial_rating: 6
presentation: 3
questions: - How well does the model generalize onto different structural backbones (not only U-Net)?
- In Table 2 we can see that larger steps 25+ impact the time of your method. Is there the possibility of performing a very high step generation to see the overhead from your method?
soundness: 3
strengths: The authors analyze the problem of increased memory consumption in cache‑based acceleration of video‑generation methods. They propose a training‑free approach specifically for diffusion‑based models, decomposing inference into three stages, encoding, denoising, and decoding, and explicitly addressing the memory overhead associated with the denoising and decoding phases. The paper presents well‑established, yet effective, techniques to mitigate these issues and demonstrates a noticeable improvement in performance. Overall, the work is well written, and evaluated.
summary: This paper explores training-free acceleration for video generation based on diffusion models by addressing memory surges during inference. The authors decompose inference into encoding, denoising, and decoding stages, identifying that cache-based acceleration often increases memory use. To mitigate this, they propose three stage-specific strategies — Asynchronous Cache Swapping, Feature Chunking, and Latent Slicing for Decoding, which reduce memory consumption while keeping the time overhead lower than the acceleration gain. Experiments show that the proposed approach achieves faster inference and lower memory usage with only minor quality degradation compared to the baseline.
weaknesses: The principal weakness lies in the level of novelty. Most of the contributions already appear in prior work and pertain mainly to implementation‑level details rather than to the fundamental underlying problem. Although the proposed improvements are effective, they contribute little new insight to solving the issue; instead, they address it at an application‑level.

---

confidence: 5
contribution: 2
initial_rating: 2
presentation: 2
questions: - There already exist more advanced cache-based acceleration methods for video generation, such as FasterCache [1], TeaCache [2], and PAB [3]. It would be valuable if the authors could include experimental results comparing their method with these approaches, in addition to DeepCache, which was originally designed for image synthesis with UNet architectures.

- The base models selected for evaluation appear relatively limited in scope. More recent and widely adopted architectures such as WAN2.1 and HunyuanVideo exhibit substantially higher memory demands. It would be informative to see how the proposed method performs on these more challenging and representative models.

[1] Lv Z, Si C, Song J, et al. FasterCache: Training-Free Video Diffusion Model Acceleration with High Quality.

[2] Liu F, Zhang S, Wang X, et al. Timestep Embedding Tells: It’s Time to Cache for Video Diffusion Models.

[3] Zhao X, Jin X, Wang K, et al. Real-time Video Generation with Pyramid Attention Broadcast.
soundness: 2
strengths: + The paper employs a series of engineering techniques to achieve cache-based acceleration while simultaneously reducing GPU memory consumption.
summary: This paper proposes LightCache, a training-free framework that reduces GPU memory during video diffusion inference by asynchronously offloading cached tensors, chunking feature maps, and slicing VAE decoding.
weaknesses: - The proposed techniques, such as feature chunking and asynchronous CPU offloading of cached tensors, are widely used engineering practices in existing approaches (e.g., PAB). As a result, the methodological novelty of the paper appears limited, and the contribution may not be strong enough to justify publication in its current form.

- The writing quality could be improved for clarity and precision. Several parts of the paper are difficult to follow. For example, the Abstract claims that cache-based acceleration increases memory consumption in both the denoising and decoding stages. While the denoising overhead is understandable, the paper does not provide sufficient analysis or justification for why cache-based methods would increase memory usage during decoding.

---

confidence: 4
contribution: 2
initial_rating: 4
presentation: 2
questions: 1. It is unclear what specific model the TextToVideo baseline refers to in the paper, as there is no corresponding citation or reference provided.
soundness: 2
strengths: The paper provides a systematic analysis of memory bottlenecks in diffusion-based video generation and proposes simple yet practical strategies to alleviate them without retraining, offering a useful engineering reference for future work on efficient inference.
summary: This paper presents LightCache, a training-free framework for accelerating diffusion-based video generation with improved memory efficiency. By introducing Asynchronous Cache Swapping, Feature Chunking, and VAE Slicing, the method mitigates memory surges in denoising and decoding stages, achieving faster inference and lower GPU usage than DeepCache with minimal quality loss.
weaknesses: 1. The writing quality needs improvement, as several parts are confusing. For example, the latter part of the Introduction does not clearly explain how the proposed slicing strategy works, but only describes why it is needed.

2. The discussion and citation of related work are insufficient. 

    a) In particular, for the choice of base models, many more recent and memory-intensive DiT-based architectures, such as HunyuanVideo and WAN 2.2, have emerged. It remains unclear whether the proposed method, which is primarily validated on U-Net–based models, can generalize effectively to these DiT architectures or to models with significantly larger parameter scales.

    b) Secondly, cache-based acceleration methods have been extensively explored in video generation. It is recommended that the authors adopt a more advanced method specifically designed for video diffusion, such as TeaCache, for experimental comparison. Moreover, the paper lacks adequate discussion of recent cache-based approaches in this domain.

3. Many of the techniques mentioned in the paper, such as asynchronous CPU offloading, have already been explored in prior engineering practices, and therefore the proposed method lacks sufficient novelty.

4.Typo: Line 201 — “cacheing” should be corrected to “caching.”