FasterCache: Training-Free Video Diffusion Model Acceleration with High
  Quality
Authors: name: Zhengyao Lv; name: Chenyang Si; name: Junhao Song; name: Zhenyu Yang; name: Yu Qiao; name: Ziwei Liu; name: Kwan-Yee K. Wong
Published: 2024-10-25 07:24:38

Abstract: In this paper, we present \textit{FasterCache}, a novel
training-free strategy designed to accelerate the inference of video diffusion
models with high-quality generation. By analyzing existing cache-based methods,
we observe that directly reusing adjacent-step features degrades video
quality due to the loss of subtle variations. We further perform a pioneering
investigation of the acceleration potential of classifier-free guidance (CFG)
and reveal significant redundancy between conditional and unconditional
features within the same timestep. Capitalizing on these observations, we
introduce FasterCache to substantially accelerate diffusion-based video
generation. Our key contributions include a dynamic feature reuse strategy that
preserves both feature distinction and temporal continuity, and CFG-Cache which
optimizes the reuse of conditional and unconditional outputs to further enhance
inference speed without compromising video quality. We empirically evaluate
FasterCache on recent video diffusion models. Experimental results show that
FasterCache can significantly accelerate video generation (\eg 1.67times
speedup on Vchitect-2.0) while keeping video quality comparable to the
baseline, and consistently outperform existing methods in both inference speed
and video quality.