ICML 2025

Ca2-VDM: Efficient Autoregressive Video Diffusion Model
with Causal Generation and Cache Sharing
Kaifeng Gao * 1 2 Jiaxin Shi * 3 Hanwang Zhang 4 Chunping Wang 5 Jun Xiao 1 Long Chen 6
Abstract
With the advance of diffusion models, today’s
video generation has achieved impressive qual-
ity. To extend the generation length and facilitate
real-world applications, a majority of video dif-
fusion models (VDMs) generate videos in an au-
toregressive manner, i.e., generating subsequent
clips conditioned on the last frame(s) of the previ-
ous clip. However, existing autoregressive VDMs
are highly inefficient and redundant: The model
must re-compute all the conditional frames that
are overlapped between adjacent clips. This issue
is exacerbated when the conditional frames are
extended autoregressively to provide the model
with long-term context. In such cases, the compu-
tational demands increase significantly (i.e., with
a quadratic complexity w.r.t. the autoregression
step). In this paper, we propose Ca2-VDM, an
efficient autoregressive VDM with Causal gen-
eration and Cache sharing. For causal gener-
ation, it introduces unidirectional feature com-
putation, which ensures that the cache of con-
ditional frames can be precomputed in previous
autoregression steps and reused in every subse-
quent step, eliminating redundant computations.
For cache sharing, it shares the cache across
all denoising steps to avoid the huge cache stor-
age cost. Extensive experiments demonstrated
that our Ca2-VDM achieves state-of-the-art quan-
titative and qualitative video generation results
and significantly improves the generation speed.
Code is available: https://github.com/
Dawn-LX/CausalCache-VDM
*Equal contribution
1Zhejiang University, Hangzhou, China
2Manycore Tech Inc., Hangzhou, China 3Xmax.AI Ltd., Beijing,
China 4Nanyang Technological University, Singapore 5Finvolution
Group, Shanghai, China 6The Hong Kong University of Science
and Technology, Hong Kong, China. Correspondence to: ...