Training-Free Efficient Video Generation via Dynamic Token Carving
Authors: name: Yuechen Zhang; name: Jinbo Xing; name: Bin Xia; name: Shaoteng Liu; name: Bohao Peng; name: Xin Tao; name: Pengfei Wan; name: Eric Lo; name: Jiaya Jia
Published: 2025-05-22 16:21:32

Abstract: Despite the remarkable generation quality of video Diffusion Transformer
(DiT) models, their practical deployment is severely hindered by extensive
computational requirements. This inefficiency stems from two key challenges:
the quadratic complexity of self-attention with respect to token length and the
multi-step nature of diffusion models. To address these limitations, we present
Jenga, a novel inference pipeline that combines dynamic attention carving with
progressive resolution generation. Our approach leverages two key insights: (1)
early denoising steps do not require high-resolution latents, and (2) later
steps do not require dense attention. Jenga introduces a block-wise attention
mechanism that dynamically selects relevant token interactions using 3D
space-filling curves, alongside a progressive resolution strategy that
gradually increases latent resolution during generation. Experimental results
demonstrate that Jenga achieves substantial speedups across multiple
state-of-the-art video diffusion models while maintaining comparable generation
quality (8.83times speedup with 0.01\% performance drop on VBench). As a
plug-and-play solution, Jenga enables practical, high-quality video generation
on modern hardware by reducing inference time from minutes to seconds --
without requiring model retraining. Code:
https://github.com/dvlab-research/Jenga