Zero4D: Training-Free 4D Video Generation From Single Video Using
  Off-the-Shelf Video Diffusion Model
Authors: name: Jangho Park; name: Taesung Kwon; name: Jong Chul Ye
Published: 2025-03-28 17:14:48

Abstract: Recently, multi-view or 4D video generation has emerged as a significant
research topic. Nonetheless, recent approaches to 4D generation still struggle
with fundamental limitations, as they primarily rely on harnessing multiple
video diffusion models with additional training or compute-intensive training
of a full 4D diffusion model with limited real-world 4D data and large
computational costs. To address these challenges, here we propose the first
training-free 4D video generation method that leverages the off-the-shelf video
diffusion models to generate multi-view videos from a single input video. Our
approach consists of two key steps: (1) By designating the edge frames in the
spatio-temporal sampling grid as key frames, we first synthesize them using a
video diffusion model, leveraging a depth-based warping technique for guidance.
This approach ensures structural consistency across the generated frames,
preserving spatial and temporal coherence. (2) We then interpolate the
remaining frames using a video diffusion model, constructing a fully populated
and temporally coherent sampling grid while preserving spatial and temporal
consistency. Through this approach, we extend a single video into a multi-view
video along novel camera trajectories while maintaining spatio-temporal
consistency. Our method is training-free and fully utilizes an off-the-shelf
video diffusion model, offering a practical and effective solution for
multi-view video generation.