Vivid4D: Improving 4D Reconstruction from Monocular Video by Video
  Inpainting
Authors: name: Jiaxin Huang, name: Sheng Miao, name: BangBnag Yang, name: Yuewen Ma, name: Yiyi Liao

Reconstructing 4D dynamic scenes from casually captured monocular videos is
valuable but highly challenging, as each timestamp is observed from a single
viewpoint. We introduce Vivid4D, a novel approach that enhances 4D monocular
video synthesis by augmenting observation views - synthesizing multi-view
videos from a monocular input. Unlike existing methods that either solely
leverage geometric priors for supervision or use generative priors while
overlooking geometry, we integrate both. This reformulates view augmentation as
a video inpainting task, where observed views are warped into new viewpoints
based on monocular depth priors. To achieve this, we train a video inpainting
model on unposed web videos with synthetically generated masks that mimic
warping occlusions, ensuring spatially and temporally consistent completion of
missing regions. To further mitigate inaccuracies in monocular depth priors, we
introduce an iterative view augmentation strategy and a robust reconstruction
loss. Experiments demonstrate that our method effectively improves monocular 4D
scene reconstruction and completion.