EPiC: Efficient Video Camera Control Learning with Precise Anchor-Video
  Guidance
Authors: name: Zun Wang, name: Jaemin Cho, name: Jialu Li, name: Han Lin, name: Jaehong Yoon, name: Yue Zhang, name: Mohit Bansal

Recent approaches on 3D camera control in video diffusion models (VDMs) often
create anchor videos to guide diffusion models as a structured prior by
rendering from estimated point clouds following annotated camera trajectories.
However, errors inherent in point cloud estimation often lead to inaccurate
anchor videos. Moreover, the requirement for extensive camera trajectory
annotations further increases resource demands. To address these limitations,
we introduce EPiC, an efficient and precise camera control learning framework
that automatically constructs high-quality anchor videos without expensive
camera trajectory annotations. Concretely, we create highly precise anchor
videos for training by masking source videos based on first-frame visibility.
This approach ensures high alignment, eliminates the need for camera trajectory
annotations, and thus can be readily applied to any in-the-wild video to
generate image-to-video (I2V) training pairs. Furthermore, we introduce
Anchor-ControlNet, a lightweight conditioning module that integrates anchor
video guidance in visible regions to pretrained VDMs, with less than 1% of
backbone model parameters. By combining the proposed anchor video data and
ControlNet module, EPiC achieves efficient training with substantially fewer
parameters, training steps, and less data, without requiring modifications to
the diffusion model backbone typically needed to mitigate rendering
misalignments. Although being trained on masking-based anchor videos, our
method generalizes robustly to anchor videos made with point clouds during
inference, enabling precise 3D-informed camera control. EPiC achieves SOTA
performance on RealEstate10K and MiraData for I2V camera control task,
demonstrating precise and robust camera control ability both quantitatively and
qualitatively. Notably, EPiC also exhibits strong zero-shot generalization to
video-to-video scenarios.