NVComposer: Boosting Generative Novel View Synthesis with Multiple
  Sparse and Unposed Images
Authors: name: Lingen Li; name: Zhaoyang Zhang; name: Yaowei Li; name: Jiale Xu; name: Xiaoyu Li; name: Wenbo Hu; name: Weihao Cheng; name: Jinwei Gu; name: Tianfan Xue; name: Ying Shan
Published: 2024-12-04 17:58:03

Abstract: Recent advancements in generative models have significantly improved novel
view synthesis (NVS) from multi-view data. However, existing methods depend on
external multi-view alignment processes, such as explicit pose estimation or
pre-reconstruction, which limits their flexibility and accessibility,
especially when alignment is unstable due to insufficient overlap or occlusions
between views. In this paper, we propose NVComposer, a novel approach that
eliminates the need for explicit external alignment. NVComposer enables the
generative model to implicitly infer spatial and geometric relationships
between multiple conditional views by introducing two key components: 1) an
image-pose dual-stream diffusion model that simultaneously generates target
novel views and condition camera poses, and 2) a geometry-aware feature
alignment module that distills geometric priors from dense stereo models during
training. Extensive experiments demonstrate that NVComposer achieves
state-of-the-art performance in generative multi-view NVS tasks, removing the
reliance on external alignment and thus improving model accessibility. Our
approach shows substantial improvements in synthesis quality as the number of
unposed input views increases, highlighting its potential for more flexible and
accessible generative NVS systems.