AuraFusion360: Augmented Unseen Region Alignment for Reference-based
  360° Unbounded Scene Inpainting
Authors: name: Chung-Ho Wu; name: Yang-Jung Chen; name: Ying-Huan Chen; name: Jie-Ying Lee; name: Bo-Hsu Ke; name: Chun-Wei Tuan Mu; name: Yi-Chuan Huang; name: Chin-Yang Lin; name: Min-Hung Chen; name: Yen-Yu Lin; name: Yu-Lun Liu
Published: 2025-02-07 18:59:55

Abstract: Three-dimensional scene inpainting is crucial for applications from virtual
reality to architectural visualization, yet existing methods struggle with view
consistency and geometric accuracy in 360{\deg} unbounded scenes. We present
AuraFusion360, a novel reference-based method that enables high-quality object
removal and hole filling in 3D scenes represented by Gaussian Splatting. Our
approach introduces (1) depth-aware unseen mask generation for accurate
occlusion identification, (2) Adaptive Guided Depth Diffusion, a zero-shot
method for accurate initial point placement without requiring additional
training, and (3) SDEdit-based detail enhancement for multi-view coherence. We
also introduce 360-USID, the first comprehensive dataset for 360{\deg}
unbounded scene inpainting with ground truth. Extensive experiments demonstrate
that AuraFusion360 significantly outperforms existing methods, achieving
superior perceptual quality while maintaining geometric accuracy across
dramatic viewpoint changes. See our project page for video results and the
dataset at https://kkennethwu.github.io/aurafusion360/.