Splatfacto-W: A Nerfstudio Implementation of Gaussian Splatting for
  Unconstrained Photo Collections
Authors: name: Congrong Xu; name: Justin Kerr; name: Angjoo Kanazawa
Published: 2024-07-17 04:02:54

Abstract: Novel view synthesis from unconstrained in-the-wild image collections remains
a significant yet challenging task due to photometric variations and transient
occluders that complicate accurate scene reconstruction. Previous methods have
approached these issues by integrating per-image appearance features embeddings
in Neural Radiance Fields (NeRFs). Although 3D Gaussian Splatting (3DGS) offers
faster training and real-time rendering, adapting it for unconstrained image
collections is non-trivial due to the substantially different architecture. In
this paper, we introduce Splatfacto-W, an approach that integrates per-Gaussian
neural color features and per-image appearance embeddings into the
rasterization process, along with a spherical harmonics-based background model
to represent varying photometric appearances and better depict backgrounds. Our
key contributions include latent appearance modeling, efficient transient
object handling, and precise background modeling. Splatfacto-W delivers
high-quality, real-time novel view synthesis with improved scene consistency in
in-the-wild scenarios. Our method improves the Peak Signal-to-Noise Ratio
(PSNR) by an average of 5.3 dB compared to 3DGS, enhances training speed by 150
times compared to NeRF-based methods, and achieves a similar rendering speed to
3DGS. Additional video results and code integrated into Nerfstudio are
available at https://kevinxu02.github.io/splatfactow/.