WildGaussians: 3D Gaussian Splatting in the Wild
Authors: name: Jonas Kulhanek; name: Songyou Peng; name: Zuzana Kukelova; name: Marc Pollefeys; name: Torsten Sattler
Published: 2024-07-11 12:41:32

Abstract: While the field of 3D scene reconstruction is dominated by NeRFs due to their
photorealistic quality, 3D Gaussian Splatting (3DGS) has recently emerged,
offering similar quality with real-time rendering speeds. However, both methods
primarily excel with well-controlled 3D scenes, while in-the-wild data -
characterized by occlusions, dynamic objects, and varying illumination -
remains challenging. NeRFs can adapt to such conditions easily through
per-image embedding vectors, but 3DGS struggles due to its explicit
representation and lack of shared parameters. To address this, we introduce
WildGaussians, a novel approach to handle occlusions and appearance changes
with 3DGS. By leveraging robust DINO features and integrating an appearance
modeling module within 3DGS, our method achieves state-of-the-art results. We
demonstrate that WildGaussians matches the real-time rendering speed of 3DGS
while surpassing both 3DGS and NeRF baselines in handling in-the-wild data, all
within a simple architectural framework.