GSTAR: Gaussian Surface Tracking and Reconstruction
Authors: name: Chengwei Zheng; name: Lixin Xue; name: Juan Zarate; name: Jie Song
Published: 2025-01-17 16:26:24

Abstract: 3D Gaussian Splatting techniques have enabled efficient photo-realistic
rendering of static scenes. Recent works have extended these approaches to
support surface reconstruction and tracking. However, tracking dynamic surfaces
with 3D Gaussians remains challenging due to complex topology changes, such as
surfaces appearing, disappearing, or splitting. To address these challenges, we
propose GSTAR, a novel method that achieves photo-realistic rendering, accurate
surface reconstruction, and reliable 3D tracking for general dynamic scenes
with changing topology. Given multi-view captures as input, GSTAR binds
Gaussians to mesh faces to represent dynamic objects. For surfaces with
consistent topology, GSTAR maintains the mesh topology and tracks the meshes
using Gaussians. In regions where topology changes, GSTAR adaptively unbinds
Gaussians from the mesh, enabling accurate registration and the generation of
new surfaces based on these optimized Gaussians. Additionally, we introduce a
surface-based scene flow method that provides robust initialization for
tracking between frames. Experiments demonstrate that our method effectively
tracks and reconstructs dynamic surfaces, enabling a range of applications. Our
project page with the code release is available at
https://eth-ait.github.io/GSTAR/.