Gaussian Frosting: Editable Complex Radiance Fields with Real-Time
  Rendering
Authors: name: Antoine Guédon; name: Vincent Lepetit
Published: 2024-03-21 16:53:03

Abstract: We propose Gaussian Frosting, a novel mesh-based representation for
high-quality rendering and editing of complex 3D effects in real-time. Our
approach builds on the recent 3D Gaussian Splatting framework, which optimizes
a set of 3D Gaussians to approximate a radiance field from images. We propose
first extracting a base mesh from Gaussians during optimization, then building
and refining an adaptive layer of Gaussians with a variable thickness around
the mesh to better capture the fine details and volumetric effects near the
surface, such as hair or grass. We call this layer Gaussian Frosting, as it
resembles a coating of frosting on a cake. The fuzzier the material, the
thicker the frosting. We also introduce a parameterization of the Gaussians to
enforce them to stay inside the frosting layer and automatically adjust their
parameters when deforming, rescaling, editing or animating the mesh. Our
representation allows for efficient rendering using Gaussian splatting, as well
as editing and animation by modifying the base mesh. We demonstrate the
effectiveness of our method on various synthetic and real scenes, and show that
it outperforms existing surface-based approaches. We will release our code and
a web-based viewer as additional contributions. Our project page is the
following: https://anttwo.github.io/frosting/