Subsurface Scattering for 3D Gaussian Splatting
Authors: name: Jan-Niklas Dihlmann; name: Arjun Majumdar; name: Andreas Engelhardt; name: Raphael Braun; name: Hendrik P. A. Lensch
Published: 2024-08-22 10:34:01

Abstract: 3D reconstruction and relighting of objects made from scattering materials
present a significant challenge due to the complex light transport beneath the
surface. 3D Gaussian Splatting introduced high-quality novel view synthesis at
real-time speeds. While 3D Gaussians efficiently approximate an object's
surface, they fail to capture the volumetric properties of subsurface
scattering. We propose a framework for optimizing an object's shape together
with the radiance transfer field given multi-view OLAT (one light at a time)
data. Our method decomposes the scene into an explicit surface represented as
3D Gaussians, with a spatially varying BRDF, and an implicit volumetric
representation of the scattering component. A learned incident light field
accounts for shadowing. We optimize all parameters jointly via ray-traced
differentiable rendering. Our approach enables material editing, relighting and
novel view synthesis at interactive rates. We show successful application on
synthetic data and introduce a newly acquired multi-view multi-light dataset of
objects in a light-stage setup. Compared to previous work we achieve comparable
or better results at a fraction of optimization and rendering time while
enabling detailed control over material attributes. Project page
https://sss.jdihlmann.com/