Neural Relighting with Subsurface Scattering by Learning the Radiance
  Transfer Gradient
Authors: name: Shizhan Zhu; name: Shunsuke Saito; name: Aljaz Bozic; name: Carlos Aliaga; name: Trevor Darrell; name: Christop Lassner
Published: 2023-06-15 17:56:04

Abstract: Reconstructing and relighting objects and scenes under varying lighting
conditions is challenging: existing neural rendering methods often cannot
handle the complex interactions between materials and light. Incorporating
pre-computed radiance transfer techniques enables global illumination, but
still struggles with materials with subsurface scattering effects. We propose a
novel framework for learning the radiance transfer field via volume rendering
and utilizing various appearance cues to refine geometry end-to-end. This
framework extends relighting and reconstruction capabilities to handle a wider
range of materials in a data-driven fashion. The resulting models produce
plausible rendering results in existing and novel conditions. We will release
our code and a novel light stage dataset of objects with subsurface scattering
effects publicly available.