GS^3: Efficient Relighting with Triple Gaussian Splatting
Authors: name: Zoubin Bi; name: Yixin Zeng; name: Chong Zeng; name: Fan Pei; name: Xiang Feng; name: Kun Zhou; name: Hongzhi Wu
Published: 2024-10-15 09:11:30

Abstract: We present a spatial and angular Gaussian based representation and a triple
splatting process, for real-time, high-quality novel lighting-and-view
synthesis from multi-view point-lit input images. To describe complex
appearance, we employ a Lambertian plus a mixture of angular Gaussians as an
effective reflectance function for each spatial Gaussian. To generate
self-shadow, we splat all spatial Gaussians towards the light source to obtain
shadow values, which are further refined by a small multi-layer perceptron. To
compensate for other effects like global illumination, another network is
trained to compute and add a per-spatial-Gaussian RGB tuple. The effectiveness
of our representation is demonstrated on 30 samples with a wide variation in
geometry (from solid to fluffy) and appearance (from translucent to
anisotropic), as well as using different forms of input data, including
rendered images of synthetic/reconstructed objects, photographs captured with a
handheld camera and a flash, or from a professional lightstage. We achieve a
training time of 40-70 minutes and a rendering speed of 90 fps on a single
commodity GPU. Our results compare favorably with state-of-the-art techniques
in terms of quality/performance. Our code and data are publicly available at
https://GSrelight.github.io/.