Mirror-NeRF: Learning Neural Radiance Fields for Mirrors with
  Whitted-Style Ray Tracing
Authors: name: Junyi Zeng; name: Chong Bao; name: Rui Chen; name: Zilong Dong; name: Guofeng Zhang; name: Hujun Bao; name: Zhaopeng Cui
Published: 2023-08-07 03:48:07

Abstract: Recently, Neural Radiance Fields (NeRF) has exhibited significant success in
novel view synthesis, surface reconstruction, etc. However, since no physical
reflection is considered in its rendering pipeline, NeRF mistakes the
reflection in the mirror as a separate virtual scene, leading to the inaccurate
reconstruction of the mirror and multi-view inconsistent reflections in the
mirror. In this paper, we present a novel neural rendering framework, named
Mirror-NeRF, which is able to learn accurate geometry and reflection of the
mirror and support various scene manipulation applications with mirrors, such
as adding new objects or mirrors into the scene and synthesizing the
reflections of these new objects in mirrors, controlling mirror roughness, etc.
To achieve this goal, we propose a unified radiance field by introducing the
reflection probability and tracing rays following the light transport model of
Whitted Ray Tracing, and also develop several techniques to facilitate the
learning process. Experiments and comparisons on both synthetic and real
datasets demonstrate the superiority of our method. The code and supplementary
material are available on the project webpage:
https://zju3dv.github.io/Mirror-NeRF/.