VR-NeRF: High-Fidelity Virtualized Walkable Spaces
Authors: name: Linning Xu; name: Vasu Agrawal; name: William Laney; name: Tony Garcia; name: Aayush Bansal; name: Changil Kim; name: Samuel Rota Bulò; name: Lorenzo Porzi; name: Peter Kontschieder; name: Aljaž Božič; name: Dahua Lin; name: Michael Zollhöfer; name: Christian Richardt
Published: 2023-11-05 02:03:14

Abstract: We present an end-to-end system for the high-fidelity capture, model
reconstruction, and real-time rendering of walkable spaces in virtual reality
using neural radiance fields. To this end, we designed and built a custom
multi-camera rig to densely capture walkable spaces in high fidelity and with
multi-view high dynamic range images in unprecedented quality and density. We
extend instant neural graphics primitives with a novel perceptual color space
for learning accurate HDR appearance, and an efficient mip-mapping mechanism
for level-of-detail rendering with anti-aliasing, while carefully optimizing
the trade-off between quality and speed. Our multi-GPU renderer enables
high-fidelity volume rendering of our neural radiance field model at the full
VR resolution of dual 2Ktimes2K at 36 Hz on our custom demo machine. We
demonstrate the quality of our results on our challenging high-fidelity
datasets, and compare our method and datasets to existing baselines. We release
our dataset on our project website.