HybridNeRF: Efficient Neural Rendering via Adaptive Volumetric Surfaces
Authors: name: Haithem Turki; name: Vasu Agrawal; name: Samuel Rota Bulò; name: Lorenzo Porzi; name: Peter Kontschieder; name: Deva Ramanan; name: Michael Zollhöfer; name: Christian Richardt
Published: 2023-12-05 22:04:49

Abstract: Neural radiance fields provide state-of-the-art view synthesis quality but
tend to be slow to render. One reason is that they make use of volume
rendering, thus requiring many samples (and model queries) per ray at render
time. Although this representation is flexible and easy to optimize, most
real-world objects can be modeled more efficiently with surfaces instead of
volumes, requiring far fewer samples per ray. This observation has spurred
considerable progress in surface representations such as signed distance
functions, but these may struggle to model semi-opaque and thin structures. We
propose a method, HybridNeRF, that leverages the strengths of both
representations by rendering most objects as surfaces while modeling the
(typically) small fraction of challenging regions volumetrically. We evaluate
HybridNeRF against the challenging Eyeful Tower dataset along with other
commonly used view synthesis datasets. When comparing to state-of-the-art
baselines, including recent rasterization-based approaches, we improve error
rates by 15-30% while achieving real-time framerates (at least 36 FPS) for
virtual-reality resolutions (2Kx2K).