ThermalNeRF: Thermal Radiance Fields
Authors: name: Yvette Y. Lin; name: Xin-Yi Pan; name: Sara Fridovich-Keil; name: Gordon Wetzstein
Published: 2024-07-22 02:51:29

Abstract: Thermal imaging has a variety of applications, from agricultural monitoring
to building inspection to imaging under poor visibility, such as in low light,
fog, and rain. However, reconstructing thermal scenes in 3D presents several
challenges due to the comparatively lower resolution and limited features
present in long-wave infrared (LWIR) images. To overcome these challenges, we
propose a unified framework for scene reconstruction from a set of LWIR and RGB
images, using a multispectral radiance field to represent a scene viewed by
both visible and infrared cameras, thus leveraging information across both
spectra. We calibrate the RGB and infrared cameras with respect to each other,
as a preprocessing step using a simple calibration target. We demonstrate our
method on real-world sets of RGB and LWIR photographs captured from a handheld
thermal camera, showing the effectiveness of our method at scene representation
across the visible and infrared spectra. We show that our method is capable of
thermal super-resolution, as well as visually removing obstacles to reveal
objects that are occluded in either the RGB or thermal channels. Please see
https://yvette256.github.io/thermalnerf for video results as well as our code
and dataset release.