  We propose a framework for tightly-coupled lidar-visual-inertial odometry via
smoothing and mapping, LVI-SAM, that achieves real-time state estimation and
map-building with high accuracy and robustness. LVI-SAM is built atop a factor
graph and is composed of two sub-systems: a visual-inertial system (VIS) and a
lidar-inertial system (LIS). The two sub-systems are designed in a
tightly-coupled manner, in which the VIS leverages LIS estimation to facilitate
initialization. The accuracy of the VIS is improved by extracting depth
information for visual features using lidar measurements. In turn, the LIS
utilizes VIS estimation for initial guesses to support scan-matching. Loop
closures are first identified by the VIS and further refined by the LIS.
LVI-SAM can also function when one of the two sub-systems fails, which
increases its robustness in both texture-less and feature-less environments.
LVI-SAM is extensively evaluated on datasets gathered from several platforms
over a variety of scales and environments. Our implementation is available at
https://git.io/lvi-sam
