  This paper introduces SD-6DoF-ICLK, a learning-based Inverse Compositional
Lucas-Kanade (ICLK) pipeline that uses sparse depth information to optimize the
relative pose that best aligns two images on SE(3). To compute this six
Degrees-of-Freedom (DoF) relative transformation, the proposed formulation
requires only sparse depth information in one of the images, which is often the
only available depth source in visual-inertial odometry or Simultaneous
Localization and Mapping (SLAM) pipelines. In an optional subsequent step, the
framework further refines feature locations and the relative pose using
individual feature alignment and bundle adjustment for pose and structure
re-alignment. The resulting sparse point correspondences with subpixel-accuracy
and refined relative pose can be used for depth map generation, or the image
alignment module can be embedded in an odometry or mapping framework.
Experiments with rendered imagery show that the forward SD-6DoF-ICLK runs at
145 ms per image pair with a resolution of 752 x 480 pixels each, and vastly
outperforms the classical, sparse 6DoF-ICLK algorithm, making it the ideal
framework for robust image alignment under severe conditions.
