  Depth completion aims to recover a dense depth map from a sparse depth map
with the corresponding color image as input. Recent approaches mainly formulate
the depth completion as a one-stage end-to-end learning task, which outputs
dense depth maps directly. However, the feature extraction and supervision in
one-stage frameworks are insufficient, limiting the performance of these
approaches. To address this problem, we propose a novel end-to-end residual
learning framework, which formulates the depth completion as a two-stage
learning task, i.e., a sparse-to-coarse stage and a coarse-to-fine stage.
First, a coarse dense depth map is obtained by a simple CNN framework. Then, a
refined depth map is further obtained using a residual learning strategy in the
coarse-to-fine stage with coarse depth map and color image as input. Specially,
in the coarse-to-fine stage, a channel shuffle extraction operation is utilized
to extract more representative features from color image and coarse depth map,
and an energy based fusion operation is exploited to effectively fuse these
features obtained by channel shuffle operation, thus leading to more accurate
and refined depth maps. We achieve SoTA performance in RMSE on KITTI benchmark.
Extensive experiments on other datasets future demonstrate the superiority of
our approach over current state-of-the-art depth completion approaches.
