  In this paper, a robust RGB-D SLAM system is proposed to utilize the
structural information in indoor scenes, allowing for accurate tracking and
efficient dense mapping on a CPU. Prior works have used the Manhattan World
(MW) assumption to estimate low-drift camera pose, in turn limiting the
applications of such systems. This paper, in contrast, proposes a novel
approach delivering robust tracking in MW and non-MW environments. We check
orthogonal relations between planes to directly detect Manhattan Frames,
modeling the scene as a Mixture of Manhattan Frames. For MW scenes, we decouple
pose estimation and provide a novel drift-free rotation estimation based on
Manhattan Frame observations. For translation estimation in MW scenes and full
camera pose estimation in non-MW scenes, we make use of point, line and plane
features for robust tracking in challenging scenes. %mapping Additionally, by
exploiting plane features detected in each frame, we also propose an efficient
surfel-based dense mapping strategy, which divides each image into planar and
non-planar regions. Planar surfels are initialized directly from sparse planes
in our map while non-planar surfels are built by extracting superpixels. We
evaluate our method on public benchmarks for pose estimation, drift and
reconstruction accuracy, achieving superior performance compared to other
state-of-the-art methods. We will open-source our code in the future.
