# A Novel Visual SLAM Based on Multiple Deep Neural Networks
## 1 Introduction  
In the last two decades, simultaneous localization and mapping (SLAM) has become a hot area of research in robotics and computer vision. It is a fundamental module for many applications such as mobile robotics, Micro Aerial Vehicles (MAVs), autonomous driving and augmented reality because they all require real-time localization [1]. SLAM techniques use the information acquired by the sensor to build a map of an unknown environment and localize the sensor in the map. Visual SLAM has been widely studied because of the camera's low price and its ability to acquire rich information about the environment [2; 3; 4; 5; 6; 7; 8].  
Most of the currently used Visual SLAM systems rely on hand-crafted visual features [9; 10; 11; 12; 13; 14]. For example, ORB-SLAM [9; 10] performs feature matching and loop closure detection by extracting ORB [15] features in each image frame, and then estimates the camera pose and constructs a map of the environment. However, hand-crafted features may fail to provide consistent feature detection and accurate matching results in complex environments.  
With the development of deep learning, feature extraction based on deep neural network (DNN) has replaced hand-crafted features as the mainstream of relevant research [16; 17; 18; 19; 20]. DNN-based features have also been widely used in the design and research of visual SLAM systems. For example, [PERSON] et al. [21] designed the GCN-SLAM based on Geometric Correspondence Network [22], [PERSON] et al. [23] proposed the DXSLAM based on HF-Net [24], and [PERSON] et al. [25] used a visual SLAM for autonomous driving based on SuperPoint [26]. Although these works have improved the performance of visual SLAMsystems by using DNN-based features instead of hand-crafted features, these systems may not be able to track feature points continuously well. This is because they employ the same feature matching method used for hand-crafted features. Such a design severely limits the performance of the system.  
To solve the above problems, this paper proposes a visual SLAM system that uses a deep feature extractor, a deep feature matcher, and a lightweight deep loop closure detector. Figures 1 and 2 show the results of some runs of the proposed visual SLAM on the KITTI odometry dataset. The main contributions are as follows:  
1. A novel Visual Odometry was proposed by reconstructing SuperPoint [26] and SuperGlue [27] as _SG_-Detector and _SG_-Matcher and combining them with a fusion estimation method based on last and reference frame for a camera pose.
2. A deep LoopClosing module was designed by combining Seq-CALC [28] and _SG_-Matcher.
3. Compared with many existing SLAM systems, the proposed SLAM system reduces estimation error by at least 0.1 m on the KITTI odometry dataset, and it can run in real time at a speed of 0.08 s per frame.  
The rest of the paper is organized as follows. In Section 2, some related work is presented. In Section 3, the system architecture of our visual SLAM and the design principles of each module are explained in detail. In Section 4, the experiments used to test our system are described and the experimental results are analyzed in detail.