# DK-SLAM: Monocular Visual SLAM with Deep Keypoint Learning, Tracking, and Loop Closing
## 1 Introduction  
Visual localization and mapping are fundamental components in autonomous systems for motion estimation and environmental perception. These capabilities have diverse applications, spanning from self-driving vehicles and unmanned aerial vehicles (UAVs) to mobile robots and immersive wearable technologies such as virtual reality (VR) and augmented reality (AR) devices. The quest for robust localization and mapping under varying environmental conditions has increasingly drawn the research community's attention. Visual Simultaneous Localization and Mapping (SLAM) systems generally comprise two main components: front-end perception and back-end optimization. They are typically divided into two categories: direct methods and feature-based methods. Directmethods, such as DSO [1], SVO [2], and H-SLAM [3], process the pixel intensities in an image to optimize a photometric loss at the pixel level, directly using image data for map construction and pose estimation. While these methods can be highly accurate in specific scenarios, they often demand substantial computational resources and can become unstable in dynamic environments. One the other hand, feature-based methods, exemplified by the ORB-SLAM series [4; 5], focus on identifying and utilizing key image features, i.e., keypoints. These methods extract keypoints with distinctive properties, compute descriptors, and perform feature matching, outlier removal, and relative motion tracking. Their back-end processes include loop closure detection and global optimization, crucial for refining and maintaining the system's state estimates over time. Feature-based approaches typically excel in computational efficiency and robustness across diverse scenarios. However, they may struggle in environments with significant lighting changes or insufficient texture, where traditional handcrafted keypoints might not be detected or matched reliably. In recent years, deep learning has demonstrated remarkable capabilities in computer vision tasks, such as image classification and object detection. Its integration into visual odometry (VO) and SLAM has garnered considerable interest. End-to-end methods use deep neural networks (DNNs) to directly infer pose estimates from images [6; 7; 8; 9]. However, these methods often lack interpretability and may have difficulties generalizing across different scenarios due to discrepancies between training and testing data distributions. Learned-feature-based SLAM methods [10; 11; 12; 13] replace traditional feature detection with deep learning-based visual feature extraction, which is then integrated into the SLAM back-end. These methods leverage both deep learning and geometric models to enhance SLAM performance. Robust keypoint tracking and matching are vital for the reliability and effectiveness of visual SLAM systems in various applications. Although previous research has explored data-driven feature extraction in SLAM systems, there is still a need for more effective feature extraction networks tailored to the dynamic conditions encountered in SLAM applications. Current neural networks for feature extraction, such as SuperPoint [14], are typically trained on static object detection datasets, which may limit their performance in dynamic, real-world environments with varying lighting and movement. Additionally, achieving reliable matching for learned features and efficient loop closing remains a significant challenge. Unlike ORB features, which encapsulate explicit low-level image information, learned features often produce high-dimensional descriptor embeddings, complicating the process of establishing matches and filtering outliers. Moreover, enabling loop closure detection using learned features poses further difficulties. Learning-based Bag-of-Words (BoW) methods for loop closure detection involve complex training processes, often requiring extensive offline training and substantial memory storage, and they struggle with generalization to new environments.