# A Survey of Visual SLAM Based on RGB-D Images Using Deep Learning and Comparative Study for VOE
## 3 Visual SLAM and Visual Odometry Using Deep Learning: Survey
### Deep Learning-Based Module for Visual SLAM and Visual Odometry
#### 3.1.4 DL Modules Add to the Visual SLAM Algorithm
[159] suggested the Differentiable Mapping Network (DMN) based on a combination of spatial structure and an end-to-end training model for mapping. The DMN performs the construction of maps that allow embedding views in a spatial structure. Particle filters are used to localize image sequences using particle filters. The gradient descent is used to combine the map representation and localization. To evaluate the performance of the map construction DL module in the Visual SLAM system, the researchers used several datasets with RGB-D data, as shown below. The KITTI 2012 dataset, NYU RGB-D V2 dataset [49], TUM RGB-D SLAM dataset, and ICL-NUIM dataset have been presented above. The Mask-RCNN MC dataset [149] is a self-generated dataset, including 10,000 images collected from 21 types of objects commonly found in homes and laboratories (persons, robots, suitcases, chairs, air conditioners, desks, bookcases, cats, jackboard, door, TV, potted plant, book, mouse, dog, umbrella, drone, bed, laptop, cell phone, and keyboard). The data were collected based on an MS Kinect V2 connected to a laptop with Intel i5-7500, 32 GB of memory, and a GPU GTX 1080. It was mounted on a moving robot. The data were divided into 90% for training and 10% for testing. To evaluate the performance of the map construction DL module in the Visual SLAM system, the researchers used several datasets with RGB-D data, as shown below. Measures \(t_{rel}\), \(r_{rel}\), and \(RMSE\) have been presented above. The average log error (log)(\(ALE\)) measure is calculated based on Formula (10). \[ALE=\sqrt{\frac{1}{N}\sum_{p}(log(d_{p}^{gt}-log(d_{p}))^{2}} \tag{10}\]  
The absolute relative error (\(AbRE\)) measure is calculated based on Formula (11). \[AbRE=\frac{1}{N}\sum_{p}\frac{|d_{p}^{gt}-d_{p}|}{d_{p}^{gt}} \tag{11}\]  
where \(d_{p}^{gt}\) and \(d_{p}\) are the GT depth and estimated depth of pixel \(p\), respectively. The displacement error (DE--\(e^{t}\)) is the displacement error of the object compared to the GT position. The rotation error (RE--\(e^{r}\)) is the object's rotation angle error compared to the GT data. The results of Visual SLAM when using the DL map construction module are presented in Table 8. Similar to the previous modules, the map construction module using DL was also evaluated on many datasets and with many different measures, so many cells in Table 8 are empty. The above results are that the smaller the measures are, the better. e. Loop closure detection module DL  
[PERSON] et al. [160] proposed a pre-trained CNN model for creating an appropriate image representation to detect visual loop closure. The pre-trained CNN model was trained from more than 2.5 million images of 205 scene categories of the scene-centric dataset, making it easy to extract CNN whole-image descriptors and then select the most suitable layer for detecting Visual SLAM's loop closure. [PERSON] et al. [161] proposed to use and compare several DL networks to detect loop closure in Visual SLAM: PCANet, CaffeNet, AlexNet, and GoogLeNet. [PERSON] et al. [162] proposed using a pre-trained CNN model to generate whole-image descriptors for loop closure detection. To detect loop closure, the CNN model performed similarity matrix calculation. The architecture of the CNN model included convolution, a max-pooling operation, and a fully connected layer with an input image size of 221 \(\times\) 221, and the output was a vector with more than 1000 elements. [PERSON] et al. [163] proposed an unsupervised DL model with a conv. auto-encoder network architecture. The proposed network used the histogram of oriented gradients (HOGs) feature on the training data, thereby creating a compact and lightweight model for real-time loop closing. \begin{table}
\begin{tabular}{p{42.7 pt} p{42. [PERSON] et al. [167] proposed a method using two DL networks to detect loop closure detection more accurately. The proposed method ignores coarse features such as moving objects in the environment, such as cycles, bikes, pedestrians, vehicles, and any animals.