# A Survey of Visual SLAM Based on RGB-D Images Using Deep Learning and Comparative Study for VOE
## 3 Visual SLAM and Visual Odometry Using Deep Learning: Survey
### Deep Learning-Based Module for Visual SLAM and Visual Odometry
#### 3.1.1 Depth Estimation  
a. Methods  
According to the depth estimation module, [PERSON] et al. [40] proposed a deep network consisting of two stacks to directly regress depth using the coarse-scale network to estimate the global structure of the scene and using the fine-scale network to refine it using local information. [PERSON] et al. [41] proposed a deep network which is a variation of the hourglass to estimate depth by training a multi-scale deep network and relative depth annotations of the data. The input for pixel-wise depth prediction is a single image. [PERSON] et al. [42] proposed an end-to-end learning deep network for a single-view depth (scene structure) and pose estimation (camera motion) from the image sequence. To predict single-view depth, the DispNet network architecture was used in an encoder-decoder design. To predict the pose of the camera, the target view was concatenated with all the source views according to the color channel of the input image sequence. [PERSON] et al. [43] proposed a method to improve the method performance of [42] for depth estimation and camera pose by the CNN-based with a simple normalization step, thereby significantly improving the performance of depth estimation. In the proposed approach; the authors applied a Direct Visual Odometry (DVO) [44] pose predictor to predict the output pose based on the input dense depth map, thereby reducing information loss of sequence frames during scene reconstruction. [PERSON] et al. [45] proposed an unsupervised CNN learning method based on auto-encoder architecture to predict single view depth without requiring learning from annotated GT depths. The loss function in this CNN represents the difference between the source image and the inverse-warped target image; it represents the correlation of the prediction error and aligns two different depth maps without using the GT of depth maps. [PERSON] et al. [46] proposed an end-to-end unsupervised DL network to estimate monocular depth with a new information loss function that enforces left-right depth consistency inside the network. The information loss function is capable of combining three error information:smoothness/disparity smoothness loss, reconstruction/appearance matching loss, and left-right disparity consistency terms/left-right disparity consistency loss. The loss function is the sum of the above three error information. [PERSON] et al. [47] proposed an unsupervised DL method based on exploiting 3D geometry structure and semantics to build a model to estimate scene depth and ego-motion. The input of the learning method is a sequence of RGB frames and performs the following calculation steps: object masks, object ego-motion, and individual object motion. The output of the learning model is the image warped according to ego-motion. [PERSON] et al. [48] proposed an unsupervised DL network for estimating depth and motion from two consecutive frames of monocular video. The feature used for the training process is a geometry consistency constraint extracted from a self-discovered mask for dynamic scenes and occlusions to enforce scale consistency, from which the motion of a global scale can be estimated. Most of the studies presented above only used RGB images as input to the DL network to estimate the depth of the scene, as researched by [47] by moving objects in a frame sequence to estimate scene depth based on how fast objects move between pairs of color image frames. These studies aim to only use RGB images of cheap cameras without paying attention to the depth data of the datasets presented below. b. Datasets  
KITTI Dataset: The KITTI dataset [9; 10; 11] is the most popular database for evaluating Visual SLAM and VOE models and algorithms. This database includes two versions: the KITTI 2012 dataset [10] and the KITTI 2015 dataset [11]. The KITTI dataset is a computer vision dataset for autonomous driving research. It includes more than 4000 high-resolution images, LIDAR point clouds, and sensor data from a car equipped with various sensors.