# A Survey of Visual SLAM Based on RGB-D Images Using Deep Learning and Comparative Study for VOE
## 3 Visual SLAM and Visual Odometry Using Deep Learning: Survey
### Deep Learning-Based Module for Visual SLAM and Visual Odometry
#### 3.1.5 End-to-End for the Visual SLAM Algorithm
\begin{table}
\begin{tabular}{l c c c c c c c c c} \hline \hline  & & \multicolumn{2}{c}{NYU} & \multicolumn{2}{c}{NYU} & \multicolumn{2}{c}{**KITTI**} & \multicolumn{2}{c}{**Make3D [(50)]**} \\ \multirow{2}{*}{**Authors/Year**} & **Dataset/** & \multicolumn{2}{c}{V2} & \multicolumn{2}{c}{V2} & \multicolumn{2}{c}{**2012**} & \multicolumn{2}{c}{**Dataset**} & \multicolumn{2}{c}{**Dataset**} \\ \cline{3-10}  & **Methods** & \multicolumn{2}{c}{**RGB**} & \multicolumn{2}{c}{**Depth**} & \multicolumn{2}{c}{**RGB**} & \multicolumn{2}{c}{**RGB**} \\ \cline{3-10}  & & \(RMSE\) & \(REL\) & \(RMSE\) & \(REL\) & \(RMSE\) & \(REL\) & \(RMSE\) & \(REL\) \\ \hline \([197]\)/2019 & Samples\_0 & - & - & - & - & 5.437 & - & - & - \\ \hline \([197]\)/2019 & Samples\_500 & - & - & - & - & 5.389 & - & - & - \\ \hline \([196]\)/2019 & Samples\_500 & - & - & - & - & 5.14 & - & - & - \\ \hline \([192]\)/2019 & Samples\_500 & - & - & - & - & 3.033 & 0.051 & 5.658 & 0.135 \\ \hline \([198]\)/2020 & DEM\_ & samples\_0 & 0.49 & 0.135 & - & - & 4.433 & 0.101 & 10.003 & 0.529 \\ \hline \([198]\)/2020 & \multicolumn{2}{c}{w/o pre-trained} & \multicolumn{2}{c}{} & \multicolumn{2}{c}{} & \multicolumn{2}{c}{} & \multicolumn{2}{c}{} & \multicolumn{2}{c}{} & \\ \cline{3-10}  & weights & 0.637 & 0.187 & - & - & - & - & - & - \\ \hline \([198]\)/2020 & DEM\_samples\_20 & 0.314 & 0.069 & 0.443 & 0.1 & - & - & - & - \\ \hline \([198]\)/2020 & DEM\_samples\_200 & 0.194 & 0.036 & 0.223 & 0.041 & - & - & - & - \\ \hline \([198]\)/2020 & \multicolumn{2}{c}{w/o pre-trained} & \multicolumn{2}{c}{} & \multicolumn{2}{c}{} & \multicolumn{2}{c}{} & \multicolumn{2}{c}{} & \multicolumn{2}{c}{} & \multicolumn{2}{c}{} & \multicolumn{2}{c}{} \\ weights & 0.226 & 0.042 & 0.23 & 0.043 & - & - & - & - \\ \hline \([198]\)/2020 & DEM\_samples\_50 & - & - & 0.342 & 0.07 & - & - & - \\ \hline \([198]\)/2020 & DEM\_samples\_500 & - & - & - & - & 2.485 & 0.04 & 5.455 & 0.104 \\ \hline \end{tabular}
\end{table} 
Table 11: \(Cont.\)[PERSON] et al. [204] proposed a Visual-Inertial-Odometry Learner (VIOLearner) method based on an unsupervised DNN to combine RGB-D images and inertial measurement unit (IMU) intrinsic parameters of the camera to estimate the camera's moving trajectory in the environment. IMU data are fed through CNN layers, and the output is a 3D Affine matrix that estimates the change in camera pose between a source image and a target image. VIOLearner uses input data, including an RGB-D source image, a target RGB image, IMU data, and a camera calibration matrix \(K\) with the camera's intrinsic parameters. VIOLearner generates hypothesis trajectories and then corrects them online according to the Jacobians of the error image obtained with the original coordinates. [PERSON] et al. [205] proposed a DL framework, called D3 VO, for building VOE with three levels: deep depth, pose, and uncertainty estimation. The first level is to use a self-supervised network to estimate depth from stereo videos using DepthNet from a single image, the second level is to estimate the pose between adjacent images using PoseNet, and the third level is to estimate the associated uncertainty. Incorporating temporal information into the depth estimation learning process. [PERSON] et al. [206] proposed SelfVIO (self-supervised DL-based VIO) to estimate the camera's moving trajectory and estimate depth from input data of monocular RGB image sequences and IMU. SelfVIO can perform estimates of relative translation and rotation between consecutive frames parametrized as 6-DoF motion and a depth image. To recover the camera's movement trajectory in the environment, SelfVIO used the conv. layers. Studies based on end-to-end DL for building VOE systems/estimating trajectory motion in the environment often use metrics \(t_{rel}\) and \(r_{rel}\) to evaluate the results, which have been presented above. The results based on these two measures are that lower is better.