# A Novel Visual SLAM Based on Multiple Deep Neural Networks
## 3 The Proposed SLAM
### \(\mathfrak{se}(3)\)-Based Pose Fusion Estimation  
To further improve the performance of the system, we proposed a novel algorithm for relative positional fusion estimation over \(\mathfrak{se}(3)\). The reason for choosing to perform pose fusion on \(\mathfrak{se}(3)\) is that the pose can perform vector addition and scalar-vector multiplication on \(\mathfrak{se}(3)\). For more details on \(\mathfrak{se}(3)\), see [38].  
To maintain the correlation of feature points across multiple frames and to control the size of the optimization problem, visual SLAM usually calculates the camera pose by tracking the map points of the reference frame, but tracking only the map points of the reference frame results in fewer feature-landmark pairs participating in pose estimation as the camera moves away from the reference frame, which degrades the pose estimation performance. Our \(\mathfrak{se}(3)\)-based pose fusion algorithm uses the map points generated in the last frame so that there are enough feature-landmark pairs to participate in the pose estimation even when the camera is far away from the reference frame to ensure the quality of the estimation. Moreover, since the map points generated in the last frame are \"temporary\", they do not increase the size of the optimization problem.  
First, \(N_{r}\) map points are tracked in the reference keyframe and \(N_{l}\) temporary map points are tracked in the last frame using \(SG\)-Matcher. Then, we calculate the relative poses \(\hat{\mathbf{T}}_{cr}\) of the current frame relative to the reference keyframe based on the correspondence between the tracked \(N_{r}\) map points and the keypoints of the current frame, and the relative poses \(\hat{\mathbf{T}}_{cl}\) of the current frame relative to the last frame based on the correspondence between the tracked \(N_{l}\) temporary map points and the keypoints of the current frame. Lastly, the final relative pose \(\hat{\mathbf{T}}_{cr}\) is obtained by weighted fusion of \(\hat{\mathbf{T}}_{cr}\) and \(\hat{\mathbf{T}}_{cl}\) (\(\hat{\mathbf{T}}_{lr}\) is the relative pose of the last frame w.r.t. the reference keyframe) on \(\mathfrak{se}(3)\), and the absolute pose \(\hat{\mathbf{T}}_{c}\) of the current frame (\(\mathbf{T}_{r}\) is the absolute pose of the reference keyframe) is calculated, as shown in Equations (1) and (2).  
\[\hat{\mathbf{T}}_{cr}=\exp((\boldsymbol{\alpha}\ln(\hat{\mathbf{T}}_{cr})^{ \vee}+[1-\boldsymbol{\alpha}]\ln(\hat{\mathbf{T}}_{cl}\hat{\mathbf{T}}_{lr}) ^{\vee})^{\wedge}), \tag{1}\]  
Figure 4: Comparison of SuperPoint and SuperGlue networks before and after restructuring.  
\[\mathbf{T}_{c}=\mathbf{\hat{T}}_{cr}\mathbf{T}_{r}, \tag{2}\]  
where  
\[\mathbf{\alpha}=\frac{\exp(-\frac{R_{\mathbf{T}_{cr}}}{N_{r}})}{\exp(-\frac{R_{ \mathbf{T}_{cr}}}{N_{r}})+\exp(-\frac{R_{\mathbf{T}_{cl}}}{N_{l}})},\]  
where \(R_{\mathbf{T}_{cr}}\) and \(R_{\mathbf{T}_{cl}}\) are used to indicate the uncertainties of \(\mathbf{\bar{T}}_{cr}\) and \(\mathbf{T}_{cl}\), respectively, which can be obtained for free in the calculation of \(\mathbf{\bar{T}}_{cr}\) and \(\mathbf{T}_{cl}\). They are defined as follows:  
\[R_{\mathbf{T}_{cr}}=\sum_{j}^{N_{r}}\left\|\mathbf{x}_{j}^{r}-\Pi(\mathbf{ \bar{T}}_{cr}\mathbf{T}_{r}\mathbf{p}_{j}^{r})\right\|_{2}^{2},\]  
\[R_{\mathbf{T}_{cl}}=\sum_{j}^{N_{l}}\left\|\mathbf{x}_{j}^{l}-\Pi(\mathbf{T} _{cl}\mathbf{T}_{lr}\mathbf{T}_{r}\mathbf{p}_{j}^{l})\right\|_{2}^{2},\]  
where \(\mathbf{x}_{j}^{r}\) is the keypoint in the current frame corresponding to map point \(\mathbf{p}_{j}^{r}\), and \(\mathbf{x}_{j}^{l}\) is the keypoint in the current frame corresponding to the temporary map point \(\mathbf{p}_{j}^{l}\). The projection function \(\Pi\) is defined as follows:  
\[\Pi(\mathbf{p})=\begin{bmatrix}f_{x}\frac{x}{z}+c_{x}\\ f_{y}\frac{y}{z}+c_{y}\end{bmatrix},\]  
with \(\mathbf{p}=[x,y,z]^{\mathrm{T}}\) as the 3D point, \(f_{x}\) and \(f_{y}\) as the focal lengths of the camera, and \((c_{x},c_{y})\) as the principal point of the camera.