## 3 Hybrid Visual SLAM with Deep Keypoints, Depths, Poses, and Line Features
### Line-Point Bundle Adjustment and Online BoW in the Back End
#### 3.4.1 Line-Point Pose-Only Bundle Adjustment Optimization
=\mathbf{R}_{\mathbf{w}}^{\mathbf{ci }}\mathbf{P}_{\mathbf{s}}^{\mathbf{w}}+\mathbf{t}_{\mathbf{w}}^{\mathbf{ci}} \tag{18}\]  
\[\mathbf{\hat{p}}^{s}=\mathbf{K}\mathbf{P}_{\mathbf{s}}^{\mathbf{ci}} \tag{19}\]  
\[E_{ang} = E_{ss}+E_{es}\] \[E_{ss} = \frac{\overline{\mathbf{\hat{p}}^{s}\overline{\mathbf{\hat{p}}^{e}}} \cdot\overline{\mathbf{\hat{p}}^{s}\overline{\mathbf{\hat{p}}^{e}}}}{\left\|\overline {\mathbf{\hat{p}}^{e}}\overline{\mathbf{\hat{p}}^{e}}\right\|\left\|\overline{\mathbf{\hat {p}}^{e}}\overline{\mathbf{\hat{p}}^{e}}\right\|}-1\] \[E_{es} = \frac{\overline{\mathbf{\hat{p}}^{e}\overline{\mathbf{\hat{p}}^{s}}} \cdot\overline{\mathbf{\hat{p}}^{e}}\overline{\mathbf{\hat{p}}^{s}}}{\left\|\overline {\mathbf{\hat{p}}^{e}}\overline{\mathbf{\hat{p}}^{s}}\right\|\left\|\overline{\mathbf{\hat {p}}^{e}}\overline{\mathbf{\hat{p}}^{s}}\right\|}-1 \tag{20}\]  
As illustrated in Figure 3, the line feature angle error measures the angular discrepancy between the endpoints of the projected line and the observed line. As shown in Equation (20), we calculate the inner product between the directed line segments \((\overline{\mathbf{\hat{p}}^{s}\overline{\mathbf{\hat{p}}^{e}}},\overline{\mathbf{\hat{p} }^{s}\overline{\mathbf{\hat{p}}^{e}}})\) and \((\overline{\mathbf{\hat{p}}^{e}\overline{\mathbf{\hat{p}}^{s}}},\overline{\mathbf{\hat{p} }^{e}}\overline{\mathbf{\hat{p}}^{s}})\). This inner product represents the angle error. When the relative pose between the projected and observed lines is closer to the ground truth, the line feature angle errors \(E_{ss}\) and \(E_{es}\) approach zero. The current frame acquires matched keypoints and line features through the feature tracking module and projects the 3D points and line segment endpoints into the current frame's image coordinates using the pinhole camera model. The error equation is formed by the coordinate differences between the observed point-line features and the projected point-line features. This error is minimized by adjusting the pose of the current frame. When the error converges, the pose of the current frame reaches an optimal solution. The error equation based on point-line features includes the projection keypoint error \(E_{po}\) and the projection line endpoint error \(E_{line}\). We use Equations (18) and (19) to transform the 3D coordinates of keypoint features \(\mathbf{P}_{\mathbf{j}}^{\mathbf{w}}\) to the current image frame \(\mathbf{ci}\) as \(\mathbf{\hat{p}}^{\mathbf{j}}\). The corresponding keypoint in the current frame is \(\mathbf{\hat{p}}^{\mathbf{i}}\). The index \(j\) denotes the \(j\)-th keypoint in the current image frame \(\mathbf{ci}\). We calculate the difference between \(\mathbf{\hat{p}}^{\mathbf{j}}\) and \(\mathbf{\hat{p}}^{\mathbf{j}}\) as the projection point error, as shown in Equation (21). \[\mathbf{E}_{\mathbf{po}}^{\mathbf{i},\mathbf{j}}=\mathbf{\hat{p}}_{\mathbf{i}}^{\mathbf{ j}}-\mathbf{\hat{p}}_{\mathbf{i}}^{\mathbf{j}} \tag{21}\]  
As shown in Equation (22), we combine the line feature error from Equation (15) and the point error from Equation (21) to form the total optimization error. The optimal pose of the current frame, \(\xi^{*}\), is determined using the [PERSON] descent method. In this context, \(\mathbf{\Omega}_{\mathbf{i},\mathbf{j}}\) and \(\mathbf{\Omega}_{\mathbf{i},\mathbf{k}}\) represent the covariances of the observed keypoints and line features, respectively. \(\mathcal{H}\) denotes the Huber kernel function, \(N\) is the total number of tracked keypoints, and \(M\) is the total number of tracked line features. The index \(k\) indicates the \(k\)-th line feature in the current image frame \(\mathbf{c_{i}}\). \[\xi^{*}=\underset{\mathbf{i}}{\text{argmin}}\sum_{j=0}^{N}\mathcal{H}(\mathbf{E_ {po}^{i,j}}^{T}\Omega_{i,j}^{-1}\mathbf{E_{po}^{i,j}})+\sum_{k=0}^{M}\mathcal{H }(\mathbf{E_{line}^{i,k}}^{T}\Omega_{i,k}^{-1}\mathbf{E_{line}^{i,k}}) \tag{22}\]