## 3 Hybrid Visual SLAM with Deep Keypoints, Depths, Poses, and Line Features
### Line-Point Bundle Adjustment and Online BoW in the Back End  
#### 3.4.1 Line-Point Pose-Only Bundle Adjustment Optimization  
Similar to point features, line features use the difference between projected and observed features to compute the optimization error. SLAM localization is optimized by adjusting the pose estimate to minimize this difference. As illustrated in Figure 3, the blue line (\(\overline{\vec{\mathbf{p}^{s}}\vec{\mathbf{p}^{e}}}\)) represents the observed line segment, while the red line (\(\overline{\vec{\mathbf{p}^{s}}\vec{\mathbf{p}^{e}}}\)) denotes the projected line segment. As shown in Equation (15), we construct a line feature error function to refine the relative pose estimates between adjacent frames. Two types of line feature error functions are designed: the line feature distance error \(E_{out}\) and the line feature angle error \(E_{ang}\). \[E_{line}=E_{out}+E_{ang} \tag{15}\]  
As shown in Formula 16, we first compute the line coefficient \(\mathbf{u}_{i}\), which represents the infinite line passing through the starting point \(\vec{\mathbf{p}^{s}}\) and the ending point \(\vec{\mathbf{p}^{e}}\). In this coefficient, \(\vec{\mathbf{p}^{s}}\) and \(\vec{\mathbf{h}}\vec{\mathbf{p}^{e}}\) denote the homogeneous coordinates of the endpoints of the observed line. Next, we calculate the vertical distance \(E_{ps}\) between the observed starting point \(\vec{\mathbf{p}^{s}}\) and the projected line, as well as the vertical distance \(E_{pe}\) between the observed ending point \(\vec{\mathbf{p}^{e}}\) and the projected line. The sum of these two vertical distances constitutes the projection line distance error \(E_{out}\). \[\mathbf{u}_{i}=\frac{\vec{\mathbf{h}}\vec{\mathbf{p}^{s}}\times\vec{\mathbf{h}} \vec{\mathbf{p}^{e}}}{\left\|\vec{\mathbf{h}}\vec{\mathbf{p}^{s}}\times\vec{ \mathbf{h}}\vec{\mathbf{p}^{e}}\right\|} \tag{16}\]  
Figure 3: Diagram Of Projection Line Feature Error. \[E_{oul}=E_{ps}+E_{pe}=\mathbf{i}_{i}^{T}\mathbf{\hat{p}}^{s}+\mathbf{i}_{i}^{T}\mathbf{\hat{p}}^{e} \tag{17}\]  
Additionally, as shown in Equations (18) and (19), the endpoints of the projected lines in the image adhere to the pinhole camera model. We project the 3D points in world coordinates into the image coordinates using the camera pose and intrinsic matrix. Here, \(\mathbf{P}_{\mathbf{s}}^{\mathbf{w}}\) represents the starting point of the 3D line in world coordinates, while \(\mathbf{R}_{\mathbf{w}}^{\mathbf{ci}}\) and \(\mathbf{t}_{\mathbf{w}}^{\mathbf{ci}}\) denote the rotation and translation matrices that transform coordinates from the world frame \(w\) to the camera frame \(c_{i}\). The index \(i\) indicates the \(i\)-th camera frame. \([\mathbf{X}_{\mathbf{s}}^{\mathbf{ci}},\mathbf{Y}_{\mathbf{s}}^{\mathbf{ci}},\mathbf{Z}_{ \mathbf{s}}^{\mathbf{ci}}]\) are the components of the 3D line's starting point \(\mathbf{P}_{\mathbf{s}}^{\mathbf{ci}}\) in the \(i\)-th camera coordinate system. We then use the intrinsic matrix \(\mathbf{K}\) to project \(\mathbf{P}_{\mathbf{s}}^{\mathbf{ci}}\) onto the image coordinates \(\mathbf{\hat{p}}^{s}\). This projection process is similarly applied to the ending point of the 3D line. \[\mathbf{P}_{\mathbf{s}}^{\mathbf{ci}}= \begin{bmatrix}\mathbf{X}_{\mathbf{s}}^{\mathbf{ ci}}\\ \mathbf{Y}_{\mathbf{s}}^{\mathbf{ci}}\\ \mathbf{Z}_{\mathbf{s}}^{\mathbf{ci}}\end{bmatrix}