## 3 Pose Optimization Based on Point-Line Features
### Line Feature Reprojection with Depth Information Fusion  
The RGB-D cameras provide the acquisition of color images alongside appropriate depth maps, offering comprehensive depth measurement data for pixel points on line features. This research explicitly integrates line segment depth data to create line feature reprojection errors for optimal utilization of the information. Figure 3 illustrates that when a color image frame is regarded as the left camera image in a stereo camera model (i.e., the present frame), the line feature retrieved from this image is \(S_{L}E_{L}\). Assume that the corresponding spatial line segment is \(S_{W}E_{W}\). In the process of building the virtual right camera image frame, if the pixel coordinates of the left camera are (\(u\),\(v\)) with a depth measurement of \(d\), the corresponding coordinates for the right camera are as follows:  
\[\begin{array}{l}u_{R}=u-\frac{f_{x}b}{d}\\ v_{R}=v\end{array} \tag{9}\]  
Figure 2: Line feature reprojection error diagram: (**a**) line feature projection in 3D space; (**b**) line feature reprojection error in image plane.  
The equation mentioned earlier allows for calculating the associated right camera pixel locations \(S_{R}\) and \(E_{R}\). According to the principles of multi-view geometry, when depth measurements are precise, the corresponding pixel positions in the right camera picture for the pixel points on the left camera line segment should align with the line segment \(S_{R}E_{R}\). Analogous to the building approach of the line feature reprojection error in the left camera picture, we can similarly generate the reprojection error of the identical spatial line segment in the right camera image. The depth observation data from the left camera line segment \(S_{L}E_{L}\) can be incorporated into the reprojection error of the right camera line segment.  
In a stereo vision system, for a three-dimensional line segment, \(S_{W}E_{W}\), the homogeneous coordinates of its projected endpoints in the left camera image are \(S^{\prime}_{L}\) and \(E^{\prime}_{L}\) (predicted values in the optimization problem). In contrast, the homogeneous coordinates of the projected endpoints in the right camera image are \(S^{\prime}_{R}\) and \(E^{\prime}_{R}\). The observed matched line segment in the original image is \(S_{L}E_{L}\) (i.e., the measured values in the optimization problem) with the matching normalized line coefficients, \(L_{L}\). Simultaneously, utilizing depth information, we compute the virtual right camera line segment observation, \(S_{R}E_{R}\), and its associated normalized line coefficients, \(L_{R}\). The ultimate reprojection error of the spatial line segment, \(S_{W}E_{W}\), in the present frame is defined as follows:  
\[E_{\text{final}}=E_{\text{line}}(S^{\prime}_{L},E^{\prime}_{L},L_{L})+E_{ \text{line}}(S^{\prime}_{R},E^{\prime}_{R},L_{R}) \tag{10}\]  
This study enhances the precision of line feature localization by integrating multi-view depth data. To enhance the error measurement, we propose employing a virtual right camera image method that integrates depth data to generate virtual images, ensuring error consistency between the left and right images. This depth-information-driven optimization significantly mitigates localization drift and improves the resilience of the SLAM system in dynamic settings. The final reprojection error calculation enhances the system's accuracy and efficiency by integrating depth data from numerous frames and minimizing errors to refine the camera poses and three-dimensional coordinates of line features. This offers dependable assistance for later 3D reconstruction and path planning, improving the system's efficacy in intricate environments.