## 3 Methodology
### Drivable Region Mapping  
The results of semantic segmentation \(\mathbf{u}_{i}\) with depth value \(D(\mathbf{u}_{i})\) enable the transformation of pixels to \(\mathbf{p}_{i}\). For every \(\mathbf{u}_{i}\) in \(I_{t}\), \(\widetilde{P}_{t}\) is obtained, and Figure 2 shows the result of Equation (2). Additionally, \(\widetilde{P}_{t}\) is in the camera coordinates. The generation of excessive point clouds from \(I_{t}\) leads to computational memory constraints, which prevent the direct utilization of \(\widetilde{P}_{t}\). Therefore, the number of point clouds should be reduced through a voxel grid filter. For the simplicity, we denote \(\widetilde{P}_{t}\) as filtered point clouds. **Integrated Semantic Segmentation and 3D Mapping**: As we mentioned in system overview section, the semantic segmentation result \(\widetilde{P}_{t}\) and the visual SLAM results (\(\mathcal{M}_{j}\), \(\mathcal{X}\)) are obtained. To consolidate \(\widetilde{P}_{t}\) into a single \(\mathcal{M}_{j}\), it is necessary to transform camera coordinates \(\widetilde{P}_{t}\) into world coordinates, following Equation (3). The update is subsequently performed according to Equation (4). However, during this update, many duplicated \(\widetilde{P}_{t}^{W}\) can be generated at the same location, which degrades the computing performance. To prevent this, KDTree [52] is utilized to remove duplicate \(\widetilde{P}_{t}^{W}\) within a specified threshold. Additionally, loop closure is necessary. When a loop is detected in ORB-SLAM3, the system enters the loop closure phase. In the loop closure phase, \(\mathcal{X}\) is modified through pose graph optimization and global bundle adjustment (BA). Consequently, \(\mathcal{M}_{j}\) is also changed; thus, \(\widetilde{P}_{t}^{W}\) must be adjusted accordingly. Figure 1: Schematic overview of the proposed method. It consists of precise drivable region mapping (**left**) and map-merging for large-scale mapping (**right**). According to the full BA of ORB-SLAM3, the optimization problem is as follows:  
\[\left\{\mathbf{x}_{t},\mathcal{M}_{j}\right\}=\underset{\mathbf{x}_{t},\mathcal{ M}_{j}}{\arg\min}\,E_{point} \tag{5}\]  
\[E_{point}=\sum_{k\in\mathcal{K}}\sum_{j\in\chi_{k}}\rho(E(k,j)) \tag{6}\]  
where \(E(k,j)=\left\|\mathbf{u}^{j}-\pi\big{(}\mathbf{R}_{k}\mathbf{p}^{j}+\mathbf{t }_{k}\big{)}\right\|_{\Sigma}^{2}\) is the re-projection error between matched 3D points \(\mathbf{p}^{j}\in\mathbb{R}^{3}\) in keyframes \(\mathcal{K}\) and key points \(\mathbf{u}^{j}\in\mathbb{R}^{2}\) with \(j\in\chi_{k}\), the set of all matches. Additionally, \(\pi(\cdot)\) is the projection function mentioned in Equation (1), and \(\rho(\cdot)\) is the robust Huber cost function [53]. Further, the label information of \(\tilde{P}_{t}^{\text{W}}\) is considered. The label-encoded error function \(E_{c}\), which represents how correctly the current class label aligns with existing point cloud labels, is defined as  
\[E_{c}=\sum_{k}\sum_{j}\left[\mathcal{L}\big{(}\mathcal{M}_{j}(P_{t}^{*})\big{)} =\mathcal{L}\Big{(}\mathbf{R}_{k}\mathbf{p}^{j}+\mathbf{t}_{k}\Big{)}\right] \tag{7}\]  
where \([\cdot]\) indicates the Iverson bracket. Point \(\mathcal{M}_{j}(P_{t}^{*})\) is the nearest point from \(\mathbf{R}_{k}\mathbf{p}^{j}+\mathbf{t}_{k}\) within the existing 3D point map \(\mathcal{M}_{j}\). Additionally, \(\mathcal{L}(\cdot)\) indicates the label value of the input points. Therefore, the final BA is formulated as a nonlinear least-squares optimization problem as follows:  
\[\left\{\mathbf{x}_{t},\widetilde{\mathcal{M}}\right\}=\underset{\mathbf{x}_{t },\widetilde{\mathcal{M}}}{\arg\min}\,E_{point}+E_{c} \tag{8}\]  
Finally, the output of BA \(\left\{\mathbf{R},\mathbf{t}\right\}\) is used to adjust \(\tilde{P}_{t}^{\text{W}}\). \[\mathcal{M}_{j}\leftarrow\left(\mathbf{R}\tilde{P}_{t}^{\text{W}}+\mathbf{t} \right)\cup\mathcal{M}_{j} \tag{9}\]  
As shown in Figure 3, the keyframe with the corresponding \(\tilde{P}_{t}^{\text{W}}\) is matched in real time.