## 3 Methodology  
### Problem Formulation  
As shown in Figure 1, our proposed method consists of two steps: drivable region mapping and map-merging.  
In the drivable region mapping process, as the RGB-D camera navigates an unknown environment, the series of input images, denoted by \(\mathcal{I}=\left\{I_{t}\right\}_{t=1:T}\) from time \(t=1\) to \(t=T\), represents the collection of images captured. As \(I_{t}\) is an RGB-D image, it can be converted to point clouds \(P_{t}=\left\{\mathbf{p}_{i}\right\}_{i=1:N}\), where \(N\) denotes the number of points. A camera pose that corresponds to \(P_{t}\) is denoted as \(\mathbf{x}_{t}=\left[\begin{array}{cc}\mathbf{R}&\mathbf{t}\\ 0&\mathbf{1}\end{array}\right]\in\text{SE}(3)\), where \(\mathbf{R}\in\text{SO}(3)\) represents orientation and \(\mathbf{t}\in\mathbb{R}^{3}\) represents position. A camera pose can also be represented with quaternion vectors as \(\mathbf{x}_{t}=\left(\mathbf{x}_{t}^{p},\mathbf{x}_{t}^{q}\right)^{T}\), where \(\mathbf{x}_{t}^{p}=\left(x_{t}\;y_{t}\;z_{t}\right)\) and \(\mathbf{x}_{t}^{q}=\left(q_{x,t}\;q_{y,t}\;q_{z,t}\;q_{w,t}\right)\) are the position and quaternion vectors, respectively. The camera trajectory \(\mathcal{X}=\left\{\mathbf{x}_{t}\right\}_{t=1:t}\) can be obtained using visual SLAM [25] methods. Semantic segmentation outputs labels \(\xi_{t}\in\left\{1,\cdots,cl\right\}\) for each pixel of an image \(I_{t}\), where \(cl\) indicates the number of classes. The main goal of this step is to build fused point clouds \(\widetilde{P}_{t}\), which are composed of \(P_{t}\) and \(\xi_{t}\in\left\{road,sidewalk\right\}\). A 3D point \(\mathbf{p}_{i}\) in the camera coordinates is projected into a 2D pixel point \(\mathbf{u}_{i}\) on the image plane by  
\[\left[\begin{array}{c}\mathbf{u}_{i}\\ 1\end{array}\right]=\left[\begin{array}{c}\pi(\mathbf{p}_{i})\\ 1\end{array}\right]=\frac{1}{z}\mathbf{K}\mathbf{p}_{i} \tag{1}\]  
where \(\mathbf{K}\) is the intrinsic camera matrix and \(\pi(\cdot)\) is a projection function. Conversely, 2D point \(\mathbf{u}\) and a class label \(\widetilde{\xi}\) are converted into 3D point clouds with a depth value of each point [50].  
\[\mathbf{p}_{i}=\pi^{-1}(\mathbf{u}_{i})=D(\mathbf{u}_{i})\mathbf{K}^{-1} \!\left[\begin{array}{c}\mathbf{u}_{i}\\ 1\end{array}\right] \tag{2}\]  
where \(D(\mathbf{u}_{i})\) is the depth value of image pixel \(\mathbf{u}_{i}\). Thus, \(\widetilde{P}_{t}=\left\{P_{t},\widetilde{\xi}\right\}\in\mathbb{R}^{4}\) is composed of point clouds and each point's class label. The world coordinate point cloud map \(\widetilde{P}_{t}^{W}\) can be generated by the camera pose \(\mathbf{R}\) and \(\mathbf{t}\).  
\[\widetilde{P}_{t}^{W}=\left\{\mathbf{R}\widetilde{P}_{t}+\mathbf{t},\widetilde {\xi}\right\} \tag{3}\]  
The 3D feature point clouds map \(\mathcal{M}_{j}\). Here, \(j\) indicates the j-th submap index, which is constructed by combining the SLAM results with the semantic point clouds.  
\[\mathcal{M}_{j}\leftarrow\widetilde{P}_{t}^{W}\cup\mathcal{M}_{j} \tag{4}\]  
The next step is map-merging. The separately built maps \(\mathcal{M}_{j=1:N}\) across various regions are combined into a single map \(\widetilde{\mathcal{M}}\). To make \(\widetilde{\mathcal{M}}\), the collection of feature points \(\mathcal{F}_{j}=\left\{\mathbf{f}_{k,j}\right\}_{k=1:K}\) is extracted using FPFH [45] from \(\mathcal{M}_{j}\). To integrate maps developed at various times and locations, the 3D point-registration technique was used [51].