# Fast 3D Semantic Mapping in Road Scenes +
## 4 3D Semantic Mapping
### Map Regularization  
The dense CRF is widely used in the 2D semantic segmentation to enhance the performance of semantic segmentation. Some previous works [8; 9; 35] seek its application on the 3D map to model contextual relations between various class labels in a fully connected graph. It is a heuristic  
Figure 3: The Gaussian translation from a pixel \(\mathbf{p}\) in the image \(I_{i}\) of the keyframe \(\mathcal{K}_{i}\) to estimated pixels \(\hat{\mathbf{p}}\) in the image \(I_{j}\) of the keyframe \(\mathcal{K}_{j}\). approach that assumes the influence between neighbors should be proportional to their distance, visual, and geometrical similarity [9]. The CRF model is defined as a graph composed of unary potentials as nodes and pairwise potentials as edges, but the size of the model makes traditional inference algorithms impractical. Thanks to [PERSON] and [PERSON]'s work [39], a highly efficient approximate inference algorithm is proposed to handle this issue by defining the pairwise edge potentials as a linear combination of Gaussian kernels. We apply the efficient inference of the dense CRF to maximize label agreement between similar 3D points as follows. Assume the 3D semantic map \(\mathcal{M}\) containing \(M\) 3D points is defined as a random field. A CRF \((\mathcal{M},\mathbf{X})\) is characterized by a Gibbs distribution as follows:  
\[P(\mathbf{X}|\mathcal{M})=\frac{1}{Z(\mathcal{M})}\exp(-E(\mathbf{X}|\mathcal{ M})), \tag{12}\]  
where \(E(\mathbf{X}|\mathcal{M})\) is the Gibbs energy and \(Z(\mathcal{M})\) is the partition function. The maximum a posteriori (MAP) labeling of the random field is  
\[\mathbf{X}^{*}=\operatorname*{argmax}_{l\in\mathcal{L}}P(\mathbf{X}|\mathcal{ M})=\operatorname*{argmin}_{l\in\mathcal{L}}E(\mathbf{X}|\mathcal{M}), \tag{13}\]  
which is converted into minimizing the Gibbs energy by the mean-field approximation and message passing scheme. We employ the associative hierarchical CRF [35; 40] which integrates the unary potential \(\psi_{i}\), the pairwise potential \(\psi_{i,j}\), and the higher order potential \(\psi_{c}\) into the Gibbs energy at different levels of the hierarchy (voxels and supervoxels) given by:  
\[E(\mathbf{X}|\mathbf{C};\theta)=\sum_{i}\psi_{i}(X_{i}|\mathbf{C})+\sum_{i<j} \psi_{i,j}(X_{i},X_{j}|\mathbf{C};\theta)+\sum_{c}\psi_{c}(X_{c}|\mathbf{c}) \tag{14}\]  
by the indexes \(i,j\in\{1,\ldots,M\}\) correspond to different 3D points \(\mathbf{P}_{i},\mathbf{P}_{j}\) in the 3D map \(\mathcal{M}\). _Unary Potential_: The unary potential \(\psi_{i}(\cdot)\) is defined as the negative logarithm of the probabilistic label for a given 3D point:  
\[\psi_{i}(X_{i}|\mathbf{C})=-\log(P(X_{i}\to l|\mathcal{K}_{0}^{\dagger})). \tag{15}\]  
This term means the cost of 3D point \(P_{i}\) taking an object label \(l\in\mathcal{L}\) based on the incremental semantic probabilistic fusion above. The output of the unary potential for each point is produced independently, and thus, the MAP labeling produced by the unary potential alone is generally inconsistent. _Pairwise Potentials_: The pairwise potential \(\psi_{i,j}(\cdot)\) is modeled to be a log-linear combination of \(m\) Gaussian edge potential kernels:  
\[\psi_{i,j}(X_{i},X_{j}|\mathbf{C};\theta)=\mu(X_{i},X_{j})\sum_{m}\omega^{(m) }k^{(m)}(\mathbf{f}_{i},\mathbf{f}_{j};\theta), \tag{16}\]  
where \(\mu(\cdot)\) is a label compatibility function corresponding to the Gaussian kernel functions \(k^{(m)}(\mathbf{f}_{i},\mathbf{f}_{j})\). \(\mathbf{f}\) denotes the feature vector for the 3D point \(\mathbf{P}\) including the position, the RGB appearance and the surface normal vector of the reconstructed surface. Furthermore, \(\mu(\cdot)\) is defined as a Potts model given by:  
\[\mu(l,l^{\prime})=[l\
eq l^{\prime}]=\left\{ \begin{array}{cc}1&l\
eq l^{ \prime}\\ 0&l=l^{\prime}\end{array} \right.. \tag{17}\]  
This term is defined to encourage the consistency over pairs of neighboring points for the local smoothness of the 3D semantic map.