# Real-Time Semantic Segmentation of Point Clouds Based on an Attention Mechanism and a Sparse Tensor
## 3 Methods
### Global Feature-Learning Module  
To obtain global contextual information and to distinguish the importance of different spatial location information in the input features, our model adds the GFLM to the deep features of the subsampling module to obtain a spatial weight matrix through training. The specific structure is described below. First, the input feature \(\mathrm{X}\)\(\in\)\(\mathrm{R}^{(\mathrm{N}\times\mathrm{C})}\) is passed through three \(1\times 1\) convolutions, \(\mathrm{W}_{\varphi}\), \(\mathrm{W}_{\theta}\), and \(\mathrm{W}_{\gamma}\), to obtain the features \(\varphi\)\(\in\)\(\mathrm{R}^{(\mathrm{N}\times\mathrm{C})}\), \(\theta\)\(\in\)\(\mathrm{R}^{(\mathrm{N}\times\mathrm{C})}\), and \(\gamma\)\(\in\)\(\mathrm{R}^{(\mathrm{N}\times\mathrm{C})}\), respectively:  
\[\varphi =\mathrm{W}_{\varphi}(\mathrm{X}) \tag{1}\] \[\theta =\mathrm{W}_{\theta}(\mathrm{X})\] (2) \[\gamma =\mathrm{W}_{\gamma}(\mathrm{X}) \tag{3}\]  
A sampling module is added after the features \(\theta\) and \(\gamma\). In this module, 3 pooling operations of 2 \(\times\) 2, 4 \(\times\) 4, and 8 \(\times\) 8 are performed, respectively, and the results are concatenated and used as inputs for the next layer. The sampled results are denoted as \(\theta_{\mathrm{P}}\)\(\in\)\(\mathrm{R}^{(\mathrm{S}\times\mathrm{C})}\) and \(\gamma_{\mathrm{P}}\)\(\in\)\(\mathrm{R}^{(\mathrm{S}\times\mathrm{C})}\) where \(\mathrm{S}\) is the number of anchor points sampled. The similar matrix \(\mathrm{V}_{\mathrm{P}}\) between \(\varphi\) and \(\theta_{\mathrm{P}}\) is then calculated as follows:  
\[\mathrm{V}_{\mathrm{P}}\)\(=\)\(\varphi\)\(\cdot\)\(\theta_{\mathrm{P}}^{\mathrm{T}}\) (4)  
where \(\mathrm{V}_{\mathrm{P}}\)\(\in\)\(\mathrm{R}^{(\mathrm{N}\times\mathrm{S})}\), which after a normalization function, is obtained as \(\mathrm{V}_{\mathrm{P}}\)\({}^{\prime}\). The attention matrix is obtained as:  
\[\mathrm{Q}_{\mathrm{P}}\)\(=\)\(\mathrm{V}\)\(\mathrm{p}\)\(\prime\)\(\cdot\)\(\gamma_{\mathrm{P}}\) (5)  
Finally, this module sums the attention map with the input features to obtain the feature \(\mathrm{X}^{\prime}\)\(\in\)\(\mathrm{R}^{(\mathrm{N}\times\mathrm{C})}\). \(\mathrm{X}^{\prime}\) is used as the final output feature with the following equation:  
\[\mathrm{X}^{\prime}=\mathrm{W}_{\mathrm{Q}}(\mathrm{Q}_{\mathrm{P}})+\mathrm{X} \tag{6}\]  
where \(\mathrm{W}_{\mathrm{Q}}\) is the \(1\times 1\) convolution for recovering the channel number \(\mathrm{C}^{\prime}\) to \(\mathrm{C}\).  
Figure 1: Real-time point cloud semantic segmentation network model structure diagram.  
In addition, in the process of designing the structure of the global feature-learning module, the idea of a lightweight design was introduced to reduce the calculation quantity of the model. The amount of information, such as coordinates and features, in the point cloud data is extremely large. If the dot-product operation is performed directly on the features \(\varphi\) and \(\theta\), the model requires a huge amount of computation. To enhance the efficiency of the model calculations, our model adds a sample module consisting of several pooling operations after the \(1\times 1\) convolution, \(\mathrm{W_{\theta}}\) and \(\mathrm{W_{Y}}\). The sample module samples several sparse anchor points to avoid selecting all spatial points. As a result, the computational complexity is significantly reduced. The data are shown in Table 1. Whether or not the sampling module is added to the global feature-learning module has little effect on the calculation of the number of parameters, which are all 0.2 M. In addition, the FLOPs (floating-point operations per second) were 1.14 G before the sample module was added to this global feature-learning module. With the addition of the sample module, the model FLOPs are reduced by 0.2 G. Therefore, this operation effectively reduces the calculation quantity of the model.