# Variable Rate Point Cloud Attribute Compression with Non-Local Attention Optimization
## 3 Point Cloud Attribute Variational Autoencoder
### Non-Local Attention Module  
The implementation of typical non-local block (NLB) [20] in point cloud is shown in Figure 2a. An input feature map \(M\in\mathcal{R}^{N\times C}\) is output from the previous layer, where \(N\) and \(C\) indicate the point number and channel number. Three \(1\times 1\) convolution \(W_{\phi}\), \(W_{\phi}\) and \(W_{\gamma}\) are used to transform \(X\) to different features \(\phi\in\mathcal{R}^{N\times\hat{C}}\), \(\theta\in\mathcal{R}^{N\times\hat{C}}\) and \(\gamma\in\mathcal{R}^{N\times\hat{C}}\) as  
\[\phi=W_{\phi}(M),\quad\theta=W_{\theta}(M),\quad\gamma=W_{\gamma}(M), \tag{7}\]  
where \(\hat{C}\) is the channel number of the new embeddings. Then, the similarity matrix \(V\in\mathcal{R}^{N\times N}\) is calculated by a matrix multiplication as  
\[V=\phi\times\theta^{\Gamma}. \tag{8}\]  
Afterward, the softmax normalization is applied to \(V\) to get a similarity matrix as  
\[\vec{V}=softmax(V). \tag{9}\]  
The output of the attention layer is obtained by matrix multiplication as  
\[O=\vec{V}\times\gamma, \tag{10}\]  
where \(O\in\mathcal{R}^{N\times\hat{C}}\). By referring to the design of the non-local block, the final output is given by  
\[\tilde{M}=W_{o}\Big{(}O^{\Gamma}\Big{)}+M, \tag{11}\]  
where \(W_{o}\), also implemented by a \(1\times 1\) convolution, acts as a weighting parameter to adjust the importance of the non-local operation with respect to the original input \(M\) and moreover, recovers the channel dimension from \(\hat{C}\) to \(C\). Considering global information, the non-local block effectively and adaptively diverts attention to the most important regions of an image in spatial dimension.  
Inspired by the successful applications in computer vision of channel and spatial attention mechanism [24; 25; 26; 27], we use the NLAM to generate feature maps in both dimensions. As detailed in Figure 2b, the NLAM unit includes the main branch, mask branch and connection branch. The main branch still processes features, and can be implemented by any state-of-the-art structure such as a residual or inception block. In this work, we stack three sparse convolutional residual blocks. The mask branch focuses on non-local operation to learn a mask of the same size that weights output features from the main branch, which includes the non-local block, followed three residual blocks (RBs), one \(1\times 1\)  
Figure 2: (**a**) non-local block. \(\bigoplus\) is the add operation and \(\bigotimes\) is the matrix multiplication. (**b**) non-local attention module. \(\bigodot\) means element-wise product.  
convolution (Conv\({}_{1\times 1}\)) and non-linear sigmoid activation. The sigmoid layer normalizes the output to [0, 1] and then the overall attention mask \(\hat{M}\) can be written as  
\[\hat{M}=\text{Sigmoid}(\text{Conv}_{1\times 1}(\text{RBs}(\text{NLB}(X)))). \tag{12}\]  
The third skip connection is used for faster convergence [28]. Then the final output of NLAM can be represented as  
\[\hat{X}=\left(\text{RBs}(X)\odot\hat{M}\right)+X. \tag{13}\]  
where \(\circlearrowleft\) means element-wise product. The NLAM can provide fine-grained masks to distinguish the importance of latent features that will be further down sampled or compressed. Figure 3 visualizes the input point cloud Bird and its corresponding mask generated by the first NLAM. As shown in Figure 3e-h, the closer the color is to purplish red, the closer the weight is to 1, and the closer the color is to green, the closer the weight is to 0. The textured areas such as head and upper surface of the wing are allocated higher weights to retain the original features as shown in Figure 3b,c,f,g. The relatively plain areas such as belly and under surface of the wing are allocated lower weights to weaken their influence as shown in Figure 3d,h.