Graph Attention Convolution for Point Cloud Semantic Segmentation
By: Nabil Madali
Tags: Graph Neural Networks, Convolutional Network, Machine Learning, Point Cloud, Segmentation

Standard convolution has inherent limitations in semantic point cloud segmentation due to the isotropy of its characteristics. It ignores the structure of the object, resulting in a poor description of the object contour in the segmentation result, and a small part of pseudo-regions.[1] proposes a new graphic attention convolution (GAC), its convolution kernel can be dynamically carved into a specific shape to adapt to the structure of the object. Specifically, GAC assigns specific attention weights to different neighboring points, and selectively focuses on their most relevant parts according to their characteristics. The shape of the convolution kernel is determined by the learned attention weight distribution.

Illustration of the standard convolution and GAC on a subgraph of a point cloud. Left: The weights of standard convolution are determined by the neighbors’ spatial positions, and the learned feature at point 1 characterizes all of its neighbors indistinguishably. Right: In GAC, the attentional weights on “chair” (the brown dotted arrows) are masked, so that the convolution kernel can focus on the “table” points.Source[1]

Although GAC is simple, it can capture the structural features of the point cloud and perform fine-grained segmentation to avoid feature pollution between objects. Theoretically, the author conducted an in-depth analysis of GAC’s expressive power and showed how it learns the characteristics of point clouds. Empirically, through challenging experiments on indoor and outdoor data sets, the GAC proposed by the author demonstrates the most advanced performance on the basis of existing deep learning methods.

Method

Consider a graph G(V,E) constructed from a given point cloud

according to their spatial neighbors, where

and E ⊊ |V| x |V|,represent the set of vertices and edges respectively and N is the number of vertices (points).

Denote N (i) = {j : (i, j) ∈ E} ∪ {i} as the neighbor set of vertex i and let

be a set of input vertex features, each feature h_i ∈ R^F is associated with a corresponding graph vertex i ∈ V , where F is the feature dimension of each vertex.

For each fixed point, take the relative position and feature difference of the current node and surrounding nodes as input, predict the corresponding weight value α.

To handle the size-varying neighbors across different vertices and spatial scales, the attentional weights are normalized across all the neighbors of vertex i as follows:

where a˜_{ij,k} is the attentional weight of vertex j to vertex i at the k-th feature channel.

Therefore, the final output of the proposed GAC can be formulated as follows:

where * represents the Hadamard product, which produces the element-wise production of two vectors, and b_i ∈ R^K is a learnable bias.

The overall process can be as shown in the figure below,

GACNet architecture. Our GACNet is constructed on the graph pyramid of a point cloud. On each scale of the graph pyramid, the proposed GAC is applied for local feature learning, followed by the graph pooling for resolution reducing in each feature channel. After that, the learned features are interpolated back to the finest scale layer by layer for point-wise label assignment.

Similar to pointnet++, GACNet first randomly samples in the point cloud , and then crosses out an area with a radius of k (same as ball query in pointnet++) and randomly samples adjacent point, and finally calculate the weight of attention through such sampling points. Finally, the feature of each point is calculated by weighting and then Graph Pooling (in fact, max or mean pooling) is used to downsample the point cloud. The upsampling process is similar to pointnet++, using interpolation + direct connection method.

Conclusion

GAC is designed to selectively focus on the most relevant part according to their dynamically learned features. The shape of the convolution kernel is then determined by the learned distribution of the attentional weights. Though simple, GAC can capture the structured features of point clouds for finegrained segmentation and avoid feature contamination between objects.

References

1.L. Wang, Y. Huang, Y. Hou, S. Zhang and J. Shan, “Graph Attention Convolution for Point Cloud Semantic Segmentation,” 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 2019, pp. 10288–10297, doi: 10.1109/CVPR.2019.01054.