## 5 Deep-Learning-Based Methods for Point Cloud Understanding
### Point-Based Methods  
Point-based methods do not rely on an intermediate representation. However, the lack of canonical order and permutation invariance makes raw point data infeasible for convolutional network architectures that require regular input data. This was first solved in 2017 with _PointNet_[17]. Unlike the previously mentioned projection and discretization methods, points are processed through a fully connected multi-layer perceptron (MLP) network to learn per-point features. The key to this approach was a single max-pooling layer that was trained to select a subset of informative points. The output is a global signature of the input set, which can be used for shape classification tasks. However, pointwise semantic segmentation requires local and global knowledge. [PERSON] et al. [17] achieved global awareness with an additional segmentation network, which feeds the global point cloud feature vector back to the local per-point features by simple matrix-vector concatenation. Per-point class scores can be predicted from the combination of local and global point features. Based on the success of PointNet, various segmentation methods have been proposed: point-based convolution networks, graph-based networks, RNN-based networks, and point-based MLP networks. **Point-based convolutional neural networks** (CNNs) apply convolving kernels at each point in the point cloud to learn pointwise features. However, directly convolving kernels against features associated with the points results in losing shape information and variance to point ordering [106]. Many approaches have been proposed to address these issues. [PERSON] et al. [138] invented a convolution operator that bins nearest neighbors into kernel cells before convolving with kernel weights, which can be applied at each point of a point cloud (Figure 16 center). [PERSON] et al. [139] proposed continuous convolution, a new learnable operator that operates over non-grid structured data and uses kernel functions defined for arbitrary points in the continuous support domain. Similarly, [PERSON] [140] replaced discrete kernels with continuous ones to generalize convolution for point clouds. This formulation allows arbitrary point cloud sizes and can easily be used for designing neural networks similar to 2D CNNs. [PERSON] et al. [106] proposed a method to learn a \(\chi\)-transformation that weights and permutes input points and features before convolving them. The jointly learned \(\chi\)-operator is explicitly dependent on the input order, as \(\chi\) is trained to permute the feature vector to ensure permutation invariance. According to [PERSON] et al. [106], this only works to a limited extent but is still significantly better than the direct application of typical convolutions on point clouds. Real-world data are typically associated with inhomogeneous point density. Methods based on a fixed number of samples deteriorate when point density fluctuates [18; 141]. [PERSON] et al. [142] used Monte Carlo convolution for non-uniformly sampled point clouds by phrasing the convolution integral as a Monte Carlo approximation and, thus, providing a new form of robust sampling invariance. Similarly, [PERSON] et al. [143] extended the Monte Carlo approximation method with an MLP in each filter to approximate the weight functions and a density scale to re-weight the learned weight function. They call this permutation-invariant convolution operations PointConv. However, the \"naive\" implementation of PointConv is memory-consuming and inefficient. A reformulation by reducing PointConv to two standard operations (matrix multiplication and 2D convolution) takes advantage of GPU parallel computing and allows easy implementation with mainstream deep learning frameworks. [PERSON] et al.