Interpreting Representation Quality of DNNs for 3D Point Cloud Processing
Year: 2022

Abstract:
In this paper, we evaluate the quality of knowledge representations encoded in deep neural networks (DNNs) for 3D point cloud processing. We propose a method to disentangle the overall model vulnerability into the sensitivity to the rotation, the translation, the scale, and local 3D structures. Besides, we also propose metrics to evaluate the spatial smoothness of encoding 3D structures, and the representation complexity of the DNN. Based on such analysis, experiments expose representation problems with classic DNNs, and explain the utility of the adversarial training. The code will be released when this paper is accepted.

heading: 1 Introduction
text: Deep neural networks (DNNs) have exhibited superior performance in various tasks, but the blackbox nature of DNNs hampers the analysis of knowledge representations. Previous studies on explainable AI mainly focused on the following two directions. The first is to explain the knowledge encoded in a DNN, e.g. visualizing patterns encoded by a DNN [24, 36, 50] and estimating the saliency/importance/attribution of input variables w.r.t. the network output [20, 30, 57]. The second is to evaluate the representation power of a DNN, e.g. the generalization and robustness of feature representations.
This paper focuses on the intersection of the above two directions, i.e. analyzing the quality of knowledge representations of DNNs for 3D point cloud processing. Specifically, we aim to design metrics to illustrate properties of feature representations for different point cloud regions, including various types of regional sensitivities, the spatial smoothness of encoding 3D structures, and the representation complexity of a DNN. These metrics provide new perspectives to diagnose DNNs for 3D point clouds. For example, unlike the image processing relying on color information, the processing of 3D point clouds usually exclusively uses 3D structural information for classification. Therefore, a well-trained DNN for 3D point cloud processing is supposed to just use scale information and 3D structures for inference, and be robust to the rotation and translation.
Regional sensitivities. We first propose six metrics to disentangle the overall model vulnerability into the regional rotation sensitivity, the regional translation sensitivity, the regional scale sensitivity, and three types of regional structure sensitivity (sensitivity to edges, surfaces, and masses), so that we can use such sensitivity metrics to evaluate the representation quality of a DNN (as Fig. 1 (a-b) shows). Each sensitivity metric (let us take the rotation sensitivity for example) is defined as the vulnerability of the regional attribution when we rotate the 3D point cloud with different angles. The regional attribution is computed as the Shapley value, which is widely used as a unique unbiased estimation of an input variable’s attribution [11, 20, 31, 37] from the perspective of game theory. ∗This work was done when Wen Shen was an intern at Shanghai Jiao Tong University. †Quanshi Zhang is the corresponding author. This study was done under the supervision of Dr. Quanshi Zhang. He is with the John Hopcroft Center and the MoE Key Lab of Artificial Intelligence, AI Institute, at the Shanghai Jiao Tong University, China.
35th Conference on Neural Information Processing Systems (NeurIPS 2021).
high low sensitivity sensitivity high rotation sensitivity low translation sensitivity scale sensitivity sensitivity to edges sensitivity to surfaces sensitivity to masses (a)
high
low
sensitivity
sensitivity
Based on the sensitivity metrics, we have discovered the following new insights, which have exposed problems with classic DNNs.
• Insight 1. Rotation robustness is the Achilles’ heel of most DNNs for point cloud processing. The rotation sensitivities of PointNet [26], PointNet++ [28], DGCNN [43], the non-dynamic version of DGCNN (referred to as GCNN in this paper) [43], and PointConv [45] are much higher than other sensitivities (see Fig. 1 (a)).
• Insight 2. It is usually difficult for DNNs to extract rotation-robust features from 3D points at edges and corners. These points are usually vulnerable to rotation-based attacks.
• Insight 3. What’s worse, DNNs usually cannot ignore features of such points at edges and corners. Instead, such rotation-sensitive points usually have large regional attributions.
• Insight 4. PointNet fails to model local 3D structures, because convolutional operations in PointNet encode each point independently. Thus, PointNet has low sensitivity to local 3D structures.
Spatial smoothness. Beyond the sensitivity metrics, we also evaluate the spatial smoothness of knowledge representations of DNNs. We define the spatial smoothness of knowledge representations as the similarity among the neighboring regions’ attributions to network output. Most widely-used benchmark datasets for point cloud classification [46, 49] only contain objects with simple 3D structures (e.g. a bed mainly composed of a few surfaces in Fig. 1 (c)), in which adjacent regions usually have similar 3D structures. Therefore, most adjacent regions are supposed to have similar attributions to the network output. We also find that the adversarial training increases the spatial smoothness of knowledge representations.
Representation complexity. Besides regional sensitivities, we further extend the metric of multiorder interaction [53] to evaluate the representation complexity of a DNN, i.e. the maximum complexity of 3D structures that can be encoded in a DNN. The interaction of the m-th order measures the additional benefits brought by collaboration between two point cloud regions i, j under the contexts of other m regions, where m reflects the contextual complexity of the collaboration between regions i and j. High-order interactions usually represent global and complex 3D structures, corresponding to the collaboration between point cloud regions i, j and massive other regions. Loworder interactions usually describe local and simple 3D structures, without being influenced by many contextual regions (see Fig. 1 (d)). Based on such interactions, we have discovered the following new insights.
• Insight 5. Most DNNs fail to encode high-order interactions. I.e. most DNNs mainly focus on local structures and do not extract many global and complex structures from normal samples.
• Insight 6. Rotation-sensitive regions usually activate abnormal high-order interactions (global and complex structures). The very limited global structures modeled by the DNN are usually over-fitted to training samples. Thus, when the sample is rotated, the activated global and complex structures usually appear as abnormal patterns.
• Insight 7. Besides, in terms of model robustness, adversarial training based on adversarial samples w.r.t. using rotations/translations for attack is a typical way to improve the robustness to rotation and translation. Thus, we use the proposed sensitivity metrics to explain the inner mechanism of how and why the adversarially trained DNN is robust to rotation and translation. We find that (1) the adversarial training shifts the attention from the global orientations and positions to the structural information, thereby boosting the sensitivity to local 3D structures. (2) The adversarial training usually increases both the quantity and the complexity of 3D structures (high-order interactions from normal samples) modeled in a DNN, which boosts the robustness to the rotation and translation.

heading: 2 Related work
text: Deep learning on 3D point clouds: Recently, a series of studies directly used DNNs to process 3D point clouds and have achieved promising performance in various 3D tasks [26, 51, 42, 13, 27, 47, 41, 18, 10, 32, 33]. PointNet [26] was the pioneer to use point-wise multi-layer perceptron to process point clouds and aggregate all individual point features into a global feature. To further extract information from local 3D structures, PointNet++ [28] recursively applied PointNet to capture hierarchical 3D structures. Likewise, Relation-Shape CNN [19] and PointCNN [18] also focused on hierarchical structures and improved the ability to extract contextual information. Due to the irregularity of 3D point clouds, some approaches considered the point set as a graph and further defined graph convolutions. DGCNN [43] dynamically constructed local graphs and conducted feature extraction via the EdgeConv operation. SPG [17] built a superpoint graph to process largescale point clouds. GACNet [40] introduced an attention mechanism in graph convolutional networks. KCNet [34] proposed kernel correlation and graph pooling to aggregate contextual information. In addition, many approaches have been proposed to apply convolution operators to the point clouds. PointConv [45] and KPConv [39] utilized nonlinear functions to construct convolution weights from the input 3D coordinates. InterpCNN [23] interpolated features of 3D points to the discrete kernel weights coordinates, so as to implement the discrete convolution operator to point clouds. Instead of designing new architectures, our study focuses on the analysis of representation quality of existing classic DNNs for point cloud processing.
Visualization or diagnosis of representations: It is intuitive to interpret DNNs by visualizing the feature representations encoded in intermediate layers of DNNs [52, 35, 50, 22, 7], and estimating the pixel-wise saliency/importance/attribution of input variables w.r.t the network output [29, 15, 8, 57, 30, 5, 56]. Similarly, in the 3D domain, PointNet [26] visualized the subset of 3D points (namely the critical subset) that directly affected the network output. Additionally, Zheng et al. [55] specified the critical subset via building a gradient-based saliency map. In comparison, our study proposes to evaluate the representation quality of different point cloud regions.
Quantitative evaluation of representations: The quantitative evaluation of the representations of DNNs provides a new perspective for explanations. The Shapley value [31] estimates the attribution distribution over all players in a game and has been applied to quantitative evaluation of the representations of DNNs [1, 9, 20]. Based on Shapley values, some studies have investigated the interaction between input variables of a DNN [11, 21, 38, 53]. In comparison, our study aims to illustrate distinctive properties of 3D point cloud processing.

heading: 3 Analyzing feature representations of DNNs for 3D point cloud processing
text: We propose six metrics to measure regional sensitivities to rotation, translation, scale, and three types of local 3D structures, respectively, so as to exhibit the representation property of each specific point cloud region. Beyond these metrics, we can further analyze the representation quality from the following two perspectives, i.e. the spatial smoothness of knowledge representations, and the contextual complexity of 3D structures encoded by a DNN.
Preliminaries: quantifying the regional attribution using Shapley values. The Shapley value was originally introduced in game theory [31]. Considering a game with multiple players, each player aims to pursue a high award for victory. The Shapley value is widely considered as a unique unbiased approach that allocates the total reward to each player fairly, which satisfies axioms of linearity, nullity, symmetry, and efficiency [44] as the theoretical foundation. Please see our supplementary materials for details.
Given a point cloud with n regions1, N = {1, 2, · · · , n}, the Shapley value can be used to measure the attribution of each input region. We can consider the prediction process of a DNN as a game v, and each region in a point cloud as a player i. Let 2N def= {S | S ⊆ N} denote all the possible subsets of N . Let xS denote the point cloud only containing regions in S, in which regions in N\S are removed. For a DNN learned for multi-category classification, we use v(S) to denote the network output given the input xS . v(S) is calculated as log p1−p , where p = p(y = y
truth | xS) denotes the probability of the ground-truth category given xS . Note that the number of input points for each DNN is a fixed value. Therefore, in order to make the DNN successfully handle the point cloud xS without regions in N\S, we reset coordinates of points in regions of N\S to the center of the entire point cloud to remove the information of these points, instead of simply deleting these points [55]. Note that the reason for not simply deleting these points is that the number of input points must be greater than the number of points required for specific layerwise operations of DNNs used in this paper. Specifically, the sampling operation at the first layer of PointNet++ and PointConv requires the minimum number of input points to be greater than 512 points. The k-NN-based grouping [43] operation of DGCNN and GCNN requires the number of input points to be greater than 20 points. However, the computation of Shapley values requires to make inference on a point cloud fragment with a single region i (i.e., x{i}), which may have less than 20 points. Therefore, for such DNNs, we can not simply delete 3D points from the point cloud, in order to allow DNNs to work normally. To this end, for a fair comparison, we choose the alternative way, i.e., placing all points, which are supposed to be masked, to the center of the point cloud. In this way, the numerical attribution of the i-th region to the overall prediction score is estimated by the Shapley value φ(i), as follows.
φ(i) = ∑
S⊆N\{i} |S|!(n− |S| − 1)! n! (v(S ∪ {i})− v(S)), (1)
The computation of Equation (1) is NP-hard. Therefore, we use the sampling-based method in [4] to approximate φ(i).

heading: 3.1 Quantifying the representation sensitivity of a DNN
text: To analyze the quality of knowledge representations of a DNN, we define different types of sensitivities using the above regional attribution φ(i). Specifically, we measure six types of sensitivities, including the rotation sensitivity, translation sensitivity, scale sensitivity, and three types of sensitivity to local 3D structures (edge-like structures, surface-like structures, and mass-like structures). Given an input point cloud x, the Shapley value φ(i) measures the attribution of region i to the network output. The rotation/translation/scale/local-structure sensitivity of this region is quantified as the range of changes of this region’s attribution φ(i) among all potential transformations {T} of the rotation/translation/scale/local 3D structure, as follows.
∀ i ∈ N = {1, 2, · · · , n}, ai(x) = 1
Z (max T φx′=T (x)(i)−min T φx′=T (x)(i)), (2) where Z = ET [ ∑ i∈N |φx′=T (x)(i)|] is computed for normalization. Thus, the average sensitivity to all potential transformations {T} among all input point clouds x ∈ X is formulated as follows.
sensitivity = Ex∈X [ Ei∈N [ai(x)] ] (3)
In implementation, six types of sensitivities are computed as follows.
Rotation sensitivity quantifies the vulnerability of the inference caused by the rotation of 3D point clouds. Given an input point cloud x, we enumerate all rotations θ = [θ1, θ2, θ3]> from the range of [−π4 , π 4 ], i.e. given θ, we sequentially rotate the point cloud around the three axes of the 3D coordinates system, thereby obtaining a set of new point clouds {x′ = Trotation(x|θ)}. Translation sensitivity quantifies the vulnerability of the inference caused by the translation of 3D point clouds. Given an input point cloud x, we enumerate all translations ∆x = [∆x1,∆x2,∆x3]> from the range of [−0.5, 0.5] along the three axes of the 3D coordinates system, thereby obtaining a set of new point clouds {x′ = Ttranslation(x) = x+ ∆x}.
1There are many ways to segment a 3D point cloud to regions. In our experiments, we first used the farthest point sampling [26] to select n points from each point cloud as centers of n regions. Then, we partitioned the point cloud to n regions by assigning each remaining point to the nearest center point among the selected n center points.
(b) Ill stration of the spatial smoothness.
high attrib tion
lo attrib tion
not smooth
smooth
Scale sensitivity quantifies the vulnerability of the inference caused by the scale of 3D point clouds. Given an input point cloud x, we enumerate all scales α from the range of [0.5, 2], thereby obtaining a set of new point clouds {x′ = Tscale(x) = αx}. Then, we focus on other three sensitivity metrics for three types of local 3D structures, i.e. sensitivity to linearity (edge-like structures), sensitivity to planarity (surface-like structures), and sensitivity to scattering (mass-like structures). According to [12, 6], the significance of a point cloud to be edge-like structures/surface-like structures/mass-like structures is defined as follows.
linearity = λ1 − λ2 λ1 ; planarity = λ2 − λ3 λ1 ; scattering = λ3 λ1 ; (4)
where λ1 ≥ λ2 ≥ λ3 denote three eigenvalues of the covariance matrix of a region’s points. Without loss of generality, we take the linearity for example to introduce the way to enumerate a region’s all values of linearity. Given a region, we use the gradient ascent/descent method to modify the 3D coordinates p ∈ R3 of each point in this region, so as to increase/decrease the linearity, i.e. pnew = p + η ∂linearity∂p . As Fig. 2 (a) shows, we enumerate all regions’ linearity at the same time. The enumeration is implemented under the following two constraints. (1) Each value of λ1 needs to be within the range of λ1 ± γ, so as the λ2 and λ3. (2) ‖pfinal − pori‖ ≤ d, where ‖·‖ denotes the L2-norm. In this way, we obtain a set of point clouds with different degrees of local edge-like structures. In real implementation, we set η = 0.001, γ = 0.003, and d = 0.03.

heading: 3.2 Quantifying the spatial smoothness of knowledge representations
text: Besides the analysis of representation sensitivity, we also evaluate the spatial smoothness of feature representations encoded by a DNN. Because most benchmark 3D datasets [46, 49] only contain objects with simple 3D structures (see Fig. 1 (c)), except for special regions (e.g. edges), most adjacent regions of such simple objects usually have similar local 3D structures (e.g. surfaces). Therefore, most adjacent regions are supposed to have similar regional attributions, i.e. high smoothness. In this way, high spatial smoothness indicates reliable feature representations. We quantify the smoothness of the regional attribution between neighboring regions, as follows.
non-smoothness = Ex∈XETEiEj∈N (i) [ |φx′(i)− φx′(j)|
Zsmooth
∣∣∣ x′=T (x) ] (5)
whereN (i) denotes a set of nearest point cloud regions (determined by the ball-query search [28]) of region i; Zsmooth = ET [|vx′(N)− vx′(∅)|x′=T (x)] is computed for normalization; vx′(N) denotes the network output given the entire point cloud x′, and vx′(∅) denotes the network output when we mask all regions in x′ by setting all points to the center coordinates of the point cloud.

heading: 3.3 Quantifying the interaction complexity of a DNN
text: In addition, we also quantify the representation complexity of a DNN, i.e. the complexity of 3D structures encoded by a DNN. To this end, the 3D structures encoded by a DNN is represented using the interaction between different 3D point cloud regions. Here, input regions of a DNN do not work individually, but collaborate with each other to construct a specific 3D structure for inference. Zhang et al. [53] defined the multi-order interactions between two input variables. Given two input regions i and j, the interaction of the m-th order measures the additional attribution brought by collaborations
between i and j under the context of m regions. I(m)(i, j) = ES⊆N\{i,j},|S|=m [ v(S ∪ {i, j})− v(S ∪ {i})− v(S ∪ {j}) + v(S) ] , (6)
I(m)(i, j) > 0 indicates that the presence of region j increases the attribution of region i, φ(i). I(m)(i, j) < 0 indicates that the presence of region j decreases the value of φ(i). I(m)(i, j) ≈ 0 indicates that region j and region i are almost independent of each other. Please see our supplemental materials for details about the meaning of multi-order interaction and the linearity, nullity, commutativity, symmetry, and efficiency axioms of the multi-order interaction.
Here, we could consider the order m as the number of contextual regions involved in the computation of interactions between region i and region j. For example, as Fig. 2 (b) shows, regions i and j (in red), and other m regions (in blue) work together to construct a 3D structure to classify the earphone.
High-order interactions measure the effects of global collaborations among massive regions, i.e. representing complex and large-scale 3D structures. Low-order interactions measure the effects of collaborations between a few regions, i.e. usually representing simple and small-scale 3D structures. Then, we use the following metric to quantify the average strength of the m-th order interactions as the significance of the m-order complex 3D structures.
I(m) = Ex∈X [∣∣Ei,j [I(m)x (i, j)]∣∣]. (7)
If the I(m) of a low order is significantly larger than that of a high order, then the representation complexity of the DNN is limited to representing simple and local 3D structures.

heading: 4 Comparative studies
text: In this section, we conducted comparative studies to analyze properties of different point cloud regions of different DNNs. Ideally, a well-trained DNN for 3D point cloud processing was supposed to be robust to the rotation and translation, and the DNN was supposed to mainly use the scale and local 3D structures for inference. Besides, considering the 3D structures of objects in benchmark 3D datasets [46, 49] were usually simple, most adjacent regions in an object had continuous and similar 3D structures. Therefore, a well-trained DNN was supposed to have similar regional attributions among these neighboring regions. We also analyzed how complex is the 3D structure that can be encoded by a classic DNN.
We used our method to analyze five classic DNNs for 3D point cloud processing, including the PointNet [26], the PointNet++ [28], the DGCNN [43], the non-dynamic version of DGCNN (i.e. GCNN), and the PointConv [45]. All DNNs were learned based on the ModelNet10 dataset [46] and the ShapeNet part2 dataset [49]. We followed [26] to only use 1024 points of each point cloud to train all DNNs. Each point cloud was partitioned to n = 32 regions1 for the computation of all metrics.
Note that all DNNs used in our paper were well-trained. The testing accuracy was shown in Table 1.We also reported the testing accuracy of other DNNs trained on the ModelNet10 dataset, including RotationNet [14], 3DmFV-Net [3], KCNet [34], and GIFT [2]. These four DNNs obtained the 1st, 4th, 7th, and 10th testing accuracy according the ModelNet10 Benchmark Leaderboard3.
In particular, the previous study [54] has discovered that people could use rotations to attack DNNs for 3D point cloud processing. Therefore, we used the adversarial training to learn a GCNN4 w.r.t. the attacks based on rotations and translations of point clouds, so as to improve the GCNN’s robustness to the rotation and translation. We extended the method in [16] to generate such adversarial examples to attack the GCNN. Please see our supplemental materials for details. The objective function
2We only used part of the ShapeNet part dataset due to the time limitation. Please see our supplemental materials for details.
3Data from https://modelnet.cs.princeton.edu/. 4In tables, adv-GCNN denoted the adversarially trained GCNN.
was minw Ex [ maxT Loss(x′ = T (x), ytruth;w) ] , where w denoted the parameter of the GCNN. Comparative studies have revealed the following effects of the adversarial training (using rotations and translations for attack, instead of using perturbations) on knowledge representations. The adversarial training shifted the attention of DNN from orientations and positions to structural information (including global structures in normal samples, see Fig. 5), thereby increasing the sensitivity to 3D structures (see Table 2 and Fig. 3 (a)). The adversarial training also decreased the correlation between the regional sensitivity and the regional attribution (see Table 3), and increased the spatial smoothness of knowledge representations (see Table 4).
Comparative study 1, explaining the regional sensitivity of DNNs. Table 2 shows six types of regional sensitivities of six DNNs learned on the ModelNet10 dataset and the ShapeNet part dataset. In terms of translation sensitivity, PointNet was relatively sensitive to the translation, because PointNet extracted features from global coordinates of points, thereby being sensitive to the translation. In contrast, PointNet++ and PointConv were robust to translation, because these two DNNs encoded relative coordinates. Particularly, PointConv did not use global coordinates during the inference process, thereby yielding the lowest translation sensitivity (see the point cloud with the darkest blue in the second row of Fig. 3 (a)). In terms of scale sensitivity, PointNet++ was relatively sensitive to the scale, because PointNet++ encoded features of neighborhood with fixed scales. Besides, because adversarial training forced the GCNN to remove attention from rotation-sensitive features and translation-sensitive features, the adversarially trained GCNN paid more attention to structural information. Therefore, compared with the original GCNN, the adversarially trained GCNN had higher sensitivity to local 3D structures. Furthermore, we also obtained the following conclusions.
• Rotation robustness was the Achilles’ heel of classic DNNs for 3D point cloud processing. As Table 2 and Fig. 3 (a) show, all DNNs were sensitive to rotations except for the adversarially trained GCNN.
• PointNet failed to encode local 3D structures. In terms of sensitivity to local 3D structures, PointNet was the least sensitive DNN (see Table 2). It was because convolution operations of the PointNet encoded the information of each point independently, i.e. the PointNet did not encode the information of neighboring points/regions. This conclusion is also verified by the phenomenon that PointNet had darker blue point clouds than other DNNs (see the last three rows of Fig. 3 (a)).
• Most DNNs usually failed to extract rotation-robust features from 3D points at edges and corners. Given a point cloud x, we selected the most rotation-sensitive region i∗ (shown as black boxes in Fig. 3 (b)). We rotated the point cloud x to the orientations that maximized and minimized the
边缘结构敏感度
平⾯结构敏感度
团状结构敏感度
attribution of region i∗, i.e. θ1 = arg maxθ φx′(i∗) and θ2 = arg minθ φx′(i∗). Fig. 3 (b) visualizes regional attributions of point clouds rotated by θ1 and θ2. We found that rotation-sensitive regions were usually distributed on the edges and corners, which verified our conclusion.
• Most DNNs usually could not ignore features of rotation-sensitive points at edges and corners. Table 3 shows that the Pearson correlation coefficient [25] between each region’s average strength of attribution over different rotations (i.e. Eθ[|φx′(i)|], s.t. x′ = Trotation(x|θ)) and this region’s rotation sensitivity ai(x) is much larger than Pearson correlation coefficients between the attribution and other sensitivities. This means that rotation-sensitive regions (which were usually distributed on the edges and corners) usually had large regional attributions, which verified our conclusion. Besides, Table 3 also shows that the adversarial training reduced the correlation between the regional sensitivity and the regional attribution. Fig. 4 (a) visualizes the positive correlation between the regional rotation sensitivity and the regional attribution of the PointNet.
Comparative study 2, explaining the spatial smoothness of knowledge representations. Table 4 shows the non-smoothness of knowledge representations encoded by different DNNs. Note that the adversarially trained GCNN was significantly biased to the knife category. In other words, the
adversarially trained GCNN classified the empty input as the knife category with a high confidence (please see our supplementary materials for details). Therefore, to enable fair comparisons, we also reported the spatial non-smoothness of the adversarially trained GCNN on all other categories except the knife. We discovered that without considering the biased knife category, adversarial training increased the spatial smoothness of knowledge representations. Fig 4 (b) also verified this conclusion.
Comparative study 3, explaining the interaction complexity of DNNs. Fig. 5 shows multi-order interactions of different DNNs. From this figure, we discovered the following new insights.
• Most DNNs failed to encode high-order interactions (i.e. global and large-scale 3D structures). As Fig. 5 (a) shows, no matter given normal samples or adversarial samples5, classic DNNs encoded extremely low-order interactions. This indicated that most DNNs did not extract complex and large-scale 3D structures from normal samples.
• The adversarial training (using rotations and translations for attack, instead of using perturbations) increased the effects of extremely high-order interactions. As Fig. 5 (a) shows, the adversarially trained GCNN encoded extremely low-order interactions and extremely high-order interactions (i.e. the global structures of objects) from normal samples. Because the GCNN was forced to sophisticatedly select relatively complex rotation-robust features among all potential features.
• Rotation sensitivity of regions with out-of-distribution high-order interactions (i.e. abnormal complex and large-scale 3D structures) were larger than the average sensitivity over all regions. In fact, there are two types of high-order interactions. Unlike the above two insights describing whether or not a DNN encoded high-order interactions on normal samples, this insight mainly focused on abnormal out-of-distribution high-order interactions in a DNN. When a point cloud was attacked by adversarial rotations5, the adversarial rotation5 would generate out-of-distribution high-order interactions (i.e. abnormal complex and large-scale 3D structures) to attack the DNN. We measured interactions between the most rotation-sensitive region i∗ and its neighbors j ∈ N (i∗) among all input point clouds, i.e. Ex∈X [ |Ej∈N (i∗)[I (m) x (i∗, j)]| ] . As Fig. 5 (b) shows, rotation-sensitive regions usually encoded relatively more high-order interactions than the overall interaction distribtuion over all regions. This indicates that compared to most other regions, rotation-sensitive regions paid more attention to high-order interactions, although low-order interactions of rotation-sensitive regions increased too.
Broad applicability of the proposed metrics. We also conducted experiments on an AutoEncoder for 3D point cloud reconstruction task and rotation-invariant DNNs for 3D point cloud classification, in order to further demonstrate the broad applicability of the proposed metrics.
For the AutoEncoder for 3D point cloud reconstruction task, we used all layers before the fullyconnected layers in PointNet [26] as the encoder and used the decoder in the FoldingNet [48] as the decoder. We took v(S) = (z(N)−z(∅))
>(z(S)−z(∅)) ||z(N)−z(∅)||2 as the reward score in Equation (1), where
z(S) denotes the output vector of the encoder given the input point cloud consisting of regions in S. This reward score measures the utility of the encoder output z(S) given regions in S along the direction of z(N) − z(∅). The AutoEncoder was learned based on the ModelNet10 dataset. We measured the rotation sensitivity (0.099±0.043), the translation sensitivity (0.152±0.086), and the scale sensitivity (0.045±0.042) of the AutoEncoder. Compared with results in Table 2, we found that (1) the AutoEncoder for reconstruction was also sensitive to rotation; (2) the AutoEncoder for 3D
5Here, we used rotations for attack, instead of using perturbations.
point cloud reconstruction was more sensitive to translation than the DNN for classification; (3) the AutoEncoder for reconstruction was sensitive to the scale change.
Besides, we followed [32] to revise PointNet++ and DGCNN to be rotation-invariant DNNs. We measured the rotation sensitivity of the rotation-invariant PointNet++ (0.002±0.001) and DGCNN (0.010±0.004). Compared with results in Table 2, we found that rotation-invariant DNNs were much more robust to rotations than traditional DNNs (even the adversarially-trained DNN).
Effects of data augmentation on sensitivities. We conducted comparative studies to explore the effects of data augmentation on sensitivities, including translation augmentation, scale augmentation, rotation augmentation around the y-axis, and rotation augmentation around a random axis. We conducted experiments on PointNet and DGCNN. To explore the effects of each of the above augmentation on sensitivities, we learned two versions of each network architecture, i.e. one network with the specific augmentation and the other network without the specific augmentation. Please see our supplementary materials for more details about different versions of each network architecture. All DNNs were learned based on the ModelNet10 dataset. Table 5 shows that rotation/translation/scale augmentation decreased the rotation/translation/scale sensitivity of a DNN.
Besides, we compared the effects of rotation augmentation around a random axis and the effects of adversarial training on the rotation sensitivity. We conducted experiments on DGCNN and GCNN. We trained two versions of DGCNN and GCNN, one with rotation augmentation around a random axis (DGCNN, 0.052 ±0.021; GCNN, 0.048±0.024) and one with the adversarial training using rotations for attack (DGCNN, 0.036±0.015; GCNN, 0.028±0.016). We found that compared with the rotation augmentation, the adversarial training w.r.t. rotation-based attacks had a greater impact on the rotation sensitivity.

heading: 5 Conclusion
text: In this paper, we have measured six types of regional sensitivities, the spatial smoothness, and the contextual complexity of feature representations encoded by DNNs. Comparative studies have discovered several new insights into classic DNNs for 3D point cloud processing. We have found that most DNNs were extremely sensitive to the rotation. Rotation-sensitive regions were usually distributed on the edges and corners of objects, had large attributions to network output, and usually paid more attention to abnormal large-scale structures. In addition, most DNNs failed to extract complex and large-scale 3D structures from normal samples, while the adversarial training could encourage the DNN to extract global structures from normal samples.
Acknowledgments and Disclosure of Funding
This work is partially supported by the National Nature Science Foundation of China (No. 61906120, U19B2043), Shanghai Natural Science Fundation (21JC1403800,21ZR1434600), Shanghai Municipal Science and Technology Major Project (2021SHZDZX0102). This work is also partially supported by Huawei Technologies Inc.

Review summaries:
This paper measures the sensitivity of various point cloud classification networks (like PointNet, PointNet++, DGCNN) to rotation, translation, scale, and local 3D structure. It also proposes ways to measure the 3D smoothness of a network. Based on these analyses, the work provides various insights. One of the main insights being that the DNNs don’t handle rotation very well. Another insight is that training with adversarial perturbations helps.

At a higher level, this paper is proposing several ways to measure the sensitivity and smoothness of representations learnt by 3D point cloud networks. This paper presents several ways to measure the sensitivity of point cloud networks to rotation, scale, translation and local structure. The Paper also proposes different metrics to evaluate the smoothness of encoded local 3d structures and the complexity of representations learnt by the network. The analysis is done using several point-based neural networks trained on the ModelNet40 classification task and shapenet part segmentation task.

This paper introduces metrics that reflect the representation quality, and properties, acquired by a deep-net that is applied in 3D Point-Cloud (PC) processing tasks. Concretely, the authors introduce six novel metrics that in aggregate capture different types of regional sensitivities e.g., the sensitivity of the DNN in rotations on specific regions/rotation-angles of the input PC. They further explore the extent that modern architectures in standard benchmarks acquire “spatial smoothness” --  do nearby points have ‘similar’ effects/importance for the end task? And finally, they explore the acquired representation complexity of the DNN by adapting a 'multi-order interaction' metric to PC data.    These metrics are being applied on a well-selected subset of modern PC-DNN architectures such PointNet++ or DGCNN when these are trained either for PC-object classification, or part-based object-segmentation. The analysis of the metrics in these tasks/architectures highlights a number of interesting points that oftentimes are very simple/intuitive: e.g., PointNet fails to encode local 3D structures (e.g., compared to PointNet++) — or includes results that are more subtle, but still highly intuitive & rich: e.g., adversarially training a PC-DNN with rotation/translation attacks; will force it to learn more global PC structures compared to standard training; which is intuitive as the local structures, per the new metrics, are those that on average are more susceptible to rotations.

This paper studies how to understand the quality of knowledge representations encoded in point cloud DNNs. Based upon the Shapley value, the paper presents six metrics to evaluate the model vulnerability toward different types of input variations. In addition, it presents an analysis regarding the smoothness and multi-order interactions of these attributions, pointing out the restrictions of current point DNNs. The paper makes interesting observations and draws useful conclusions, supported by a range of experimental evaluations.