Interpreting Representation Quality of DNNs for 3D Point Cloud Processing
Authors: Wen Shen
Qihan Ren
Dongrui Liu
Quanshi Zhang
Venue: NeurIPS 2021 Poster (2021)
Keywords: 3D point cloud processing
Explainable AI

Abstract: In this paper, we evaluate the quality of knowledge representations encoded in deep neural networks (DNNs) for 3D point cloud processing. We propose a method to disentangle the overall model vulnerability into the sensitivity to the rotation, the translation, the scale, and local 3D structures. Besides, we also propose metrics to evaluate the spatial smoothness of encoding 3D structures, and the representation complexity of the DNN. Based on such analysis, experiments expose representation problems with classic DNNs, and explain the utility of the adversarial training. The code will be released when this paper is accepted.

Interpreting Representation Quality of DNNs for 3D Point Cloud Processing
Wen Shen DM
Qihan Ren
Dongrui Liu
Quanshi Zhang
"Shanghai Jiao Tong University
'Tongii University
Abstract
In this paper, we evaluate the quality of knowledge representations encoded in deep neural networks (DNNs) for 3D point cloud processing. We propose a method to disentangle the overall model vulnerability into the sensitivity the rotation, the translation, the scale, and local 3D structures. Besides, we also propose metrics to evaluate the spatial smoothness of encoding 3D structures, and the representation complexity of the DNN. Based on such analysis, experiments expose representation problems with classic DNNs, and explain the utility of the adversarial training. The code will be released when this paper is accepted.
Introduction
Deep neural networks (DNNss have exhibited superior performance in various tasks, but the black- box nature of DNNs hampers the analysis of knowledge representations. Previous studies on explainable AI mainly focused on the following two directions. The first to explain the knowledge
encoded in DNN, e.g. visualizing patterns encoded by DNN 124 [36] [5) and estimating the salicncy/importanccaatrrbiu
of input variables w.r. the network output (20 30 57]. The second
is to evaluate the representation power of DNN, e.g. the generalization and robustness of feature representations.
This paper focuses on the intersection of the above two directions, i.e. analyzing the quality of knowledge representations DNNs for 3D point cloud processing. Specifically, we aim to design metrics to illustrate properties of feature representations for different point cloud regions, including various types of regional sensitivities, the spatial smoothness of encoding 3D structures, and the representation complexity of a DNN. These metrics provide new perspectives to diagnose DNNs for 3D point clouds. For example, unlike the image processing relying on color information, the processing of 3D point clouds usually exclusively uses 3D structural information for classification. Therefore, a well-trained DNN for 3D point cloud processing is supposed to just use scale information and 3D structures for inference, and be robust to the rotation and translation.
Regional sensitivities. We first propose six metries to disentangle the overall model vulnerability into the regional rotation sensitivity, the regional translation sensitivity, the regional scale sensitivity, and three types of regional structure sensitivity (sensitivity to edges, surfaces, and masses), so that we can use such sensitivity metrics to evaluate the representation quality of a DNN (as Fig, (a-b) shows). Each sensitivity metric (let us take the rotation sensitivity for example) is defined as the vulnerability of the regional attribution when we rotate the 3D point cloud with different angles. The regional attribution computed as the Shapley value, which is widely used as a unique unbiased estimation of an input variable's 's attribution
20 31 37
from the perspective of game theory.
"This work was done when Wen Shen QQanssi Zhang is the corresponding author. This study was done under the supervision of Dr. Quanshi Zhang. He is with the John Hopcroft Center and the MoE Key Lab of Artificial Intelligence, Al Institute, at the Shanghai Jiao Tong University, China.
intern Shanghai Jiao Tong University.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Most sensitive cotation
rolation sctulivity
inasllaton scmiliviy
sasile sensiivity
Thsse
adjacent zeglons supposed
similar attrbuttoons
rotation
similar colom)
0,00
(c) The smootheess of regiooaal attributions.
scale
region with high mustiion scasllivity
edges
munorcycle
sensitivity edges censitivity sensitivity
wheal
surbbces
masses
Ta34
law-arder interaction:
sensirivity
200
collabontiions samungg segions G.)) high-arder interaction:
2.04
and few other
collabocaiiess
(a) Comparison six types of sensitivities of Pointett++
and masivee other regions
Visualization regional sensitivitiis PointNett++.
Illustration the
Figure I: (a) Comparison of regional sensitivities of PointNet++ 28 (b) Visualization of regional sensitivities of PointNet++ [28). where the heatmap is normalized in each sample. These colorbars are shown in log-scale, (c) Illustration of the regional smoothness. (d) Illustration of the representation complexity.
Based on the sensitivity metrics, we have discovered the following new insights, which have exposed problems with classic DNNs.
Insight 1. Rotation robustness is the Achilles' heel of most DNNs for point cloud processing. The
rotation sensitivities of PointNet (20], PointNet++ (28), DGCNN 43 the non-dynamic version of
DGCNN (referred to as GCNN in this paper) and PointConv 45 sensitivities (see Fig. (a)).
are much higher than other
Insight 2. It is usually difficult for DNNs extract rotation-robust features from 3D points edges and corners. These points are usually vulnerable to rotation-based attacks.
Insight 3. What's worse, DNNs usually cannot ignore features of such points at edges and corners. Instead, such setstive- points usually have large regional attributions.
Insight 4. PointNet fails to model local 3D structures, because convolutional operations in PointNet encode each point independently. Thus, PointNet has low sensitivity local 3D structures.
Spatial smoothness. Beyond the sensitivity metrics, we also evaluate the spatial smoothness of knowledge representations of DNNs. We define the spatial smoothness of knowledge representations as the similarity among the neighboring regions' attributions to network output. Most widely-used benchmark datasets for point cloud classification 446 49] only contain objects with simple 3D structures (e.g. a bed mainly composed of a few surfaces in Fig. [ (c)), in which adjacent regions usually have similar 3D structures. Therefore, most adjacent regions are supposed have similar attributions to the network output. We also find that the adversarial training increases the spatial smoothness of knowledge representations.
Representation complexity. Besides regional sensitivities, we further extend the metric of multi- order interaction [33 to evaluate the representation complexity of DNN, i.e. the maximum complexity of 3D structures that can be encoded in a DÑN. The interaction of the m-th order measures the additional benefits brought by collaboration between two point cloud regions i. under the contexts of other m regions, where m reflects the contextual complexity of the collaboration between regions and j. High-order interactions usually represent global and complex 3D structures, corresponding to the collaboration between point cloud regions i, and massive other regions. Low- order interactions usually deseribe local and simple 3D structures, without being influenced by many contextual regions (see Fig. [ (d)). Based on such interactions, we have discovered the following new insights.
Insight Most DNNs fail to encode high-order interactions. Lee most DNNs mainly focus on local structures and do not extract many global and complex structures from normal samples.
Insight Senation-eenstive regions usually activate abnormal high-order interactions (global and complex structures). The very limited global structures modeled by the DNN are usually over-fitted to training samples. Thus, when the sample is rotated, the activated global and complex structures usually appear abnormal patterns.

---

Insight 7. Besides, in terms of model robustness, adversarial training based on adversarial samples w.r.t. using rotations/raanllations for attack is a typical way to improve the robustness to rotation and translation. Thus, we use the proposed sensitivity metrics to explain the inner mechanism of how and why the adversarially trained DNN robust to rotation and translation. We find that (1) the adversarial training shifts the attention from the global orientations and positions to the structural information, thereby boosting the sensitivity to local 3D structures. (2) The adversarial training usually increases both the quantity and the complexity of 3D structures (high-order interactions from normal samples) modeled in a DNN, which boosts the robustness to the rotation and translation.
2 Related work
Deep learning 3D point clouds:
Recently, series of studies directly used INNNs process
3D point clouds and have achieved promising performance in various 3D tasks (26] 51 42 B 27
47
32 33 PointNet [26] was the pioneer to use point-wise multi-layer perceptron to process point clouds and aggregate all individual point features into a global feature. To further extract information from local 3D structures, PointNet++ (28) recursively applied PointNet to capture hierarchical 3D structures. Likewise, Relation-Shapp CNN [9]] and PointCNN [8] also focused on hierarchical structures and improved the ability to extract contextual information. Due to the irregularity of 3D point clouds, some approaches considered the point set as graph and further defined graph convolutions. DGCNN [43] dynamically constructed local graphs and conducted feature extraction via the EdgeConv operation. SPG [17] built a superpoint graph to process large- scale point clouds. GACNet (40 introduced an attention mechanism in graph convolutional networks. KCNet (34 proposed kernel correlation and graph pooling to aggregate contextual information. In addition, many approaches have been proposed to apply convolution operators to the point clouds. PointConv [4]] and KPConv 33 utilized nonlinear functions to construct convolution weights from the input 3D coordinates. InterpCNN 23 interpolated features of 3D points to the discrete kernel weights coordinates, so to implement the discrete convolution operator to point clouds. Instead of designing new architectures, our study focuses on the analysis of representation quality of existing classic DNNs for point cloud processing.
18
10,
Visualization or diagnosis of representations: It is intuitive to interpret DNNs by visualizing the feature representations encoded in intermediate layers of DNNs [22 35 50
22 77 and estimating
the pixel-wise salieney/imootanceattiibuu
of input variables w. the network output 29 15
57 30 5 56) Similarly, in the 3D domain, PointNet (26) visualized the subset of 3D points (namely
the critical subset) that directly affected the network output. Additionally, Zheng et al. (55) specified the critical subset via building tradient-based saliency map. In comparison, our study proposes to evaluate the representation quality of different point cloud regions.
Quantitative evaluation of representations: The quantitative evaluation of the representations of DNNs provides a new perspective for explanations. The Shapley value (31 estimates the attribu- tion distribution over all players in a game and has been applied to quantitative evaluation of the representations of DNNs I 20]. Based on Shapley values, some studies have investigated the In comparison, our study aims to
interaction between input variables of a DNN [1] 21 38 53
illustrate distinctive properties of 3D point cloud processing.
3 Analyzing feature representations of DNNs for 3D point cloud processing
We propose six metrics to measure regional sensitivities to rotation, translation, scale, and three types of local 3D structures, respectively, so as to exhibit the representation property of each specific point cloud region. Beyond these metries, we can further analyze the representation quality from the following two perspectives, i.e. the spatial smoothness of knowledge representations, and the contextual complexity of 3D structures encoded by DNN.
quantifying the regional attribution using Shapley values. The Shapley value was originally introduced in game theory [3]] Considering game with multiple players, each player aims to pursue a high award for victory. The Shapley value widely considered as unique unbiased approach that allocates the total reward to each player fairly, which satisfies axioms of linearity, nullity, symmetry, and efficiency (4 as the theoretical foundation. Please see our supplementary materials for details.
Preliminaries:
3

---

Given point cloud with n regions N = {1,2. ... nn. , the attribution of each input region. We can consider the prediction process of DNN as game
the Shapley value can be used to measure
v, and each region in point cloud player i. Let 2 def {S S S .) denote all the possible subsets of N. Let æ denote the point cloud only containing regions in S, in which regions in N removed. For DNN learned for multi-category classification,
use vsS) denote the network output given the input I.s. ((S) is calculated as log I-pp where p ply yrruth Is) denotes the probability of the ground-truth category given Is. Note that the number of input points for each DNN is a fixed value. Therefore, in order to make the DNN successfully handle the point cloud æg without regions in S, reset coordinates of points in regions of N the center of the entire point cloud to remove the information of these points, instead of simply deleting these points [55]. Note that the reason for not simply deleting these points is that the number of input points must be greater than the number of points required for specific layerwise operations of DNNs used in this paper. Specifically, the sampling operation at the first layer of PointNet++ and PointConv requires the minimum number of input points to be greater than 512 points. The k-NN-based grouping [43 operation of DGCNN and GCNN requires the number of input points be greater than 20 points. However, the computation of Shapley values requires to make inference on point cloud fragment with a single region i (i.e,, z (()).. which may have less than 20 points. Therefore, for such DNNs, we can not simply delete 3D points from the point cloud, in order to allow DNNs to work normally. To this end, for fair comparison, we choose the alternative way, i.e., placing all points, which are supposed to be masked, to the center of the point cloud. In this way, the numerical attribution of the i-th region to the overall prediction score is estimated by the Shapley value Φ(t), as follows.
|S| - 1)! n!
((1) Σ SCNV()
(v(Su v(S))
(1)
The computation of Equation approximate ((1).
| is NP-hard. Therefore, we use the sampling braseing-bbsedd based method in [4] to
3.1 Quantifying the representation sensitivity of DNN
To analyze the quality of knowledge representations of a DNN, we define different types of sensi- tivities using the above regional attribution @(t). Specifically, we measure six types of sensitivities, including the rotation sensitivity, translation sensitivity, scale sensitivity, and three types of sensitivity to local 3D structures (edge-like structures, surface-like structures, and mass-like structures). Given an input point cloud v, the Shapley value σ6²) measures the attribution of region i to the network output. The aaatiior/ranssaaio//sccllllllll sttruucue sensitivity of this region quantified as the range of changes of this region's 's attribution o(i) among all potential transformations {T} of the rotatioo/rranslaaiooscccll//o/o 3D structure, as follows.
ViEN 11,2, .n), ax(x)) (max )T (i) min (='=T(=)
(2)
where Z = 1TIEien ((== is computed for normalization. Thus, the average sensitivity to all potential transformations {T) among all input point clouds X is formulated as follows.
sensitivity = Exex [[EENN [a.]]]
In implementation, six types of sensitivities are computed as follows.
Rotation sensitivity quantifies the vulnerability of the inference caused by the rotation of 3D point clouds. Given an input point cloud Jr we enumerate all rotations 0 [0,, 02, 0g] from the range of - I1. i.e. given 0, we scquentially rotate the point cloud around the three axes of the 3D coordinates system, thereby obtaining a set of new point clouds fa' Trotation (x))))
Translation sensitivity quantifies the vulnerability of the inference caused by the translation of 3D point clouds. Given an input point cloud x, we enumerate all translations Ar [ Axy, Arg] from the range of [-0.5, 0.5] along the three axes of the 3D coordinates system, thereby obtaining a Ar).
set of new point clouds {a' Tiranslation (x)
'There are many ways to segment a 3D point cloud to regions. our experiments, we first used the farthest point sampling (66 to select n points from each point cloud as centers of n regions. Then, we partitioned the point cloud to regions by assigning each remaining point to the nearest center point among the selected center points.

---

regions f. and 21 regions increase the linearity
inerease the planurity
increase the scatiering
m m-
m m
regions f.j. and regions
anouuer
construct the local
construct the global 3D structure ohe carphome
perspective
structure car-cap
(4) Increasing all regions" aatpiit/pplaarttssssttttiieee
(b) Visualizing the internction of the m-th order.
Figure 2: (a) Visualization of increasing the linearity/lannriiyyssttttii of all regions. (b) Visualiza- tion of multi-order interactions.
Scale sensitivity quantifies the vulnerability of the inference caused by the scale of 3D point clouds. Given input point cloud we enumerate all scales from the range of [0.5, 2). thereby obtaining a set new point clouds {x' = (z) = =ar..
Then, we focus on other three sensitivity metries for three types of local 3D structures, i.e. sen- sitivity to linearity (edge-like structures), sensitivity planarity (surface-like strucfures), and sensitivity to scattering (mass-like structures). According to [2, 66. the significance of a point cloud be edge-like structure/suufface-ll like structures/mass-iike structures defined as follows.
A1 X2 A1
A2 A A1
Aa
linearity
planarity
scattering
A1
where d1 2 A 2 A denote three eigenvalues of the covariance matrix of a region's 's points.
Without loss of generality, we take the linearity for example to introduce the way to enumerate region's s all values of linearity. Given a region, we use the gradient ascent/descent method to modify the 3D coordinates p E R3 of each point in this region,
to increase/decrease the linearity, i.e. As Fig. 2 (a) shows, we enumerate all regions' linearity the same time. The enumeration implemented under the following two constraints. (1) Each value of X needs to be within the range of a + so as the X2 and λg. (2) ||p p I d, where ||-I| denotes the 1.2-norm. In this way, we obtain set of point clouds with different degrees of local edge-like structures. In real implementation, we set 0.001, 0.003, and d = 0.03.
p new
Alinearity
0p
3.2 Quantifying the spatial smoothness of knowledge representations
Besides the analysis of representation sensitivity. we also evaluate the spatial smoothness of feature representations encoded by DNN. Because most benchmark 3D datasets [46] [9 only contain objects with simple 3D structures (see Fig. (c)), except for special regions (e.g. edges), most adjacent regions of such simple objects usually have similar local 3D structures (e.g. surfaces). Therefore, most adjacent regions are supposed to have similar regional attributions, iee. high smoothness. In this way, high spatial smoothness indicates reliable feature representations. We quantify the smoothness of the regional attribution between neighboring regions, as follows.
non-smoothness = ExEx Er (1) oz (il
(5)
Zsmooth
TT=
where (i) denotes set of nearest point cloud regions (determined by the ball-query search (28) of Ugr [[))== is computed for normalization; Uar (N) denotes
region Zsmooth 15 [[rr
the network output given the entire point cloud x', and v.r vū)( denotes the network output when we
mask all regions in x' by setting all points to the center coordinates of the point cloud.
3.3 Quantifying the interaction complexity of a DNN
In addition, we also quantify the representation complexity of a DNN, i.e. the complexity of 3D structures encoded by DNN. To this end, the 3D structures encoded by a DNN is represented using the interaction between different 3D point cloud regions. Here, input regions of a DNN do not work individually, but collaborate with each other to construct specific 3D structure for inference. Zhang al. (53) defined the multi-order interactions between two input variables. Given two input regions and j. the interaction of the m-th order measures the additional attribution brought by collaborations
5

---

Table 1: Classification accuracy of different DNNs.
Model
PointNet PointNet++ PointConv DGCNN GCNN adv-GCNN' RotationNet [14 NDFVV-Net KCNet
GIFT
ModelNetIO
93.5
94.7
94.6
94.4
95.1
91.0
98.46
95.2
94,4
92.35
ShapeNot part
99.2
99.7
99.5
100.0
100.0
97.9
between and j under the context of m regions.
[mm) =
(() +(()),
(6)
(i.5) 0 indicates that the presence of region increases the attribution of region i, ((1). (i.5).
(i.5) < 0 indicates that the presence of region j decreases the value of 6(i). (i.j) 2 0
indicates that region j and region are almost independent of each other. Please see our supple- mental materials for details about the meaning of multi-ordes interaction and the linearity, nullity, commutativity, symmetry, and efficiency axioms of the multi-order interaction.
Here, we could consider the order as the number of contextual regions involved in the computation of interactions between region and region j. For example, as Fig. 2 (b) shows, regions and j (in red), and other m regions (in blue) work together to construct a 3D structure to classify the earphone.
High-order interactions measure the effects of global collaborations among massive regions, i.e. representing complex and large-scale 3D structures. Low-order interactions measure the effects of collaborations between few regions, i.e. usually representing simple and small-scale 3D structures. Then, we use the following metric to quantify the average strength of the m-th order interactions as the significance of the m-order complex 3D structures.
T(m) ExEx [| (t. ((.))]]
If the [(mm of a low order significantly larger than that of a high order, then the representation complexity of the DNN is limited to representing simple and local 3D structures.
4 Comparative studies
In this section, we conducted comparative studies to analyze properties of different point cloud regions of different DNNs. Ideally, well-trained DNN for 3D point cloud processing was supposed to be robust to the rotation and translation, and the DNN was supposed to mainly use the scale and local 3D structures for inference. Besides, considering the 3D structures of objects in benchmark 3D were usually simple, most adjacent regions in an object had continuous and similar 3D structures. Therefore, well-trained DNN was supposed to have similar regional attributions among these neighboring regions. We also analyzed how complex is the 3D structure that can be encoded by classic DNN.
datasets [46] [4]
We used our method to analyze five classic DNNs for 3D point cloud processing. including the PointNet (26), the PointNet++ 288, the DGCNN 43 the non-dynamic version of DGCNN (i.e. GCNN), and the PointConv 45). All DNNs were learned based on the ModelNet1 0 dataset
(46) and the ShapeNet parrdattsee [49] We followed (26 to only use 1024 points of each point cloud to train all DNNs. Each point cloud was partitioned to n = 32 regions' for the computation of all metrics.
Note that all DNNs used in our paper were well-trained. The testing accuracy was shown in Table [] We also reported the testing accuracy of other DNNs trained on the MedelNet10 dataset, including RotationNet [14, 3DmFV-Net 3. KCNet 34. and GIFT 2 These four DNNs obtained the 1st, 4th, 7th and 10th testing accuracy according the ModelNet10 Benchmark Leaderboard
In particular, the previous study [54] has discovered that people could use rotations to attack DNNs for 3D point cloud processing. Therefore, used the adversarial training learn GCNN w.r.f. the attacks based on rotations and translations of point clouds, so as to improve the GCNN's 's robustness to the rotation and translation. We extended the method in [16] to generate such adversarial examples to attack the GCNN. Please see our supplemental materials for details. The objective function
We only used part of the ShapeNet part dataset due to the time limitation. Please see our supplemental
materials for details. Data from https://modelnet.c es. wwwprriceton.cdeeo edu/. In tables. ady-GCNN denoted the adversarially trained GCNN.

---

Table 2: Average sensitivities over all regions among all samples.
Datasel
Model
rotation sensitivity
translation sensitivity
scale sensitivity
sensitivity to edges
sensitivity to surfaces
sensitivity
to masses
PointNet
0.159:10.070
0.110=0.053
0.02410:017
0.007=00007
0.010=0.009
0.009=0.009
PointNett++
0.171+0.064
0.004-00004
0.054=0.027
0.018=0011
0.026=0.016
0.029=0.019
PointConv
0.145=+0.000
2.3e-4=1.944
0.027=00.019
0.010=00007
0.015=00011
0.017=0.013
ModelNer1O
DGCNN
0.174:10.755
0.048=0.024
0.020:0.014
0.016=0.009
0.022=00011
0.023::00015
GCNN
0.174:+0007
0.050=0.026
0.020=00014
0.017=0001
0.022=0.014
0.023=0.015
adv-GCNN+
0.034:10.012
0.007
0.020
0.022:00014
0.027
0.029:
PointNet
0.107=00.05
0.071=0.032
0.023=0.020
0.005=00005
0.004=0.004
0.005=0.005
PointNct++
0.142:10.057
0.001=0.000
0.044100025
0.014=00009
0.014=0.009
0.016=0.011
ShapeNet part
PointConv
0.168.10.073
1.4e-5.1..5e
0.05310.042
0.01710.013
0.01610.0111
0.019100015
DGCNN
0.141::0.069
0.067=0.053
0.020=000015
0.014=0.011
0.013=00011
0.016=00013
GCNN
0.141::0.065
0.072-0.038
0.021
0.014=00011
0.013=0.010
0.016:20015
adv-GCNN
0.028+0.012
0.009=00008
0.025=0.020
0.028=00022
0.024=0.015
0.028=0.019
Table 3: Pearson correlation coefficients between regional attributions and sensitivities.
ModeINet10 datuset
ShapeNet part dataset
Models
rotation sensitivity
translation sensitivity
scale sensitivity
rotation sensitivity
translation sensitivity
scale
sensitivity
PointNet
0.648-40.266
0.637++0116 165
0.473:20.19 194
0,528=0.278
0.549=0.204
0.538:+0.275
PointNet++
0.811+0.113
0.415::0.188
0.592 142
0.629=10.14
0.266
0,543:10.171
PointCony
0.601+40.234
0.009++0.15 179
0.473:+0.174
0.739++0.16
-0.006-+0.170
0.617:+0.168
DGCNN
0.788:101111
0,622:10166
0.494:10.224
0.725:10.176
0.649=00174
0,458:10.201
GCNN
0.832:10.082
0.610::0.11
0.464:10.231
0.696:10.158
0.682::0.198
0.431:10.199
adv-GCNN*
0.488:401167
0.298::.234
0.414:10256
0.343=00234
0.255=0.223
0.476:40.30
was min. Eg maxy Loss(r' = (z)). ((ruth;;)) where w denoted the parameter of the GCNN. Comparative studies have revealed the following effects of the adversarial training (using rotations and translations for attack, instead of using perturbations) on knowledge representations. The adversarial training shifted the attention of DNN from orientations and positions to structural information (including global structures in normal samples, see Fig. (5. thereby increasing the sensitivity to 3D) structures (see Table 2
and Fig. 3 (a)). The adversarial training also decreased the correlation between the regional sensitivity and the regional attribution (see Table 3), and increased the spatial smoothness of knowledge representations (see Table 4
Comparative study 1, explaining the regional sensitivity of DNNs. Table 2 shows six types of regional sensitivities of six DNNs learned the ModelNet10 dataset and the ShapeNet part dataset. In terms of translation sensitivity, PointNet was relatively sensitive to the translation, because PointNet extracted features from global coordinates of points, thereby being sensitive to the translation. In contrast, PointNet++ and PointConv were robust to translation, because these two DNNs encoded relative coordinates. Particularly, PointConv did not use global coordinates during the inference process, thereby yielding the lowest translation sensitivity (see the point cloud with the darkest blue in the second row of Fig. 3 (a)). In terms of scale sensitivity, PointNet++ was relatively sensitive to the scale, because PointNet++ encoded features of neighborhood with fixed scales. Besides, because adversarial training forced the GCNN to remove attention from senssiivee features and translation- -enssiivve features, the adversarially trained GCNN paid more attention to structural information. Therefore, compared with the original GCNN, the adversarially trained GCNN had higher sensitivity to local 3D structures. Furthermore, we also obtained the following conclusions.
Rotation robustness was the Achilles' heel of classic DNNs for 3D point cloud processing. As Table and Fig. 3 (a) show, all DNNs were sensitive to rotations except for the adversarially trained GCNN.
PointNet failed to encode local 3D structures. In terms of sensitivity to local 3D structures, PointNet was the least sensitive DNN (see Table 2). It was because convolution operations of the PointNet encoded the information of each point independently, i.e. the PointNet did not encode the information of neighboring points/regions. This conclusion is also verified by the phenomenon that PointNet had darker blue point clouds than other DNNs (see the last three rows of Fig. (a)).
Most DNNs usmally failed to extract rolution- robust features from 3D points at edges and corners. Given a point cloud F, we selected the most cotation-sensitiio region i" (shown as black boxes in Fig. (b)). We rotated the point cloud to the orientations that maximized and minimized the
7

---

PointNet PoimNet! PoimComm DGONN GCNN ad-GCCN
PuintNet PointNet!
PoinCoon DGCNN GCNN ad-OCNN
Senchii 10 000
serssitvilyy sensiivity
misses t0 surfaa
1.000
TO'O
TO
high
sensitivity
sensitivity
minimum
maximum
(a) Visualization six types of regional sensitivitics.
(b) Visualization regional attributions.
Figure 3: Visualization of regional sensitivities and regional attributions. (a) Visualization of regional sensitivities. The regional sensitivities of all point clouds are normalized the same colorbar, which is shown in log-scale. (b) Visualization of regional attributions. For each point cloud, we selected the most rotation-sensitive region i* (shown black boxes) and visualized the pair of point clouds with specific orientations corresponding to the maximum and the minimum regional attributions of the region i*, More visualization results are shown in our supplemental materials.
2 sunpnee Iilluss
sample sample 5
high attribution
CCON
adverarially GCNV
-02
-4
onabe uediinty (a) Positive correlation between the regional attribution and the regional rotation sensitivity.
neaice wardinty
netoe werkiny
attribution
(b) Visualization of spatial smoothness of regional attributions
Figure 4: (a) Positive correlation between the regional rotation sensitivity and the regional attribution of the PointNet. (b) Visualization of spatial smoothness of regional attributions. Experimental results show that the adversarial training increased the smoothness of neighboring regions' attributions.
attribution of region i", i.e. 0 arg maxe du' (i") and 0₂ arg ming Φ- (i"). Fig. (b) visualizes regional attributions of point clouds rotated by 0 and 02. We found that were usually distributed on the edges and corners, which verified our conclusion.
regions
Most DNNs usually could not ignore features of rotation-sensitvve points at edges and corners. Table 3 shows that the Pearson correlation coefficient 25 between each region's 'ss average strength
of attribution over different rotations (i.e. Eel (i)l). s.t. x' = Trotation (z(0)) and this region's 's rotation sensitivity (x)x) is much larger than Pearson correlation coefficients between the attribution and other sensitivities. This means that on the edges and corners) usually had large regional attributions, which verified our conclusion.
regions (which were usually distributed
Besides, Table 3 also shows that the adversarial training reduced the correlation between the regional sensitivity and the regional attribution. Fig. (a) visualizes the positive correlation between the regional rotation sensitivity and the regional attribution the PointNet.
Comparative study 2, explaining the spatial smoothness of knowledge representations. Table 4 shows the non-smoothness of knowledge representations encoded by different DNNs. Note that the adversarially trained GCNN was significantly biased to the knife category. In other words, the

---

Table 4: The non-smoothness of attributions between adjacent regions.
ShapeNet part dataset
ModelNet10 dataset
ShapeNet part dataset
(removing the biased category)
Models
rotation
translation
rolation
translation
rotation
translation
PointNet
0.071=0.039
0.029=0.017
0.025+00.009
0.016=0.005
0.025#0.010
0.015++00003
PointNet++
0,09110.041
0.04110.022
0.036:10.011
0.022-10016
0.03410.010
0.017:10.003
PointConv
0,047=00014
0.05610.108
0.0801100119
0.040=0.017
0.081
0.039:100018
DGCNN
0.071+00.024 0.031+00010
0.047++0.199
0.026:+00077
0.044+0.0117
0.021++0.005
GCNN
0.083=0.026
0.034=0.012
0.050::0:019
0.027=0.010
0.049=0.020
0.025
adv-GCNN
0.029:10.012
0.030+t0.013
0.054:001100
0.056:10.114
0.022:+0.008
0.023:10.008
adversarially trained GCNN classified the empty input as the knife category with high confidence (please see our supplementary materials for details). Therefore, to enable fair comparisons, we also reported the spatial non-smoothness of the adversarially trained GCNN on all other categories except the knife. We discovered that without considering the biased knife category, adversurial training increased the spatial smoothness of knowledge representations. Fig 4 (b) also verified this conclusion.
Comparative study 3, explaining the interaction complexity of DNNs. Fig. Shows multi-order interactions of different DNNs. From this figure, we discovered the following new insights.
Most DNNs failed to encode high-order interactions (i.e. global and large-scale 3D structures). As
Fig. 5 (a) shows, no matter given normal samples or adversarial samples classic DNNs encoded extremely low-order interactions. This indicated that most DNNs did not extract complex and large-scale 3D structures from normal samples.
The adversarial training (using rolations and translations for attack, instead of using perturbations) increased the effects of extremely high-order interactions. As Fig. [5] (a) shows, the adversarially trained GCNN encoded extremely low-order interactions and extremely high-order interactions (i.e. the global structures of objects) from normal samples. Because the GCNN was forced to sophisticatedly select relatively complex rotation-robust features among all potential features.
Rotation sensitivity of regions with oft-of-distribbue complex and large-scale 3D structures) were larger than the average sensitivity over all regions. In fact, there are two types of high-order interactions. Unlike the above two insights describing whether or not DNN encoded high-order interactions on normal samples, this insight mainly focused on abnormal out-of-distribute
high-order interactions (i.e. abnormal
high-order interactions in a DNN. When a point cloud was attacked by adversarial rotations the adversarial rotation would generate interactions (i.e. abnormal complex and large-scale 3D structures) to attack the DNN, We measured interactions between the most cotation-sensitiii region i" and its neighbors j E (i"))) among all
high-orde
input point clouds, i.e. Exex [EjeN((=> [46"m) (t*, [)]]] As Fig. (b) shows, rotation-sensitiio sensitive regions usually encoded relatively more high-order interactions than the overall interaction distribtuion over all regions. This indicates that compared to most other regions, rotation-sensitive regions paid more attention to high-order interactions, although low-order interactions of rotation-sensitive regions increased too.
Broad applicability of the proposed metries. We also conducted experiments on an AutoEncoder for 3D point cloud reconstruction task and rotation-invariant DNNs for 3D point cloud classification, in order to further demonstrate the broad applicability of the proposed metrics.
For the AutoEncoder for 3D point cloud reconstruction task, we used all layers before the fully- connected layers in PointNet [26] as the encoder and used the decoder in the FoldingNet [48) as
the decoder. We took v(S)= (=(N)) [=(N)= (x())=2(0)) as the reward score in Equation
where
z(S) denotes the output vector of the encoder given the input point cloud consisting of regions in
S. This reward score measures the utility of the encoder output 2(S) given regions in along the
direction of =(N) =()). The AutoEncoder was learned based on the ModelNet10 dataset. We measured the rotation sensitivity (0. (0.099)0.0433, the translation sensitivity (0. 0..0006),, and the scale sensitivity (0.045=00.02)) of the AutoEncoder. Compared with results in Table we found that
(I) the AutoEncoder for reconstruction was also sensitive to rotation; (2) the AutoEncoder for 3D
Here, used rotations for attack, instead of using perturbations.

---

normal samples
adversariil samples (using rotations attack, instead petturbbaioos)
Stapeeer ModkiNetio StopeNee ModdiNetl
PointNet
PointNet++
PointConv
DGCNN
GCNN
adv-GCNN
5,00
g 1100
2.00
0.25
0.25
0.25
0.00
order n-2
0.00
order n-2
0.00
order n-2
0.00
order n-2
0.00
order n--
0.00
0 order n-2
22.0
2.50
0
x 0.22
0.50
0.00
order
0.00
order regions
0.00
order among most intaiion-smsiiiie regions
order
0.00
0.00
order
0.00
order
among all
PointNet
PointNet++
PoinsConv
DGONN
GCNN
adv-GCNN
5,00
5.00
E025
0.20
0.50
0.00
onder
0.00
order
0.00
order B-2
0.00
0
order
n-2
0.00
0
order R-2
0.00
onder B-2
5,00
22.50
10.00
1.00
0.550
1.00
0.00
0.00
order
0.00
order n-2
0.00
n-22
0.00
order 0-2
Figure 5: Multi-order interactions of different DNNs. (a) Comparison of multi-order interactions
between normal samples and adversarial samples". (b) Comparison of multi-order interactions between all regions and Dotation-sensitive regions.
Table 5:
Sensitivities of DNNs with or without different types of augmentation. For DNNs with or without canssation/scale/rttann augmentation, we reported the translation//caalcrotaii sensitivity.
Network
trano aug
scale aug
miation ang
rotation around axis
architecture
around y jxaxs
w/
w/o
w/
w/o
w/
w/o
w/o
PointNet
20053
0.025+00017 060
0.03==0.047
0. 108=00047
0. 1.15550000
0.057=0.080
0.155=0.066
DGCNN
0.048:00024
0.098100054
0.02010.015
0.028:20.220
0. 05.0030
0.173=0.072
0.052=00021
0.173:00002
point cloud reconstruction was more sensitive to translation than the DNN for classification; (3) the AutoEncoder for reconstruction was sensitive to the scale change.
Besides, we followed 32) to revise PointNet++ and DGCNN to be cotation-invariant DNNs. We measured the rotation sensitivity of the rotation-invariant PointNet++ (0.002=00.00) and DGCNN (0.01011.0004). Compared with results in Table 2
we found that votation-invariant DNNs were much more robust to rotations than traditional DNNs (even the snvversarilll--trinee DNN).
Effects of data augmentation on sensitivities. We conducted comparative studies to explore the effects of data augmentation on sensitivities, including translation augmentation, scale augmentation, rotation augmentation around the y-axis, and rotation augmentation around random axis. We conducted experiments on PointNet and DGCNN. To explore the effects of each of the above augmentation on sensitivities, we learned two versions of each network architecture, i.e. one network with the specific augmentation and the other network without the specific augmentation. Please see our supplementary materials for more details about different versions of each network architecture. All DNNs were learned based on the ModelNet10 10 dataset. Table that rotation/translation//o sensitivity of a DNN.
augmentation decreased the rotatiio/translatioon/cc.
Besides, we compared the effects of rotation augmentation around a random axis and the effects of adversarial training on the rotation sensitivity. We conducted experiments DGCNN and GCNN. We trained two versions of DGCNN and GCNN, one with rotation augmentation around a random axis (DGCNN, 0.052 +0.021; GCNN, 0.048 +0.044) and one with the adversarial training using rotations for attack (DGCNN, 0.035:0..01;; GCNN, 0.028+0.016). We found that compared with the rotation augmentation, the adversarial training w.r.f. rotation-based attacks had a greater impact on the rotation sensitivity.
5 Conclusion
In this paper, we have measured six types of regional sensitivities, the spatial smoothness, and the contextual complexity of feature representations encoded by DNNs. Comparative studies have discovered several new insights into classic DNNs for 3D point cloud processing. We have found regions were usually
that most DNNs were extremely sensitive to the rotation. Ccoaationseensiiee distributed on the edges and corners of objects, had large attributions to network output, and usually paid more attention to abnormal large-scale structures. In addition, most DNNs failed to extract complex and large-scale 3D structures from normal samples, while the adversarial training could encourage the DNN to extract global structures from normal samples.
10

---

Acknowledgments and Disclosure of Funding
This work is partially supported by the National Nature Science Foundation of China (No. 61906120, U19B2043), Shanghai Natural Science Fundation 22JC114038002212214446666000 Shanghai Mu- nicipal Science and Technology Major Project (20211HHZDZX01002,, This work is also partially supported by Huawei Technologies Inc.
References
Marco Ancona, Cengiz Oztireli, and Markus Gross. Explaining deep neural networks with a polynomial time algorithm for shapley value approximation. Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 272-281, Long Beach, California, USA, 09-15 Jun 2019. PMLR.
[2] Song Bai, Xiang Bai, Zhichao Zhou, Zhaoxiang Zhang. and Longin Jan Latecki. Gift: A real-time and scalable 3d shape search engine. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5023 5032, 2016.
[3] Yizhak Ben-Shabat, Michael Lindenbaum, Anath Fischer. 3d point cloud classification and segmen- tation using 3d modified fisher vector representation for convolutional neural networks. arXiv preprint arXiv:/711.0000 2017.
[4] Javier Castro, Daniel Gómez, and Juan Tejada. Polynomial calculation of the shapley value based on sampling. Computers & Operations Research, 36(5):1726 1730, 2009.
[5] Aditya Chattopadhay, Anirban Sarkar, Prantik Howlader, and Vineeth N Balasubramanian. Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks. In 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 339-847. IEEE, 2018.
[6] Demantké, C. Mallet, N. David, and Vallet. Dimensionality Based Scale Selection in 3d LIDAR Point Clouds. ISPRS - International Archives of the Photogranmetry, Remote Sensing and Spatial Information Sciences, 3812:97-102. 2011.
[7] Alexey Dosovitskiy and Thomas Brox Inverting visual representations with convolutional networks. In Proceedings the IEEE Conference on Computer Vision and Pattern Recognition, pages 4829-4837, 2016.
[8] Ruth Fong and Andrea Vedaldi, Interpretable explanations of black boxes by meaningful perturbation. In Proceedings of the IEEE International Conference on Computer Vision, pages 3429 3437, 2017.
[9] C. Frye, Damien de Mijolla. T. Begley, Laurence Cowton. Megan Stanley, and Ilya Feige. Shapley explainability on the data manifold. In ICLR, 2021.
[10] Ankit Goyal, Hei Law, Bowei Liu, Alejandro Newell, and Jia Deng. Revisiting point cloud shape classification with simple and effective baseline. arXiv preprint XXiv:2106.05304 2021.
[11] Michel Grabisch and Mare Roubens. An axiomatic approach the concept of interaction among players in cooperative games. International Journal of game theory, 2844)5547-565, 1999.
[12] S. Guinard and Landrieu. Weakly supervised segmentation aided classification of urban scenes from lidar point clouds. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XI11--/W11111-155,, 2017.
[13] Qingyong Hu, Bo Yang, Linhai Xie, Stefano Rosa, Yulan Guo, Zhihua Wang, Niki Trigoni, and Andrew Markham. Randla-net: Efficient semantic segmentation of large-scale point clouds. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition, pages 11 108-11117, 2020.
[14] Asako Kanezaki, Yasuyuki Matsushita, and Yoshifumi Nishida. Rotationnet: Joint object categorization and pose estimation using multiviews from unsupervised viewpoints. In Proceedings of the IEEE Conference on Computer Vision and Patter Recognition, pages 5010-5019, 2018.
[15] Pieter-Jan Kindermans, Kristof T Schütt, Maximilian Alber, Klaus-Robert Müller, Dumitru Erhan, Been Kim, and Sven Dähne. Learning how to explain neural networks: Patterneet and patternattribution. arXiv preprint arXiv: /e0e::005005500 2017.
[16] Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial examples the physical world, 2017.

---

[17] Loic Landrieu and Martin Simonovsky. Large-scale point cloud semantic segmentation with superpoint graphs. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pages 4558-4566 2018.
[18] Yangyan Li, Rui Bu, Mingchao Sun, Wei Wu, Xinhan Di, and Baoquan Chen. Pointenn: Convolution on t-rransforeed points. In Advances in Neural Information Processing Systems, pages 820-830, 2018.
[19] Yongcheng Liu, Bin Fan, Shiming Xiang, and Chunhong Pan. Relation-shap shape convolutional neural network for point cloud analysis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8895-8904, 2019.
[20] Scott Lundberg and Su-In Lee. A unified approach interpreting model predictions. arXiv preprint urXiv:1705.07874 2017.
[21] Scott M Lundberg. Gabriel G Erion, and Su-In Lee. Consistent individualized feature attribution for tree ensembles. arXiv preprint arXiv:1802.03880 03888888 2018.
[22] Aravindh Mahendran and Andrea Vedaldi. Understanding deep image representations by inverting them. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pages 5188 5196, 2015.
[23] Jiageng Mao, Xiaogang Wang, and Hongsheng Li. Interpolated convolutional networks for 3d point cloud understanding. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1578-1587, 2019.
[24] Alexander Mordvintsev, Christopher Olah, and Mike Tyka. Inceptionism: Going deeper into neural networks. 2015.
[25] K Pearson, Notes on regression and inheritance in the case of two parents proceedings of the royal society of london, 58, 240-242, 1895.
[26] Charles Qi, Hao Su, Kaichun Mo, and Leonidas Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 652-660, 2017.
[27] Charles Qi, Or Litany, Kaiming He, and I Leonidas Guibas. Deep hough voting for 3d ohject detection in point clouds. In Proceedings of the JEEE/CVF International Conference on Computer Vision, pages 9277-9286, 2019.
[28] Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet+++: Deep hierarchical feature learning on point sets in a metric space. In Advances in Neural Information Processing Systems, pages 5099-5108, 2017.
[29] Marco Túlio Ribciro, Sameer Singh, and Carlos Guestrin. "why should trust you?": Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135 1144. ACM, 2016.
[30] Ramprasaath R Selvaraju, Michael Cogswell, Abhishck Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision, pages 618-626, 2017.
[31] Lloyd S Shapley. A value for n-person games. Contributions to the Theory of Games, 2(28):307-317, 1953.
[32] Wen Shen, Binbin Zhang, Shikun Huang, Zhihua Wei, and Quanshi Zhang. 3d-rotation-equiivriaat quaternion neural networks. Computer Vision ECCV 2020: 16th European Conference, Glasgow, UK, August 23 28, 2020, Proceedings, Part XX 16, pages 531-547. Springer, 2020.
[33] Wen Shen, Zhihua Wei, Shikun Huang, Binbin Zhang, Panyue Chen, Ping Zhao, and Quanshi Zhang. Verifiability and predictability; Interpreting utilitics network architectures for point cloud processing. In Proceedings of the IEEL/CVF Conference on Computer Vision and Pattern Recognition, pages 10703- 10712, 2021.
[34] Yiru Shen, Chen Feng, Yaoqing Yang, and Dong Tian. Mining point cloud local structures by kernel correlation and graph pooling. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4558-4557, 2018
[35] K Simonyan. A Vedaldi, and A Zisserman. Deep inside convolutional networks: visualising image classification models and saliency maps. arXiv preprint arXiv:1312.603. 2017.
12

---

[36] Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034, 2013.
[37] Mukund Sundararajan and Amir Najmi. The many shapley values for model explanation. In International Conference on Machine Learning, pages 9269 9278.. PMLR, 2020.
[38] Mukund Sundararajan, Kedar Dhamdhere, and Ashish Agarwal. The shapley taylor interaction index. In International Conference on Machine Learning, pages 9259 9268. PMLR, 2020.
[39] Hugues Thomas, Charles R Qi. Bean-Emmanuel Deschaud, Beatriz. Marcotegui, François Goulette, and Leonidas J Guibas. Kpconv: Flexible and deformable convolution for point clouds. In Proceedings of the IEEE International Conference on Computer Vision, pages 641 1-6420, 2019.
[40] Lei Wang. Yuchun Huang. Yaolin Hou, Shenman Zhang, and Jie Shan. Graph attention convolution for point cloud semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10296-10305, 2019.
[41] Weiyue Wang, Ronald Yu, Qiangui Huang, and Ulrich Noumann. Sgpn: Similarity group proposal network 3d point cloud instance segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2569-2578, 2018.
[42] Yue Wang and Justin Solomon. Deep closest point: Learning representations for point cloud registration. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3523-3532, 2019.
[43] Yue Wang, Yonghin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon. Dynamic graph cnn for learning on point clouds. ACM Transactions on Graphics (TOG), 3865):146, 2019.
[44] Robert J Weber. Probabilistic values for games. The Shapley Value. Essays Honor of Lloyd S. Shapley, pages 101-119, 1988.
[45] Wenxuan Wu, Zhongang Qi, and Li Fuxin. Pointconv: Deep convolutional networks on 3d point clouds. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, pages 9621 1-630, 2019.
[46] Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 3d shapenets: A deep representation for volumetric shapes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1912-19200 2015.
[47] Guandao Yang, Xun Huang, Zekun Hao, Ming- Yu Liu, Serge Belongic, and Bharath Hariharan. Point- flow: 3d point cloud generation with continuous normalizing flows. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4541 44550, 2019.
[48] Yaoqing Yang, Chen Feng, Yiru Shen, and Dong Tian. Foldingnet: Point cloud auto-encoder via deep grid deformation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 206-215, 2018
[49] Li Yi. Vladimir G. Kim, Duygu Ceylan, I-Chao Shen, Mengyan Yan, Hao Su, Cewu Lu, Qixing Huang, Alla Sheffer, and Lconidas Guibas. A scalable active framework for region annotation in 3d shape collections. SIGGRAPH Asia, 2016.
[50] Jason Yosinski, Jeff Clune, Anh Nguyen, Thomas Fuchs, and Hod Lipson. Understanding neural networks through deep visualization. arXiv preprint arXiv: 150.06699, 2015.
[51] Lequan Yu, Xianzhi Li, Chi-Wing Fu, Daniel Cohen-Or, and Pheng-Ann Heng. Pu-net: Point cloud upsampling network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2799, 2018.
[52] Matthew D Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In European Conference on Computer Vision, pages 118-833. Springer, 2014.
[53] Hao Zhang, Xu Cheng, Yiting Chen, and Quanshi Zhang. Game-theoretic interactions of different orders. arXiv preprint arXiv:2010. 2010.14788 2020.
[54] Yue Zhao, Yuwei Wu, Caihua Chen, and Andrew Lim. On isometry robustness of deep 3d point cloud models under adversarial attacks. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 1201-1210, 2020.
[55] Tianhang Zheng, Changyou Chen, Junsong Yuan, Bo Li, and Kui Ren. Pointcloud saliency maps. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 2019.
13

---

[56] Bolei Zhou, Adilya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Object detectors emerge in deep cnns. In ICLR, 2015.
[57] Bolei Zhou, Adity Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 2921-2929 2016.
14


Reviews:
[{"review_id": "l3p7pk5Uiq3", "reviewer": "Reviewer_xqQ9", "summary": "This paper introduces metrics that reflect the representation quality, and properties, acquired by a deep-net that is applied in 3D Point-Cloud (PC) processing tasks. Concretely, the authors introduce six novel metrics that in aggregate capture different types of regional sensitivities e.g., the sensitivity of the DNN in rotations on specific regions/rotation-angles of the input PC. They further explore the extent that modern architectures in standard benchmarks acquire “spatial smoothness” --  do nearby points have ‘similar’ effects/importance for the end task? And finally, they explore the acquired representation complexity of the DNN by adapting a 'multi-order interaction' metric to PC data.  \n\nThese metrics are being applied on a well-selected subset of modern PC-DNN architectures such PointNet++ or DGCNN when these are trained either for PC-object classification, or part-based object-segmentation. The analysis of the metrics in these tasks/architectures highlights a number of interesting points that oftentimes are very simple/intuitive: e.g., PointNet fails to encode local 3D structures (e.g., compared to PointNet++) — or includes results that are more subtle, but still highly intuitive & rich: e.g., adversarially training a PC-DNN with rotation/translation attacks; will force it to learn more global PC structures compared to standard training; which is intuitive as the local structures, per the new metrics, are those that on average are more susceptible to rotations.", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper is original, written with clarity, and introduces ideas, per the evaluation metrics that I think will be very helpful for the PC-Deep-Learning community. The exposition is clear and most findings well align with intuitions known by practitioners.  For instance, it is widely known that PointNet based variants do not cope well with rotations and that ad-hoc mechanisms that are typically applied to address this issue (e.g., the pc/feature rotation transformation loss in the original PointNet) have very limited success -- which is partially why a lot of work keeps happening in rotation-invariant or equivariant PC-DNNs. This paper brings a very elegant instrument to measure such a dependence (among others): the Shapley values. These can capture the sensitivity of different PC \"sub-regions\" to a specific quantity of interest, not only rotations but other structures like edge vs. surface vs. volume like structures. To this end, another interesting finding is that PointNet++ is the network that pays most attention to intrinsic 3D structures: \"masses\", which can be why in many settings is still the better performing, e.g., see [1].\n\nThe findings regarding how the adversarially trained net shifted its attention from orientations and positions to structural information and increased its spatial smoothness are important and they demonstrate a use case of how the new metrics can be used to examine/analyze the behavior of black-box models, training losses, etc.\n\nThe only concern that I have with the paper is that IMO lacks extensive use-cases analysis. While the networks architectures tested seem adequate; the datasets/tasks are less extensive. E.g., why not using ShapeNet  (or ModelNet50) for a larger classification problem? How is the training size affects the different metrics? How about trying auto-encoders or other forms of learning used for point-cloud processing? While, this seems like an extra mile that another paper can do; if you had included analysis say on AutoEncoders, GANs etc. to complete the picture, I would argue this paper should have very strong acceptance. \n\n\nMinor:\nL187. is represented ->  are\nFigure 3. (b) -> are these objects supposed to be at different angles? I cannot tell. \n\n\n[1] Goyal et al. Revisiting Point Cloud Shape Classification with a Simple and Effective Baseline\n\n\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.", "review_text": "", "extra_scores": {"ethical_concerns": "I do not have any ethical concerns about this paper. ", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}, {"review_id": "ZjuaJdTbK9X", "reviewer": "Reviewer_NChg", "summary": "This paper studies how to understand the quality of knowledge representations encoded in point cloud DNNs. Based upon the Shapley value, the paper presents six metrics to evaluate the model vulnerability toward different types of input variations. In addition, it presents an analysis regarding the smoothness and multi-order interactions of these attributions, pointing out the restrictions of current point DNNs. The paper makes interesting observations and draws useful conclusions, supported by a range of experimental evaluations.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This is a diagnosis paper studying an important problem. The submission is clearly written, making it easy to follow. The content is also well organized with enough technical details provided. Generally speaking, the submission is technically sound. The motivation behind different metric designs is clear and the experimental results successfully support the main claims. The diagnosis is quite new for 3D point cloud analysis. Its difference from previous works is also clearly discussed. The related work is adequate as far as I can tell.\n\nI still have some questions and I hope the authors could provide more explanations to strengthen the submission.\n1. Is the conclusions task dependent? Would the observation change if analyzing with another task, say semantic segmentation?\n2. Is the smoothness score defined for a specific transformation or averaged across all types of transformations considered?\n3. Is there any intuition why does the biased knife category appears for adversarial training?\n4. Why does the GCNN mainly encodes extremely low-order or high-order interactions but not something in between?\n5. How does the size of the selected regions influence the observations and analysis?\n\nI think the paper is making an interesting analysis which reveals the limitations of current point DNNs and could provide intuitions for future designs.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "As I mentioned, there are some details unclear. Addressing them could potentially strengthen the submission. It would be interesting to include sparse voxel conv based methods given that it is one of the main streams for sparse point cloud analysis. It would also be interesting to see whether rotation equivariant networks could mitigate the rotation sensitivity issue.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "MCyQ3zjXfi7", "reviewer": "Reviewer_8Z6V", "summary": "At a higher level, this paper is proposing several ways to measure the sensitivity and smoothness of representations learnt by 3D point cloud networks. This paper presents several ways to measure the sensitivity of point cloud networks to rotation, scale, translation and local structure. The Paper also proposes different metrics to evaluate the smoothness of encoded local 3d structures and the complexity of representations learnt by the network. The analysis is done using several point-based neural networks trained on the ModelNet40 classification task and shapenet part segmentation task.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "1. Paper is well written and claims are stated clearly.\n2. The ideas explored in the paper are important for understanding how much point-based networks are vulnerable to different perturbations in the input and also how smooth/complex the learned representations are. This might lead to the design of better architecture that is robust to these perturbations.\n3. Experiments are done on a variety of architectures and elucidate the ideas presented in the paper.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "1. This is not a limitation per se, but it would be interesting to see how much data augmentation changes sensitivity to different perturbations.\n2. Furthermore, there are several works on rotation invariant neural networks for 3d shapes. I wonder if the representations learnt by these architectures are different when evaluated using proposed metrics.\n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4.5"}, "extra_text": {}}, {"review_id": "A4My_nl5jAN", "reviewer": "Reviewer_qvpe", "summary": "This paper measures the sensitivity of various point cloud classification networks (like PointNet, PointNet++, DGCNN) to rotation, translation, scale, and local 3D structure. It also proposes ways to measure the 3D smoothness of a network. Based on these analyses, the work provides various insights. One of the main insights being that the DNNs don’t handle rotation very well. Another insight is that training with adversarial perturbations helps.", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths\n\n- The kind of analysis used in the paper to quantify different aspects of DNN could be useful for 3D point cloud processing.\n\nWeakness\n\n- The data augmentation could have a very significant impact on the conclusions in the paper (as explained below) but those details are missing. Specifically, it is unclear what kind of data augmentation strategies are used while training. For most of the best-performing models on point cloud classification, rotation augmentation is not used while training the DNN [1]. Only scale and translation augmentations are used. This is because while testing the object models are already axis-aligned. Hence, rotation augmentation does not help in improving the test set performance.  If this standard practice (i.e using scale and translation augmentation and not using rotation augmentation) is used in this paper as well, then the insights provided in the paper are very much expected, as a DNN would only be invariant to changes it has seen while training. Moreover, it would explain why adversarially trained GCNN, which would take rotated samples as input while training would be less sensitive to rotation. This would severely limit the contribution of the current work. Therefore without knowing this detail it would be difficult to judge the importance of the work.\n\n- The paper lacks discussion about prior works which also point out that DNN for point cloud processing is sensitive to rotation (for example [1, 2]). The paper should add such a discussion to contextualize the findings with regards to what is already known.\n\n- It would also be good to mention the accuracy of the DNNs used for the analysis to make sure that well-trained DNNs have been used. More details about the exact training protocol should also be provided.\n\n- The resetting of absent points (L124-L132) could introduce unnecessary artifacts impacting the performance of DNNs. Most DNNs like PointNet, PointNet++, DGCNN should be able to handle the different number of points by default. It would be important to compare if simply removing points and replacing missing points with center lead to the same performance.\n\n[1] Revisiting point cloud shape classification with a simple and effective baseline\n\n[2] PointCNN: Convolution On -Transformed Points, NeuRIPS 2018\n\nMinor Comments:\n\n- It seems like many insights mentioned in the introduction repeat the same information. It would be useful to mention the insights in a more concise form. The reviewer feels that simpler and more fundamental insights would be more useful than many complicated ones; at least in the introduction.\n\n\nOverall Recommendation:\n\nIn the current version, critical information is missing. Hence, one cannot judge how useful the contributions are. There are some other issues as well (see weak points). I look forward to the response by the authors. For now, I would recommend a borderline reject rating for the paper.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Please refer to weaknesses discussed in the main review.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper introduces metrics that reflect the representation quality, and properties, acquired by a deep-net that is applied in 3D Point-Cloud (PC) processing tasks. 
This paper presents several ways to measure the sensitivity of point cloud networks to rotation, scale, translation and local structure.
All reviewers laud the originality, clarity of the manuscript, and the ideas. 
The initial paper lacked comparative analysis on different datasets, which the rebuttal seems to have addressed. 
All reviewers agree that this paper provides an interesting result for the community to justify acceptance. 
Decision: Accept (Poster)