Adversarially Robust 3D Point Cloud Recognition Using Self-Supervisions
Authors: Jiachen Sun
Yulong Cao
Christopher Choy
Zhiding Yu
Anima Anandkumar
Zhuoqing Mao
Chaowei Xiao
Venue: NeurIPS 2021 Poster (2021)
Keywords: Adversarial Training
Point Cloud Recognition
Self-supervised Learning

Abstract: 3D point cloud data is increasingly used in safety-critical applications such as autonomous driving. Thus, the robustness of 3D deep learning models against adversarial attacks becomes a major consideration. In this paper, we systematically study the impact of various self-supervised learning proxy tasks on different architectures and threat models for 3D point clouds with adversarial training. Specifically, we study MLP-based (PointNet), convolution-based (DGCNN), and transformer-based (PCT) 3D architectures. Through extensive experimentation, we demonstrate that appropriate applications of self-supervision can significantly enhance the robustness in 3D point cloud recognition, achieving considerable improvements compared to the standard adversarial training baseline. Our analysis reveals that local feature learning is desirable for adversarial robustness in point clouds since it limits the adversarial propagation between the point-level input perturbations and the model's final output. This insight also explains the success of DGCNN and the jigsaw proxy task in achieving stronger 3D adversarial robustness.

Adversarially Robust 3D Point Cloud Recognition Using Self-Supervisions
Jiachen Sun " Yulong Cao ' Christopher Choy?, 2 Zhiding Yu 2, Anima Anandkumar 2.3
Z. Morley Mao I and Chaowei Xiao
University of Michigan, NVIDIA, Caltech, ASU
Abstract
3D point cloud data is increasingly used in safety-critical applications such autonomous driving. Thus, the robustness of 3D deep learning models against adversarial attacks becomes major consideration. In this paper, we systemati- cally study the impact various self-supervised learning proxy tasks on different architectures and threat models for 3D point clouds with adversarial training. Specifically, we study MLP-based (PointNet), convolution based (DGCNN), and ransformer-basee (PCT) 3D architectures. Through extensive experimentation, we demonstrate that appropriate applications of self-supervision can significantly enhance the robustness in 3D point cloud recognition, achieving considerable improvements compared to the standard adversarial training baseline. Our analysis reveals that local feature learning is desirable for adversarial robustness in point clouds since it limits the adversarial propagation between the point-level input perturbations and the model's final output. This insight also explains the success of DGCNN and the jigsaw proxy task in achieving stronger 3D adversarial robustness.
Introduction
Point cloud data is one of the most broadly used representations in 3D computer vision. It is a versatile data format available from various sensors and computer-aided design (CAD) models. Given such advantages, many deep learning-based 3D perception systems have been proposed [1-6] and achieved great success in safety-critical applications (e.g., autonomous driving) [7-9]. Although deep learning [5, 10] on point clouds has exhibited high performance, they are particularly vulnerable to adversarial attacks [1 [1]-13]. Because the wide applications safety-critical fields, is imperative to study the adversarial robustness of point cloud recognition models.
However, there are at present two obstacles on the path reaching robust point cloud recognition:
Architectural Diversity. Deep 3D point cloud recognition is an emerging field. Many 3D architec- tures with various feature aggregation methods have been proposed. We broadly categorize them into three families based on how networks aggregate the geometric information: multi-layer-peree perceptron (MLP)-based networks [0], convolutional networks [3.4, 6, 14-16], and transformer sosssebberbbeddddd net- works [17, 18]. These networks have been studied mostly using clean accuracy as their main metric, but an in-depth study of their robustness is lacking.
Vulnerability to Adaptive Attacks. A few defenses against 3D attacks have been recently pro- posed [19-21]. Some of the methods, however, merely obfuscate attackers by limiting the malicious agents from accessing the defense systems and true gradients [22]. They have been shown vulnerable to adaptive attackers who have the full knowledge of the defenses and can approximate the gradi- ents [21]. Similar to the 2D domain [23], adversarial training (AT) [21] provides more longstanding robustness in 3D even against adaptive attacks [22]. Nevertheless, further robustness improvements on adversarial training are highly desired for practical usage and deployments.
Coorespondencee to jiachensOumich ch. edu
35th Conference on Neural Information Processing Systems (NcurIPS 2021)

---

PanthetosChikin Bactbore
MLP Clanatipaion
Zoom
Kusolaaiio Logat
SD Josaw
Brandhing Layer
Pooking
e:
Autoenccder
Auoillary Hoad
Pindral Ouipal
SShite Ano X Drop
Adverand Pre-aaining Adverssrrill Fine-trinn = Adverand Juir Traning (b) Illustration of Adversarial Pre-training for Fine-tuning
(a) Illustration of Different Threats and Self- Supervised Learning Tasks.
(APF) and Adversarial Joint Training (AJT).
Figure I: Overview of Our Analysis in 3D Point Cloud Classification.
Self-supervised learning (SSL) has been incorporated into adversarial training in 2D image perception models lately. It has shown great potential to enhance adversarial robustness without requiring any additional data or labels [24, 25]. Given such achievements, a natural question emerges: can we mimic the application of SSL to improve adversarial robustness in 3D point cloud recognition? Such a label-free strategy preferred due to the cost and difficulty
point cloud data annotation [26].
Summary of Our Contributions:
In this paper, we present a systematic analysis of the adversarial robustness 3D point cloud recogmition using self-supervisions on three representative architectures: a multi-layer-- layer-perceptron (MIP) network (PointNet) [5], a convolutional network (DGCNN) [15], and a transformer-based network (PCT) [17]. Specifically, we use two strategies to integrate self-supervised learning and adversarial training, including (1) adversarial pre-training for fine-tuning (APF), which uses the SSL tasks only for pre-training, and (2) adversarial joint training (AJT), which jointly trains the SSL task with the recognition task, as shown in Figure 1. To further study the importance self-supervised tasks for adversarial robustness, we select three representative SSL proxy tasks, including 3D rotation prediction [27], 3D jigsaw [28], and autoencoding [29]. Our key observations are as follows:
We show that pre-training on SSL tasks improves adversarial robustness of the tne-ttnnd models. Unlike the 2D domain, where both APF and AJT have enhanced the robustness, our study finds that only APF consistently achieves robustness improvements 3D. AJT does not always help since the distributional gap between data for SSL and recognition tasks will distract each other in AJT. Evaluation results of various unforeseen attacks further confirm such improvements by APF:
We find that the convolutional network, iee, DGCNN, is more robust than the other architectures in point cloud recognition tasks. Moreover, 3D jigsaw SSL task, which predicts the permutation of 3D point cloud patches, helps achieve stronger robustness than the others. Both convolutional architecture and jigsaw SSL task enforce the model to learn better local semantics. Intuitively, robust local features help limit the propagation of adversarial effect from point-level input perturbations to the model's final output.
We demonstrate that fine-tuned models from different pre-training tasks have different vulnerabili- ties, and adversarial examples generated by attacking them do not transfer well among each other. Thus, further leverage two simple yet powerful ensemble methods boost the adversarial robustness by substantial margin. Our best ensemble models, for instance, achieve robust accuracy of 53.5% ++1..6)) 69.4% (+7.4%) and 57.9% (+8.8%) with PointNet, DGCNN, and PCT on the representative dataset, ModelNet40 [30].
2 Analysis Methodology
In this section, we detail our adversarial robustness analysis methodology. We first introduce the principal 3D point cloud recognition architectures and the threat models used in study. We then introduce two ways to generalize and improve AT using 3D point cloud SSL proxies.
2.1 3D Point Cloud Recognition Models and Threats
We introduce the adopted model designs and the formulations of threats to 3D point clouds below.
network PointNet [5], a convolutional network Dynamic Graph CNN (DGCNN) [15], and a transformer-based network Point Cloud Transformer (PCT) [17] as our primary backbone architectures, denoted as Ma... Specifically, PointNet directly aggregates learned features from each point to form global embedding for final
Model Variants. We use shared centtilayeerperceptroo------

---

recognition. By doing so, the point-level features are independent of each other before global pooling. PointNet embraces high efficiency that has been widely utilized as base operation in complex tasks [8,31]. DGCNN instead builds a graph based on k-nearest neighbors (kNN) and uses a variant of continuous convolution (EdgeConv) on edges to enable local feature learning. The combination of local and global representation learning helps DGCNN achieve a higher clean accuracy. PCT extends the Transformer [32], the dominant framework in natural language processing, for point cloud recognition. The core of PCT is the full attention mechanism, which establishes much more flexible scheme, where each point has the potential to affect every other point in the point cloud. The classification head He, parameterized with 9n for these backbones is an MLP and the part segmentation head He. is a set of 1 x convolutions. We use Fe, parameterized with 9 (0, := [0m: 0%. represent the overall model architecture, consisting of the stacked backbone . and recognition head H. where F = H o M. Given the input point cloud æ; the model F aims to predict the corresponding label y. where y = Faz)). More details of the architectures are in the supplements.
Threats. There are mainly three types of threats against point cloud perception models, which could be abstracted as point shifting (PS), point dropping (PD), and point adding (PA) attacks (Figure (()).. We formally define these threats within Ep projected gradient descent (PGD) style attacks. First, we assume PS adversary is able to shift all existing points within a lp norm ball:
x xz+11 (x + sign(V (a. y; F))); Zo = U(-c,e)
(1)
where v, the adversarial example in the s-th iteration, II the projection function to project the adversarial example to the pre-defined perturbation space S, a is the attack step size, and U( U(-e, e) represents a uniform distribution from -6 c. Second, we allow PD adversary to discard the top
salient points [33] in each iteration until total drop of N points:
1s+1 x. arg topkæEæ, saliency(x,, y: F); kxs<N
(2)
where we follow [33] to implement saliency() () to calculate the importance scores of the input points w.r.t. to the prediction accuracy and/ / denotes the dropping operation. We detail saliency () in the supplements due to space limits. Third, PA adversary is capable of adding new points bounded by an l norm ball to the original point cloud. PA randomly initializes N new points from the original point coordinates and only perturbs the added points. We follow the same setting as PS to formulate the perturbation:
Es+1 = 5( + sign V., (( vs), y: F))); xo sample(@ N) + U(-e,) (3)
where Tori is the original point cloud and sample() initializes N new points denoted as Zn. We believe the chosen threats mostly cover the existing attack surfaces on point cloud data. It is also beneficial to study how different threats would affect the adversarial robustness in point cloud recognition under our adversarial training strategies which will be introduced next.
2.2 Adversarial Training with Self-Supervisions
We first introduce the chosen 3D self-supervised learning methods, followed by two strategies to incorporate these pretext tasks in adversarial training.
3D Self-Supervised Learning. The primary goal of self-supervised learning (SSL) is to learn effective feature representations with unlabeled data. Given a pretext task Pt. the pre-training process is still conducted in a supervised manner with self- self-generated data x' and label y from pristine data æ, where (x', y²) (x)). Therefore, a target loss function C, y²; FF,)) will be minimized during the optimization, where Or consists of the shared backbone parameters @,, and customized branch parameters 0. (i.e., 0, == i fz)). We utilize the following 3D SSL tasks in our study (Figure (a))..
3D Rotation [27]: Similar to the rotation task in 2D vision [34], the data and label are generated by rotating the original point clouds to pre-defined angles 1) in the 3D space. Therefore, the problem to correctly predict 3D rotation angles w.E. the input point cloud. The objective function can be formulated as cross-entropy (CE) loss: Lrotation CE(x rotation .y rooaation Frotation).
3D Jigsaw [28]: Different from the jigsaw task in 2D vision [35] which is defined as classification problem, 3D jigsaw solicits a segmentation model. A point cloud is evenly divided to 133 small cubes and shuffled to different positions. Points inside each small cube are assigned to label signaling its original position. The problem, thus, is to correctly predict the original cube position of each point. Similarly, its objective is to minimize a CE loss: Ljigeaw CE(x (ig y yigaaw, F.iggaaw)
3

---

Autoencoder [29,36]: An autoencoder utilizes an encoder 2 = E(x) to learn a compact representa- tion and a decoder (()())) to reconstruct the point cloud. We utilize different backbones as the encoder F.()) and NoldingNet [29] as the decoder (.)) due to its satisfactory performance. We use three different positional encodings: plane, 3D sphere, and 3D gaussian in our experiments, we use the Chamfer distance [37] as the reconstruction loss: Le Chamfer( D (F(E Exx")) x"e; ; Fue).
The detailed description of the pretext tasks can be found in the supplements.
Adversarial Pre-training for Fine-tuning (APF). As introduced in 11, adversarial training (AT) [23, 38, 39] has been demonstrated to be one of the most longstanding and practical defenses. We thus enable AT in both pre-training and fine-tuning stages:
arg min 0
E(m.y) max C(x + 0, 0.
(4)
where L Lyr for loss functions in pre-training (t) and fine-tuning (f) stages, is the adversarial perturbations, and represents its manipulation space (i.e. foo in our study). AT essentially solves min-max problem. In the inner loop, the optimizer tries find adversarial examples that maximize the target loss, and the outer loop updates the network parameters to correctly recognize the generated adversarial examples. In contrast, standard training (ST) is simply to optimize arg ming ) [L y, 0)].
In the pre-training stage of APF, we leverage both standard and adversarial training to get the pre- trained backbones Me... and MM... Given a pre-trained backbone parameterized by 6n. in the second stage, we adversarially fine-tune all 0 == [0m 0n. for the recognition task, as illustrated in Figure 1(b). The network branches at the penultimate vector for the rotation task and the first global feature [21] for the jigsaw and autoencoder tasks since they use the segmentation head.
Adversarial Joint Training (AJT). Besides pre-training for fine-tuning, joint training another way to apply SSL. The objective function is formulated as:
arg min O,.
E (a,y)~D
max (f( + +., + Lo 0,) EE8
(5)
where À is a hyperparameter to balance the SSL and recognition tasks. Two tasks share the same backbone 6... with two different branches, parameterized by 0, and o.. respectively. We also enable dual batch normalization [40] in AJT for æ and x' since they should belong to different underlying distributions. We use two model-agnostic tasks, i.e., 3D rotation and jigsaw in AJT.
Similarly, in our AJT analysis, all L, and Ly can be formulated as CE loss. We empirically set and leverage the same branching point with APF. The whole network is trained to predict the supervised task with the original head and the SSL task with the auxiliary head (Figure (bb)..
3 Experiments and Results
In this section, we present our experimental setups and results. We first introduce the adopted datasets and adversarial settings. We leverage PS the primary threat in our study since we find it is the most powerful adversary and the other two adversaries easier to detect. Next, we detail our extensive evaluation of two fundamental point cloud recognition tasks: classification and part segmentation.
3.1 Evaluation Setups
Datasets. We leverage four datasets (D): ModelNet40 [30] (40 classes), ModelNet10 [30] (10
classes), ScanObjectNN [41] (15 classes), and ShapeNetPart [42] throughout our experiments. Specifically, the first three datasets are utilized for the classification task and the last one is for the part segmentation task. ModelNet and ShapeNetPart are captured from CAD models, and ScanObjectNN is scanned and extracted from real-worke indoor scenes. For each point cloud, we randomly sample 1024 points and normalize them to an edge-length-2 2 cube ([-1, 1) for experimentation. We follow the default split of training and test sets in [5] and [43]. For SSL, we randomly sample y from the pre-defined label sets and further generate x based on y in each iteration. Specifically, we choose 7 6, 18 and k = 3,4 4 for rotation and jigsaw tasks, followed by the suggestion of [27] and [28].
Adversary. As introduced in 1, for PS, we exploit 7-step and 200-step foo PGD attacks [23] targeting the cross-entropy loss for adversarial training and testing, respectively. We follow Sun et al. [21] to empirically set the perturbation boundary 0.05 (||a]|oc S 0.05) since perturbed

---

point clouds with e = 0.05 are at the edge of correct human predictions of objects. Numerically, e 0.05 out of the range [-1,1] is also similar to the commonly used 25; in 2D adversarial training [23]. Note that, different from discrete RGB values in 2D images, point cloud's features (coordinates) are continuous. We thus utilize PGD step size a = 0.01 and a = 0.005 the training and testing phases, respectively. We allow a stealthy perturbation of N = 100 points for PD and PA since we believe a larger number of modification would be easy to detect [13,33], We drop the most A: = 14 and 5 salient points every iteration in training and testing phases for PD, respectively. The behind rationale is that a severer attacker is preferred in the testing phase to evaluate the true robust accuracy. The attack setups in PA are the same as those in PS since they utilize similar 2o PGD perturbation methods. We exploit PS as the primary threat for the classification and part segmentation tasks, and leverage the other two threats further demonstrate our robustness improvements in the classification task.
Original Point Cloud Chair
Point Shifting Bookshe
Sailent Point Gropping
Point Addlnng Lamp
Tollet
Figure 2: Adversarial Examples of Different Threats.
3.2 Point Cloud Classification
In this section, we first introduce the setups of our point cloud classification analysis. Next, we introduce the detailed study of APF and AJT. We further evaluate our robust models with unforeseen attacks to demonstrate that our robustness improvements are non-trivial. We find that pre-trained models from different SSL tasks preserve different vulnerabilities; hence we use simple yet powerful ensemble methods to boost the robustness. Lastly, we evaluate our methods in different threats to demonstrate their generality.
Training Details. We generally follow data augmentation methods (e.g., jitter and translation) in DGCNN [15] in our study. All pre-trained and fine-tume tnned models in APF are trained using Adam [44]. We use batch sizes of 32 for PointNet and DGCNN, and 128 for PCT. The initial learning rate is set to 0.001 for PointNet and DGCNN, and 5 x 10 for PCT. Both pre-training and fine-tuning take 250 epochs, where 10x decay happens at the 100-th, 150-th, and 200-th epoch. We leverage the same training setups in AJT. All experiments are done on 1 to 4 NVIDIA V100 GPUs [45].
Model Adaptation to AT. First, PointNet [5] leverages exponential learning rate decay, and DGCNN and PCT utilize cosine annealing learning rate decay. Through preliminary experiments, we find that piecewise decay of the learning rate im- proves the default baseline by a noticeable margin. Second, two T-Nets in PointNet [5] will also make AT unstable since they holistically modify the point cloud and features. We thus remove them in our experiments. Third, PCT by default leverage farthest point sampling (FPS) to sample and group local points, which will cause AT unstable since the perturbed point cloud samples different anchor points in each iteration. We replace FPS with EdgeConv each point. It is worth noting that such a change will not modify the application of self-attention in PCT. As presented in Table 12, the modifications will both improve the standard and adversarial training baseline, and we introduce the details in the supplements.
Table I: Summarized Results (%) on Mod- elNet40 by Model Adaptation.
PointNer
DGCNN
PCT
CA
RA
CA
RA
CA
RA
Defiult ST
$8.0
0.0
91.3
2.9
91.9
0.0
Selected ST
88.7
0.3
91.5
5.2
92.1
2,3
Defanlt AT Selected
87.1
33.6
90.6
58.
88.4
44.2
3.2.1 Self-Supervised Pre-training Helps Adversarial Fine-tuning
We systematically evaluate all configurations in APF under PS attack. As introduced earlier, we use standard and adversarial training to get the pre- traited models. From Table 2, we can make several interesting observations. First, we find that our APF strategy generally enhances the adversarial robustness. The best-fine-tuned models achieve 14.2%, 5.4%, and 2.2% robustness improvements in PointNet, DGCNN, and PCT on ModelNet40, respectively. The enhancements on the real-world dataset, ScanObjectNN, i.e., 1.8%, 10.4%, and 6.9% in PointNet, DGCNN, and PCT, are also significant, which demonstrate the generality of APF. Second, we find that DGCNN outstands to be the most robust architecture, consistently achieving 155% stronger robustness than the other two models on both ModelNet40 and ScanObjectNN, Lastly, jigsaw- APF offers more robustness improvements than the other two methods while maintaining slightly higher clean accuracy (CA).
'The 1-st and 2-nd highest accuracy among tuned models in each column are noted, and we use the same mark throughout this paper. CA and RA denote clean and robust accuracy, respectively.
5

---

Table 2: Evaluation Results (%) of Adversarial Pre-training for Fine-tuning and Task Ensembles.
ModeINesO
ScanObjeeNNN
ModelNetto
PointNet
DGCNN
PCT
PoinNeet
DGCNN
PCT
PointNet
DOCNN
PCT
Protest Task
Parameters
|
CA
RA
CA
RA
RA
RA
CA
RA
CA
RA
RA
CA
CA
AT Baseline
NAA
|
87.2
[37,99]
90.6
62.00
89.7
49.1
09.9
2377
74.4
30.91
72.4
(0.5
96.6
79.7
98.1
96.3
97.4
500
3D Kotation
1 6
18
57.2 87.2
48.3
08.0
91.4
91.1
63.6
64.1
90.2
90.2
50.7
49.5
69.5
69.1
24.5
25.0
73.8
75.7
32.9
32.2
72.5
72.6
20.1
20.5
97.1
96.8
79.3
79.0
98.5
97.7
$1.9
85.3
97.8
97.2
50.4
80.3
Advenaniaa
? 6
87.6
42.1
90.8
61.8
90.4
50.8
69.6
25.3
75.0
36.8
71.6
28.7
97.0
79.9
97.7
87.5
98.0
K2.2
3D Rotation
18
57.4
45.7
90.9
02.9
90.4
30.1
09.3
24.5
75.0
36.3
73.1
26.9
97.0
79.7
98.0
58.1
97.4
85.7
3L ligsaw
87.6
50.1
90.0
67.4
51.1
50.2
70.2
79:0
76.2
33.8
72.4
73.8
23.2
24.6
98.0
98.0
Advemariia
58.2
52.1
89.6
65.6
89.5
51.3
69.0
24.8
77.5
41:3
72.5
26.3
97.0
B00.6
95.5
90.5
97.4
83.5
3D ligsow
89.9
$99.6
51.0
40.6
73.1
27.4
97.01
98.0
89.1
97.55
83.9
sphere plane
57.4
50.0
59.9
62.8
90.2
50.7
09.9
25.1
76.1
36.0
71.3
24.1
97.0
S0L.5
98.2
96.3
97.1
90.1
Ausoenceder
$7.1
48.8
90.1
62.2
90.2
50.2
69.1 4
25.5
76.2
35.6
71.1
22.6
96.5
901.5
97.8
$7.6
97.0
90.1
glussian
$7.4
48.9
90.8
63.3
89.7
50.3
69.7
23.8
75.6
35.8
71.3
24.8
96.8
$0.5
97.8
56.4
97.1
80.1
Advensriia
sphere
87.1
49.7
90.0
62:2
90.3
30.0
70.4
25.0
75.2
36.2
72.6
22.2
96.7
80.4
97.5
87.3
97.5
Astoenceder
gaussion
plane
56.9
57.1
46.5
48.5
90.7
89.7
62.7
61.8
90.2
89.7
30.0
30.5
68.8
09.2
24.0
25.0
74.7
75.6
36.
38.0
72.6
73.3
21.5
23.4
97.0
97.0
9016
50..2
97.5
98.0
55.4
56.1
97.4
97.7
$2.5
83.2
Max husecable
$8.5
53.5
91.4
69.4
90.5
55.4
70.8
27.3
79.1
42.6
74.00
28.9
97.2
82.5
98.5
91.0
98.1
85.2
Mean Ensemblle
91.4
57.9
70.8
20.9
79.0
74.2
98.6
98.0
84.9
Specifically, jigsaw-based APF, on average, further boosts DGCNN's robust accuracy (RA) by 2.8%. 2.2%, and 2.7% on three datasets, respectively.
Insights. Different from 2D images that possess both texture and shape information, 3D point clouds naturally bias towards shape. In 2D image space, it is widely recognized that local and global features correspond to the texture and shape information, respectively [46]. Recent studies have demonstrated that appreciation of global/shape features can help improve model robustness on image classification [47]. However, we find some distinctions in point cloud recognition. As mentioned above, PointNet with only global feature learning will be easily affected by the perturbed points (Table 2). Due to the sparsity of point clouds, the local feature actually represents the smoothness of the object's surface. Thus, learning robust local features is critical for correctly recognizing a perturbed point cloud, as it limits the adversarial effect propagation to the model output. We also include a preliminary study on contrastive pre-training [48] and find that it can be viewed as a global feature learning scheme as well. Detailed results can be found in the supplements.
As summarized above, DGCNN achieves the strongest robustness under AT, attributed to the hier- archical usage of EdgeConv [15]. EdgeConv dynamically aggregates local features by exploiting ANN. Such an aggregation method has the ability to calibrate the adversarial effect in the local feature learning stage. Although transformer-based architectures have gained tremendous visibility recently [49], we find that PCT does not have major robustness improvement compared to PointNet. Self-attention increases the capacity of the model architecture, but it also enlarges the receptive field of the model [50]. In PCT, each point can influence every other point's feature, which will potentially increase the model's s fragility [51].
Moreover, we also find that grssawbbase. APF is the most effective method to improve adversarial robustness, aligning well with our above insights. Jigsaw SSL makes the model learn to reassemble the randomly displaced local point clusters, where the model is enforced to learn the displaced local features. Meanwhile, to correctly reconstruct the point cloud, jigsaw SSL also requires the model to capture the global and holistic semantics. Nevertheless, rotation and autoencoder- methods focus more on global feature learning. Therefore, we believe jigsaw-based APF a perfect candidate to strengthen the association between local and global features in point cloud learning, hence improving the adversarial robustness under APF.
pre-training
3.2.2 Adversarial Joint Training Does not Always Improve Robustness
We further analyze the implication of SSL tasks in AJT. As presented in Table 3, AJT can still enhance the robustness in PointNet and DGCNN, For instance, AJT improves their RA by 1.8% and 8.0% on ScanObjectNN, respectively. However, AJT overall cannot outperform APF in point cloud recognition. Especially, we find AJT even degrades the RA of PCT compared to the standard AT.
Insights We find this also to be related to the natural characteristic of point cloud data. Although SSL. can help models learn strong priors and context information, it is still a separate learning task. Rotated and disassembled images still preserve similar local features to the original images since the

---

Table 3: Evaluation Results (%) of Adversarial Joint Training.
ModeINas40
ScanOhjearNN
ModelNertO
PointNer
DGUNN
Poimter
DGUNN
PCT
PointNet
DGCNN
PCT
Protest Tisk
Parameters
CA
RA
CA
RA
CA
RA
CA
RA
CA
RA
CA
RA
CA
RA
CA
RA
CA
RA
Baseline
N/A
87.7
37.9
90.6
66.0
89.7
49.1
69.9
23.7
74.4
30.9
72.4
20.5
96.6
79.7
98.1
97.4
80.0
Rotation
1 18
86.5
8688
62.0
60.7
44.3
67.8
76.2
74.2
37.8
87.2
20.3
97.0
96.6
97.9
98.1
75.6
73.8
3D Tigsow
mm 3
8776
87.2
42.5
46.7
91.0
91.1
61.7
62.3
9012
89.8
43.1
40.9
69.4
70.0
24.6
25.5
75,9
77.1
38.4
38.9
72.1
73.7
20.5
20.7
96.8
96.8
79.8
77.9
98.0
98.4
88.6
$7.9
97.7
97.1
76.8
78.0
sphere plane ganssion
87.5
44.4
90.9
60.1
89.6
49.2
08.9
24:2
75.5
36.5
72.5
20.5
96.7
19.88
98.2
50.3
97.5
80.3
Autoencoder
67.4
42.1
90.7
61.9
89.3
48.7
68.5
23.0
75.6
94-7
72.8
20.6
96.7
70.7
98.1
86-2
97.4
70.9
86.9
43.9
90.9
61.9
88.9
49.2
68.7
24.4
76.1
35.1
72.1
20.5
96.6
79.7
98.3
96.9
97.5
80.0
RGB values do not change, so that the auxiliary optimization in AJT will not distract AT but help models learn robust global features [24]. However, point cloud models take point coordinates zyz as input. Rotated and disassembled point clouds have significant variations in their coordinates' numeric values. Although we apply dual batch normalization [52] migrate the feature heterogeneous problems, such discrimination will consequently distract model learning in AJT, and thus hurt the RA performance. The usage of self-attention in PCT will further expand this impact since it introduces global receptive field [51]. We find that the results of AJT using autoencoders are more stable than the other two tasks. We believe is because the input for the autoencoder is the same as the recognition task so that the distributional gap is small. However, it is still worse than the pre- pre-training scheme.
3.2.3 Robustness against Unforeseen Attacks and Noises
We have so far shown that APF generally improves the robustness of point cloud classification. In this section, we leverage unforeseen attacks to further demonstrate that the enhancements are non-trivial. We select the fine-tuned models with the highest RA in Table 2, and all results here averaged from five runs using different random seeds.
We leverage unforeseen attacks: Auto Attack (AA) [53], Momentum Iterative Method (MIM) [54], and PGD with both cross-entropy and margin loss (i.e., Lmargin Z(x), max Z(x);; where (xx) is the logit output of the target classifier). We elaborate the details of AA (i.e., A-PGD) and MIM in the supplements. All attacks use 200 steps to find the potential adversarial examples. As shown in Figure 3(a) and 3(b), the best fine-tuned models consistently achieve higher RA than models from standard AT with 2.6% 15.3% and 1.0% 10.1% improvements on ModelNet40 and ScanObjectNN, respectively. We find that DGCNN still outperforms the other two models achieving the highest RA even against the strongest Auto Attacks [53].
Moreover, we exploit the transfer attack to confirm the effectiveness of APF. We fine-tune five models using the same (best) setting but different random seeds for transfer-based attacks and test the transferability of generated adversarial examples among different models. As Figure 3(d) and 3()) show, adversarial examples cannot transfer well and there is ~ 100% gap in RA compared to attacks on target models, which further validates the real robustness improvements.
We also find that Gaussian and uniform noises are ineffective in breaking model robustness, and the RA is therefore close to the CA. As shown in Figure 3(d) and 3(e), the fine-tuned models still outperform standard AT baseline by 0. 1% 2.2% and 1.0% - 5.7% on ModelNet40 and ScanObjectNN, respectively, in RA. More evaluation results of our adversarial training strategy are elaborated in the supplements.
3.2.4 Attack Transferability and Task Ensemble
Since we leverage three pre- text tasks for self-supervised pre- training, we also test the trans- ferability of adversarial exam- ples generated by models pre- trained on different SSL tasks. As shown in Figure 4, different fine-tuned models preserve dif- ferent vulnerabilities, and there at least 100 gain of RA when transferring attacks mod-
0.483
0.687
0.967
0.641
0.735
0.778
0.507
0.734
0.724
3.9
0.735
0.323
0.729
0.528
0.67U
0.6993
0.504
0.099
0.659
0.678
0,500
0,750
0.707
0.635
0,780
0.781
0.506
Terges
Target
(a) PointNet.
(b) DGCNN.
(c) PCT.
Figure 4: Robust Accuracy on Transfer Attacks among Fine-tuned Models from Different SSL Tasks on Ne1400
7

---

(a) ModelNet40.
(b) ScanObjectNN,
(c) ModelNet10.
(d) ModelNet40.
(e) ScanObjectNN
ModelNet10.
Figure 3: Evaluation Results of Unforescen Attacks. Plots in the first row present the evaluation results of unforeseen adversarial attacks, where "-M" denotes the attacks on margin loss, Plots in the second row present the evaluation results of noise and transfer attacks.
els fine-tuned from different tasks on ModelNet4O. We thus combine the models and fur- ther test the ensemble model's robustness. We use two ensemble methods, which are tak- ing the max and mean value of the stacked logits from individual models i.e., Lensemble = max htt///ean(LLLaa fotationn L jigsow L Lauutoeeoderr)) where L denotes the i-th value in the logit L. By doing so, form "wider" model where the gradients can still propagate smoothly back to the input. Therefore, we also exploit the 200-step PGD attack to test their robustness. As presented in the last two rows of Table 2, both ensemble models can further boost the RA by a significant margin while maintaining similar clean accuracy. For example, our best ensemble models achieve robust accuracy of 53.5% (+15.6%), 69.4% (+7.4%) and 57.9% (+8.8%) with PointNet, DGCNN, and PCT on ModelNet40 respectively. It is worth noting that our fine-tuning strategy does not enforce a diversity regulation, unlike [25], which further demonstrates that pre-training on different pretext tasks indeed brings distinct context information into pre- rrettrined models.
3.2.5 Robustness against Other Point Cloud Threats
As mentioned before, we also consider two other threats that specifically target point cloud recognition. As we have demonstrated that jigsaw-based APF reaches the best robustness enhancement under point shifting (PS), replace PS attack in AT with salient point dropping (PD) and point adding (PA) to test the effectiveness of jigsaw-base based APF strategy.
Table 4: Evaluation (%) of PD on ScanObjeetNN.
Table 5: Evaluation (%) of PA on ScanObjectNN.
PointNer
DGCNN
PCT
PointNer
DGCNN
PCT
Pretext
Parameters
CA
Parameiers
CA
RA
ST Baseline
N/A
68.8
48.5
79.3
60.9
74.2
$6.1
ST Buseline
N/A
69.8
57.6
79.3
38.0
74.2
39.6
AT Baseline
N/A
70.9
64.2
76.2
77.1
73.3
73.7
AT Raseline
NIA
69.0
59.4
77.4
63:5
72.1
56.8
3D Tigsaw
71.3
81.2
81.4
76.8
76.9
Tigaan
70.4
70.2
59.9
66.3
66.6
75.7
Advenarial
71.1
64.7
78.7
79.5
75.4
76.2
Adversatial
70.4
02.5
77.5
67.8
75.2
64.4
3D Jiguaw
3D ligsaw
We find that based APF can still strengthen the robustness under these two threats on three datasets. Especially, as shown in Table 4 and 5, DGCNN still achieves the highest RA, which on average outperforms PointNet and PCT by 14.8% and 2.3% in PD-AT, and 6.6% and 4.9% in PA-AT, respectively, which is consistent with our previous findings. Moreover, as PD and PA tend to be much weaker adversaries than the PS attack, the enhancements mainly appear in the challenging real-workh dataset, ScanObjectNN, where jigsaw-based based APF boosts the RA of DGCNN by 3.6% and 4.7% under PD and PA. The key insights still hold in ModelNet datasets under these two threats, and we present the detailed results in the supplements due to space constraints.
8

---

3.3 Point Cloud Part Segmentation
In addition to object classification, we conduct the first study on analyzing the robustness of the point cloud part segmentation on ShapeNetPart [42] with 17 classes of 3D objects. Apart from semantic segmentation that all samples from a dataset share the same set of labels, part segmentation task assigns unique label set for each class of objects. For instance, the class of chair has the label set: [seat, back, arm, leg) and the airplane has [wing, body, tail, engine), ). as illustrated in Figure 5. We use the same backbone with a segmentation head for this task and mean nnterseciion-veer-uiion (mloU) [43] as the evaluation metric. Especially since each class possesses its own label set, the models also take the category (one-hot vector) as input. As we find APF is an effective and efficient strategy, we use the PS attack with the same AT and APF setups as the classification task to train the models ($ 3.1).
Table 6: Evaluation Results (%) of the Part Segmentation Task.
back
PoiniNei
DOCNN
PCT
Protect
Parsmeters
C-mlot
R-mtoU
C-mioU
R-mioU
C-mioU
R-mloU
amm
ST Baseline
NA
83.2
32,5
84.2
42.0
84.2
54.6
AT Reseline
N/A
79.2
62.7
82.9
60.5
82.6
67.8
sext
79.1
63.0
82.8
69.3
82.8
67.3
3D Rotizion
18
79.3
63.0
82.9
00.5
82.9
67.5
Adversarial
1
79.3
62.9
83.9
69.2
82.2
67.
6-0.06
log
3D Rotition
17 18
79.1
62.6
82.9
69.0
81.0
65.7
3D ligsaw
A
79.4
79.4
63.1
63.0
82-7
87.9
20.1
70.1
87.4
82.8
68.4
08.1
cannot modify the strong positional pattem of part segmertation.
Figure 5: Illustration of the Point Cloud Part Segmentation Using "Chair".
Adversurial
A
79:7
62.5
$2.8
09.7
$1.1
65.8
3D Jigsaw
79.9
62.7
62.7
70.3
66.7
Table 6² summarizes the quantitative results. To our surprise, we find that part segmentation is more robust task than classification. The robust mIoU is relatively high even in the ST baseline. Moreover, DGCNN still achieves the highest clean and robust mIoU in part segmentation, further confirming its supremacy in point cloud learning. Furthermore, jiggaaa jigsaw- based APF can also promote the robustness of part segmentation by 0.4%, 0.8%, and 0.6% in PointNet, DGCNN, and PCT, respectively.
Insights. We attribute the natural robustness of part segmentation to the intrinsic feature of point clouds. Since all point clouds in ShapeNetPart are sampled from CAD models, they posed to face the same direction. Thus, models may memorize the 3D space corresponding to the specific label during learning. Taking the class of chair an example, the points whose E [-1,0] most likely belong to the "leg" label. However, bbsed adversary with = 0.05 might not be able to perturb this strong pattern (Figure 5). We also envision that the robustness of part segmentation may help and transfer to other complex learning tasks if appropriately used. Moreover, DGCNN with jigsaw-based APF still delivers the best performance, which further generalizes our previous findings on the importance of balancing local and global features in point cloud learning.
4 Related Work
In this section, review a few topics related to the adversarial robustness of 3D perception: 3D deep learning, self-supervised learning, and adversarial attacks and defenses.
4.1 Deep Learning on 3D Point Clouds
In image-based perception, there has been stellar progress on architectures and learning algorithms for convolutional neural networks. However, in 3D perception, there has been no consensus on the type of models for 3D data partially because there is no standard data format for 3D perception. Yet, point clouds are the most commonly used data format because raw data from both 3D scanners and triangular meshes can easily be converted into a point cloud. Some of the earlier 3D networks use dense voxel grids for perception [1, 21,2,55,66 56), which discretize a point cloud to voxel grids for classification, segmentation, and object detection. Later, to overcome cubic memory complexity, various types of data structures have been proposed: octrees [3, 4], surfaces [57], graphs 588], point sets [5, 14, 17], and sparse tensors [6, 16). In this work, we focus on 3D point cloud perception with PointNet [5], DGCNN [15], and PCT [17] our evaluation backbones since they are widely used and achieve state-of-ther the-art results for point cloud recognition [59].
CC-mloU and R-mloU denote clean and robust mean interseettion-vve--uiiee respectively.

---

4.2 Self-Supervised Learning Approaches
Self-supervised unsupervised learning (SSL) techniques have shown effective in learning powerful representation without manual efforts, where the labels are generated from the data itself [60-62]. The embeddings pre-trained through self-supervision could be utilized for fine-tuning multiple downstream supervised tasks with better generalization and calibration [63, 64]. Because of the impressive success of SSL in 2D computer vision [34], recent studies have also proposed several self-supervisions for 3D point clouds including autoencoders [29], generative adversarial networks (GANs) [36]. 3D rotation [27], and 3D jigsaw [28]. Those primitives hold promises to improve the clean accuracy of point cloud learning. This work systematically explores whether and how SSL could be leveraged to improve the adversarial robustness in point cloud recognition.
4.3
Adversarial Attacks and Defenses
Despite the accomplishments that DNNs have achieved, adversarial attacks [65] are becoming the major obstacle in real-world deep learning deployments, especially in safety-critical areas [11, 12, 66- 69]. Numerous attacks have been widely studied for various tasks in the 2D [70-76] and 3D [13, 77- 79] domains. To address this problem, many defense methods have been proposed to enhance the robustness against ad adversarial attacks in the 2D [80-88] and 3D domains [19-21]. However, most of them including adding randomization [20,89,90 90], model distillation [83], adversarial detection [84], and input transformation [19,80 80-82, 91] have been compromised by adaptive attacks 121, 92]. Certified methods are recently applied to 3D, but they focus on threat models with modified number of points [93] and isometric transformations [94]. which are inapplicable to &o norm. Adversarial training (AT) [23, 38, 39, 95,,, on the other hand, provides one of the most longstanding and practical defenses. Various methods have been proposed to improve AT in the 2D domain [52, 96, 97]. Jeddi et al. establish fast adversarial fine-tuning scheme [98]. Hendrycks et al. [24] have shown to jointly train the adversarial loss and SSL loss to improve the robustness. Chen et al. [25] have applied adversarial pre- pre-training and fine-tuning strategy to improve the robustness. In this work, we explore to apply and improve AT in 3D point cloud recognition.
5 Conclusion
In this work, we systematically explore the impact of self- sepervueee learning (SSL) on the adversarial robustness 3D point cloud recognition. We find tangible robustness improvements by the adversarial pre- pre-training for fine-tuning strategy. We also experimentally show that robust local features are critical to achieving robustness in 3D, explaining the success of DGCNN and the jigsaw proxy task. Our results shed light for future research on designing more robust models and SSL schemes for 3D point clouds. By providing empirical evaluations on the robustness, our study also motivates future studies in developing theoretical guarantees on the robustness in the 3D domain.
Acknowledgements
We appreciate our area chairs and anonymous reviewers for their insightful comments. We thank Qingzhao Zhang for proofreading our manuscript. Jiachen Sun thanks Zhao Su for her considerate care and help during COVID-19. This project is partially supported by NSF grants CMMI-2038215 and CNS-193004|
References
[1] Dominic Zeng Wang and Ingmar Posner. Voting for voting in online point cloud object detection. In Robotics:
Science and Systems, volume 1, pages 10-15607. Rome, Italy, 2015.
[2] Daniel Maturana and Sebastian Scherer. Voxnet: 3d convolutional neural network for real-time object recognition. In 2015 JEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 922-928. IEEE, 2015.
[3] Gernot Riegler, Ali Osman Ulusoy, and Andreas Geiger. Octnet: Learning deep 3d representations high resolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.
[4] Peng-Shuai Wang, Yang Liu, Yu-Xiao Guo, Chun- Yu Sun, and Xin Tong. O-enn: Octrec- based con- volutional neural networks for 3d shape analysis. ACM Transactions Graphics (TOG), 36(4):1-11, 2017.
[5] Charles R Qi, Hao Su. Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 652-660, 2017.
10

---

[6] Christopher Choy, Jun Young Gwak, and Silvio Savarese. 4d spatio-temporal convnets: Minkowski convolutional neural networks. In Proceedings the IEEE Conference on Computer Vision and Pattern Recognition, pages 3075-3084, 2019.
[7] Tianwei Yin, Xingyi Zhou, and Philipp Krähenbühl. Center-based 3d object detection and tracking. CVPR, 2021.
[8] Shaoshuai Shi, Xiaogang Wang, and Hongsheng Li. Pointrenn: 3d object proposal generation and detection from point cloud. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 770-779, 2019.
[9] Shaoshuai Shi, Chaoxu Guo, Li Jiang. Zhe Wang, Jianping Shi, Xiaogang Wang, and Hongsheng Li. Pv-remm: Point-voxel feature set abstraction for 3d object detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10529-1055 2020.
[10] Charles R Qi, Li Yi, Hao Su, and Leonidas Guibas. Pointnct++: Deep hierarchical feature learning on point sets metric space. arXiv preprint 1706:02413 2017.
[11] Yulong Cao, Chaowei Xiao, Benjamin Cyr, Yimeng Zhou, Won Park, Sara Rampazzi, Qi Alfred Chen, Kevin Fu, and Z Morley Mao. Adversarial sensor attack on lidar based perception in autonomous driving. In Proceedings of the 2019 ACM SIGSAC conference computer and communications security, pages 1267-22811 2019.
[12] Jiachen Sun, Yulong Cao, Qi Alfred Chen, and Z. Morley Mao. Towards robust lidar-based perception in autonomous driving: General black-box adversarial sensor attack and countermeasures. In 29th USENIX Security Symposium (USENIX Security 20), pages 877-894, USENIX Association, August 2020.
[13] Chong Xiang, Charles R Qi, and Bo Li. Generating 3d adversarial point clouds. In Proceedings of the IEEFJCVF Conference on Computer Vision and Pottern Recognition, pages 9136-9144, 2019.
[14] P. Hermosilla, T. Ritschel, P-P Vazquez, A. Vinacua, and T. Ropinski. Monte carlo convolution for learning on non-uniformly sampled point clouds. ACM Transactions on Gruphics (Proceedings of SIGGRAPH Asia 2018), 37(6), 2018.
[15] Yue Wang. Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon. Dynamic graph cnn for learning on point clouds. Acm Transactions On Graphics (tog), 38(5):1-12, 2019.
[16] Benjamin Graham and Laurens van der Maaten. Submanifold sparse convolutional networks. arXiv preprint XrXi::706.000 703,01307, 2017.
[17] Meng-Hao Guo, Jun-Xiong Cai,, Zheng-Ning Liu, Tai-Jiang Mu, Ralph R Martin, and Shi-Min Hu. Pet: Point cloud transformer. arXiv preprint arXiv:2012. 2.6688, 2020.
[18] Hengshuang Zhao, Li Jiang, Jiaya Jia, Philip Torr, and Vladlen Koltun. Point transformer. arXiv preprint 20l.09900909166 2020.
[19] Hang Zhou, Kejiang Chen, Weiming Zhang, Han Fang. Wenbo Zhou, and Nenghai Yu. Dup-net: De- noiser and upsampler network for 3d adversarial point clouds defense. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1961-1970, 2019.
[20] Xiaoyi Dong, Dongdong Chen, Hang Zhou, Gang Hua, Weiming Zhang, and Nenghai Yu. Self-robust 3d point recognition via gather-vector guidance. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11513-11521. IEEE, 2020.
[21] Jiachen Sun, Karl Koenig, Yulong Cao, Qi Alfred Chen, and Z. Morley Mao. On the adversarial robustness of point cloud classification, 2020.
[22] Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In International Conference on Machine Learning, pages
283. PMLR, 2018.
[23] Alcksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv: 06083, 2017.
[24] Dan Hendrycks, Mantas Mazeika, Saurav Kadavath, and Dawn Song. Using self-supervised learning improve model robustness and uncertainty. arXiv preprint 1.XXvv1906.11344 2019.
[25] Tianlong Chen, Sijia Liu, Shiyu Chang, Yu Cheng, Lisa Amini, and Zhangyang Wang. Adversarial robustness: From self-supervised pre-training to fine-tuning, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 699-708, 2020.
[26] Charles R Qi, Yin Zhou, Mahyar Najibi, Pei Sun, Khoa Vo, Boyang Deng, and Dragomir Anguelov. Offboard 3d object detection from point cloud sequences. arXiv preprint arXiv:2103.0555 2021. [27] Omid Poursaeed, Tianxing Jiang, Quintessa Qiao, Nayun Xu, and Vladimir G Kim. Self-supervised learning of point clouds via orientation estimation. arXiv preprint arXiv:2008.0 00305, 2020.
[28] Jonathan Sauder and Bjarne Sievers. Self supervieed deep learning on point clouds by reconstructing space. arXiv preprint arXiv: 0009000.833.00000
11

---

[29] Yaoqing Yang. Chen Feng, Yiru Shen, and Dong Tian. Foldingnet: Point cloud auto-encoder via deep grid deformation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 206-215, 2018.
[30] Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 3d shapenets: A deep representation for volumetric shapes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1912- 1920, 2015.
[31] Alex H Lang- Sourabh Vora, Holger Caesar. Lubing Zhou, Jiong Yang. and Oscar Beijbom. Pointpillars: Fast encoders for object detection from point clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12697-12705, 2019.
[32] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. arXiv preprint arXiv:1706. 170.:1770..3362.. 2017.
[33] Tianhang Zheng, Changyou Chen, Junsong Yuan. Bo Li, and Kui Ren. Pointeloud saliency maps. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages
2019.
[34] Spyros Gidaris, Praveer Singh, and Nikos Komodakis. Unsupervised representation learning by predicting image rotations, arXiv preprint
2018.
[35] Mehdi Noroozi and Paolo Favaro. Unsupervised learning of visual representations by solving jigsaw puzzles. European conference on computer vision, pages 69 84. Springer, 2016.
[36] Panos Achlioptas, Olga Diamanti. Ioannis Mitliagkas, and Leonidas Guibas. Learning representations and generative models for 3d point clouds. In International conference on machine learning, pages 40-49.
PMLR, 2018.
[37] Gunilla Borgefors. Distance transformations in arbitrary dimensions. Computer vision, graphics, and image processing, 27(3) 334,, 1984.
[38] Eric Wong, Leslie Rice, and J Zico Kolter. Fast is better than free: Revisiting adversarial training. arXiv preprint arXiv:2001 03994, 2020.
[39] Ali Shafahi, Mahyar Najibi. Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer. Larry Davis. Gavin Taylor, and Tom Goldstein. Adversarial training for free! arXiv preprint 1rXiv:1904.12865 2019.
[40] Cihang Xie, Mingxing Tan, Boqing Gong, Jiang Wang, Alan L Yuille, and Quoe V Le. Adversarial examples improve image recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pages 819 9288 2020.
[41] Mikaela Angelina Uy. Quang- Hieu Pham, Binh-Son Hua, Thanh Nguyen, and Sai- Kit Yeung. Revisiting point cloud classification: A new benchmark dataset and classification model on real-world data. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1588-1597, 2019.
[42] Li Yi. Vladimir G Kim, Duygu Ceylan, I-Chao Shen, Mengyan Yan, Hao Su, Cewu Lu, Qixing Huang, Alla Sheffer, and Leonidas Guibas. A scalable active framework for region annotation in 3d shape collections. ACM Transactions on Graphics (ToG), 35(6):1-2,, 2016.
[43] Angel X Chang. Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. Shapenet: An information-ric 3d model repository.
arXiv preprint arXiv: 15/2.03012, 2015.
[44] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint
arXiv: 1xXiv:4412.666 1411.6980,. 2014.
[45] NVIDIA V100 TENSOR CORE GPU, https://www..viieee www.e.coooooeeaeeeeeee en- conn-s/dataaa. htt///ttaaacetttevoo/..o. 2020.
[46] Anh Bui, Trung Le, He Zhao, Paul Montague, Olivier Vel, Tamas Abraham, and Dinh Phung. Improving adversarial robustness by enforcing local and global compactness. arXiv preprint arXiv:2007.05122 2020.
[47] Prasad Chalasani, Jiefeng Chen, Amrita Roy Chowdhury, Somesh Jha, and Xi Wu. Concise explanations of neural networks using adversarial training. arXiv, pages arXiv-1810, 2018
[48] Saining Xie, Jiatao Gu, Demi Guo, Charles R Qi, Leonidas Guibas, and Or Litany. Pointcontrast: Unsupervised pre-training for 3d point cloud understanding. In European Conference on Computer Vision, pages 574 591. Springer, 2020.
[49] Alexey Dosovitskiy. Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai Thomas Unterthiner, Mostafa Dehghami, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010. //929, 2020.
[50] Han Zhang, lan Goodfellow, Dimitris Metaxas, and Augustus Odena. Self-attention generative adversarial networks. International conference on machine learning, pages 7354-73646 PMLR, 2019.
[51] Chong Xiang, Arjun Nitin Bhagoji, Vikash Sehwag, and Prateek Mittal. Patchguard: Provable defense against adversarial patches using masks small receptive fields. arXiv preprint arXiv:2005 1088,, 2020.
12

---

[52] Cihang Xie and Alan Yuille. Intriguing properties of adversarial training at scale. In International Conference on Learning Representations, 2020.
[53] Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International Conference on Machine Learning, pages 2206 2216. PMLRR 2020.
[54] Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Xiaolin Hu, and Jun Zhu. Discovering adversarial examples with momentum. arXiv preprint arXiv:1710.06088 2017.
[55]
Shuran Song and Jianxiong Xiao. Deep Sliding Shapes for amodal 3D object detection in RGB-D images. In CVPR, 2016.
[56] Lyne P. Tchapmi, Christopher B. Choy, Iro Armeni, Jun Young Gwak, and Silvio Savarese. Segcloud: Semantic segmentation of 3d point clouds. In International Conference on 3D Vision (3DV), 2017.
[57] Maxim Tatarchenko, Jaesik Park, Vladlen Koltun, and Qian- Yi Zhou. Tangent convolutions for dense prediction in 3d. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3887-3896, 2018
[58] Hugues Thomas, Charles R Qi, Jcan-EEmaauul Deschaud, Beatriz Marcotegui, François Goulette, and Leonidas J Guibas. Kpconv: Flexible and deformable convolution for point clouds. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 6411-6420, 2019. www.ootwwooooooo.eooooooe com/sota/
[59] 3D Point Cloud Classification Benchmark on ModelNet40. 3d- p--ppoonttlloo cloud- www.sslllaaaiiiooo on- modemmete0,, 2021.
[60] Alexander Kolesnikov, Xiaohua Zhai, and Lucas Beyer. Revisiting self-supervised visual representation learning. In Proceedings of the IEEF/CVF Conference on Computer Vision and Puttern Recognition, pages 1920-1929, 2019.
[61] Honglak Lee. Roger Grosse, Rajesh Ranganath, and Andrew Y Ng. Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In Proceedings of the 26th annual international conference on machine learning, pages 609-616, 2009.
[62] Carl Doersch, Abhinav Gupta, and Alexei A Efros. Unsupervised visual representation learning by context prediction. In Proceedings of the IEEE international conference on computer vision, pages 1422-1430. 2015.
[63] Dumitru Erhan, Aaron Courville, Yoshua Bengio, and Pascal Vincent. Why does unsupervised pre- pre- training help deep learning? In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pages 201-208. JMI. Workshop and Conference Proceedings, 2010.
[64] Ishan Misra and Laurens van der Maaten. Self-supervised learning of pretext-invarian representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6707-6717, 2020.
[65] Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 2013.
[66] Kevin Eykholt, Ivan Eytimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno, and Dawn Song. Robust physical world attacks on deep learning visual classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1634, 2018.
[67] Yulong Cao, Chaowei Xiao, Dawei Yang, Jing Fang, Ruigang Yang, Mingyan Liu, and Bo Li. Adversarial objects against lidar- based autonomous driving systems. arXiv preprint arXiv:1907.00 1907005418, 2019.
[68] Haonan Qiu, Chaowei Xiao, Lei Yang, Xinchen Yan, Honglak Lee, and Bo Li. Semanticadv: Generating adversarial examples via autributeccondition image editing. In European Conference on Computer Vision, pages 19 37. Springer, 2020.
[69] Yulong Cao, Ningfei Wang, Chaowei Xiao, Dawei Yang, Jin Fang, Ruigang Yang, Qi Alfred Chen, Mingyan Liu, and Bo Li. Invisible for both camera and lidar: Security of multi-sensor fusion based perception in autonomous driving under physical-world attacks. In 2021 IEEE Symposiun on Security and Privacy (SP), pages 176-194. IEEE, 2021.
[70] Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp). pages 97. IEEE, 2017.
[71] Chaowei Xiao, Bo Li, Jun-Yan Zhu, Warren He, Mingyan Liu, and Dawn Song. Generating adversarial examples with adversarial networks. arXiv preprint arXiv: 0026100000,0......00000000000110 2018
[72] Chenglin Yang, Adam Kortylewski, Cihang Xie, Yinzhi Cao, and Alan Yuille. Patchattack: A black- texture- based attack with reinforcement learning. In European Conference on Computer Vision, pages 681-698. Springer, 2020.
13

---

[73] Cihang Xie, Jianyu Wang, Zhishuai Zhang, Yuyin Zhou, Lingxi Xie, and Alan Yuille. Adversarial examples for semantic segmentation and object detection. In International Conference on Computer Vision. IEEE, 2017.
[74] Lifeng Huang. Chengying Gao, Yuyin Zhou, Cihang Xie, Alan Yuille, Changqing Zou, and Ning Liu. Universal physical camouflage attacks object detectors, 2019.
[75] Lifeng Huang. Chengying Gao. Yuyin Zhou, Cihang Xie, Alan L Yuille, Changqing Zou, and Ning Liu. Universal physical camouflage attacks on object detectors. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition. pages 720-729, 2020.
[76] Chaowei Xiao, Jun- Yan Zhu, Bo Li, Warren He, Mingyan Liu, and Dawn Song. Spatially transformed adversarial examples. arXiv preprint arXiv: 1001: 026112 2018.
[77] Yuxin Wen, Jiehong Lin, Chen, and Kui Jia. Geometry aware generation adversarial and cooperative point clouds. 2019.
[78] Abdullah Hamdi, Sara Rojas, Ali Thabet, and Bernard Ghanem. Advpe: Transferable adversarial per- turbations on 3d point clouds. In European Conference on Computer Vision. pages 241-257. Springer. 2020.
[79] Chaowei Xiao, Dawei Yang, Bo Li, Jia Deng, and Mingyan Liu. Meshadv: Adversarial meshes for visual recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6898 6907, 2019.
[80] Yuzhe Yang, Guo Zhang, Dina Katabi. and Zhi Xu. Me-net: Towards effective adversarial robustness with matrix estimation, arXiv preprint urXiv:1905. 11971,, 2019.
[81] Weilin Xu, David Evans, and Yanjun Qi. Feature squeezing: Detecting adversarial examples in deep neural networks. arXiv preprint arXiv:1704. 01 155, 2017.
[82] Mitali Bafna, Jack Murtagh, and Nikhil Vyas. Thwarting adversarial examples: An L_O-robustsparse fourier transform. arXiv preprint 111.:1111..000..3. 2018.
[83] Nicolas Papernot, Patrick McDaniel, Xi Wu. Somesh Jha, and Ananthram Swami. Distillation as defense to adversarial perturbations against deep neural networks. In 2016 IEEE symposium on xecurity and privacy (SP), pages 582-597. IEEE, 2016.
[84] Dongyu Meng and Hao Chen. Magnet: a two-pronged defense against adversarial examples. Proceedings of the 2017 ACM SIGSAC conference on computer and communications security, pages 135-147, 2017.
[85] Huan Zhang. Hongge Chen, Chaowei Xiao, Sven Gowal. Robert Stanforth, Bo Li, Duane Boning, and Cho-Jui Hsieh. Towards stable and efficient training of verifiably robust neural networks. arXiv preprint arXiv:1906.0633 2019.
[86] Chaowei Xiao, Ruizhi Deng. Bo Li. Fisher Yu, Mingyan Liu, and Dawn Song. Characterizing adversarial examples based on spatial consistency information for semantic segmentation. In Proceedings of the European Conference on Computer Vision (ECCV), pages 117-234,222 2018.
[87] Huan Zhang, Hongge Chen, Chaowei Xiao, Bo Li. Duane S Boning, and Cho-Jui Hsieh. Robust deep reinforcement learning against adversarial perturbations on observations. 2020.
[88] Chaowei Xiao. Ruizhi Deng, Bo Li, Taesung Lee, Benjamin Edwards, Jinfeng Yi, Dawn Song, Mingyan Liu, and Ian Molloy. Advit: Adversarial frames identifier based on temporal consistency in videos. In Proceedings of the JEEE/CVF International Conference on Computer Vision, pages 3968-3977, 2019.
[89] Daniel Liu, Ronald Yu, and Hao Su. Extending adversarial attacks and defenses to deep 3d point cloud classifiers. In 2019 IEEE International Conference on Image Processing (ICIP), pages 2279-2283 IEEE, 2019.
[90] Guneet S Dhillon, Kamyar Azizzadenesheli, Zachary C Lipton. Jeremy Bernstein, Jean Kossaifi, Aran Khanna, and Anima Anandkumar. Stochastic activation pruning for robust adversarial defense. arXiv preprint arxiv: 1003vv 803,01442 2018.
[91] Nicolas Papernot and Patrick McDaniel. Extending defensive distillation. arXiv preprint arXiv: 1705.55544, 2017.
[92] Florian Tramer, Nicholas Carlini, Wieland Brendel, and Aleksander Madry. On adaptive attacks to adversarial example defenses. arXiv preprint arXiv:2002. 2020.
[93] Honghin Liu, Jinyuan Jia, and Neil Zhenqiang Gong. Pointguard: Provably robust 3d point cloud classification. arXiv preprint (0X.::2003.03000 2021.
[94] Tobias Lorenz, Anian Ruoss, Mislav Balunović, Gagandeep Singh, and Martin Vechev. Robustness certification for point cloud models. arXiv preprint arXiv:2103. 2021.
[95] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint 1412 2014.
14

---

[96] Cihang Xie. Mingxing Tan, Boqing Gong, Alan Yuille, and Quoe V Le. Smooth adversarial training. arXiv preprini arXiv:2006.14553 14536, 2020.
[97] Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent Ghaoui, and Michael Jordan. Theo- retically principled trade-off between robustness and accuracy. In International Conference on Machine Learning, pages 7472 7482. PMLR, 2019.
[98] Ahmadreza Jeddi. Mohammad Javad Shafiee, and Alexander Wong. simple fine-tuning is all you need: Towards robust deep learning via adversarial fine-tuning. arXiv preprint arXi::00/2. 2020.
15


Reviews:
[{"review_id": "cL_i2PmQacT", "reviewer": "Reviewer_Dx1d", "summary": "The paper systematically studies the impact of various self-supervised learning proxy tasks on different architectures and threat models for 3D point clouds. Experiments on three datasets imply that self-supervised learning (SSL), as a pretraining task, improves the robustness of the 3D point-cloud-based model for classification. It is observed that DGCNN and the 3D jigsaw SSL task are more effective w.r.t. robustness, which is claimed to benefit from more robust local features.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strongness:\n+ Comprehensive experiments on multiple datasets\n+ The overall message is clear: DGCNN and the 3D jigsaw SSL task are more effective w.r.t. robustness.\n\n---\n\nWeakness and Questions:\n1. The clarity of the paper can be improved. \n    - L164: $D_t$ is not well defined yet. I assume that it is a data point indexed by $t$.\n    - There are many abbreviations, e.g. ST, AT, CA, etc. It is better to highlight those words where they are defined, so that the readers are able to quickly localize their meanings when they read the experiment section. Alternatively, the definitions of those abbreviations can be emphasized again in the table or figure captions. Besides, the definition of RA seems to be missing, while I assume that it stands for the accuracy after the attack.\n    - (minor) The overview paragraph of each section is not informative enough currently. Although it actually describes the following content, it fails to guide me when I first read it.\n2. From Table 2, it seems that the impact of SSL (self-supervised learning) and AT (adversarial training) are not complementary. For example, AT + SSL (jigsaw) seems to perform similarly or even worse than SSL only. Can the authors explain it? \n3. Is there any data augmentation in ST and AT baseline? If no, can the authors add baselines with data augmentation similar to your SSL tasks (rotation and jigsaw)?\n4. It is questionable whether the contribution of the paper meets the standard of NeurIPS. From my perspective, the paper focuses on scientific observations by comparing different existing network architectures and pretext tasks w.r.t robustness. According to the related works of the paper, there are no significant technical contributions. However, the observations proposed by this paper are interesting: learning robust local features are important. Despite the comparison with PointNet and PCT, the lack of deeper analysis on local features is a drawback. For example, a common practice of SSL is contrastive learning, and there are several works about 3D SSL (like PointContrast), which also focus on local feature learning. More pretext tasks relevant to local feature learning can be included to verify the point. Besides, the autoencoder (shape reconstruction) can also rely on local features, like NSFA(Detail preserved point cloud completion via separated feature aggregation.), which can provide an apple-to-apple comparison about the impact of local feature learning.\n\n---\n\nTypo:\nL201: default -> by default\n\n---\n**Update after rebuttal**\nAccording to the additional results presented in the rebuttal, I slightly raise my score to 6 (Marginally above the acceptance threshold), and hope to see the additional results are included in the revision.\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors have not adequately addressed the limitations.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "LHcHRoUQIre", "reviewer": "Reviewer_CMgR", "summary": "This paper presents studies on exploring the impact of various self-supervised learning tasks on three different architectures and three different threat models for 3D point clouds:\n(1) three primary backbone architecture: a shared multi-layer-perceptron-based network PointNet, DGCNN, and a transformer-based network Point Cloud Transformer (PCT) [17];\n(2) three threat models: point shifting (PS), point dropping (PD), and point adding (PA); and\n(3) three self-supervised learning pretext task: 3D Rotation (predict 3D rotation angles), 3D Jigsaw, and Autoencoder (point cloud reconstruction).\nThen, two strategies were used to integrate self-supervised learning and adversarial training, including (1) adversarial pre-training for fine-tuning (APF), which uses the SSL tasks only for pre-training, and (2) adversarial joint training (AJT), which jointly trains the SSL task with the recognition task.\n\nOverall, the contribution of this paper is not on the method (Section 2) but on the experiments that compare different architectures, strategies, and models, and then show that what combinations will lead to better robustness.\n\nIn short, the experiment considers four datasets (ModelNet40, ModelNet10, ScanObjectNN, and ShapeNetPart) and two tasks: classification and segmentation.", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Major weaknesses:\n\n(1) The technical novelty is weak, as the contribution is more on the experiments and results rather than the method.\n\n(2) For the threat models:\n- From Figure 2, Point addition (PA) seems to be just adding random points near the surface, so the attack looks weak.\n- Line 176 says that a larger number of modifications would be easy to detect. This argument sounds strange, cause this work is to explore how AT (adversarial training) can help to improve the robustness of the trained network, so dropping just 10% points does not sound sufficient.  It is better to perform the experiment by trying different (and larger) amount of point drops. \n- The point drop is only 10%; doing so does not really affect global tasks, particularly classification. See Fig.5 in this paper: \"SampleNet: Differentiable Point Cloud Sampling\".  Dropping large proportion of points (>10%) still can't affect the classification accuracy.  Therefore, the setting for PD looks too weak, particularly for the classification experiment.\n- Also, from Figure 2, it seems that all the salient points were on the bottom of the chair. Does it make sense? Why not consider existing methods for 3D key point detection?\n\n(3) About the third architecture, i.e., PCT:\n- The description on PCT [17] is too short and not informative in the paper. From supp., it seems that this PCT is developed based on EdgeConv (DGCNN) with k=32 (?).  Since this paper [17] is not published yet, should motivate why this is a good point transformer to pick, since there are other point transformers on arxiv, e.g., https://github.com/Strawberry-Eat-Mango/PCT_Pytorch and https://github.com/lucidrains/point-transformer-pytorch\n- But in fact, why not also try other point transformers at the same time and see if you have consistent results?  The result in Section 3.1 seems to suggest that DGCNN is better than PCT [17], but I wonder if it is also true for other transformers (see above).\n\n(4) I can see that the authors have tried their best to discuss the \"insight\" in each experiment but it is still very hard to understand the connections between the experimental results and the underlying reasons.\n\nOverall, my rating is more borderline.\n\n======================================================\n\nOther comments:\n\nLine 69: please use the full word for AT here. It was defined too early in line 33.\n\nLine 177: what is the meaning of 14,5?  14 or 5?\n\nMissing reference:\n- Adversarial robustness: From self-supervised pre-training to fine-tuning. CVPR 2020.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Even though this is an exploratory paper that focuses on the experiments, the authors better discuss the limitations of the experiments.", "review_text": "", "extra_scores": {"ethical_concerns": "Nil.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "457J04Rtjgo", "reviewer": "Reviewer_euZV", "summary": "This paper systematically analyzes robustness of self-supervised adversarial training against adversarial attacks, which conducts thorough evaluation on several representative point cloud classifiers with different types of adversaries. Beyond point cloud classification, the task of part segmentation is also compared and analyzed. ", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper reveals positive impact of adversarial training on improving adversarial robustness of multiple point cloud classifiers. I really appreciate that the authors conduct a thorough evaluation under different settings. The observation that robust local features are vital to achieve adversarial robustness is interesting and also meets our common knowledge.  ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The following concerns can be important to further improve the paper:\n1. There exists a large number of abbr. to make the paper very difficult to digest. Only the important and popular terms are suggested to use abbr.. More importantly, the definition of performance metrics CA and RA is missing in the paper. I suppose CA and RA could refer to clean accuracy and robust accuracy, but their forms remain unclear to the reviewer.\n\n2. Technically, the threat models such as PS, PD, and PA and their experimental settings are important and sensitive to adversarial robustness. However, generation of adversaries should be well investigated and evaluated, which is missing in the experiments. For example, how existence of outliers affect adversarial robustness?\n\n3. The APT performs better than AJT in experiments. The authors claim that distribution gap between SSL and original recognition tasks leads to poor classification performance of AJT. However, in UDA on point cloud classification, SSL is verified its effectiveness to mitigate domain gap (see [a]). Can you explain different performance of your paper and [a]?\n[a] I.  Achituve, H. Maron, and G. Chechik. Self-supervised learning for domain adaptation on point cloud. CVPR 2021. \n\n4. Insight given in the experiments of part segmentation encourages a more interesting SO(3)/SO(3) setting used to evaluate adversarial robustness of rotation-equivariant methods. \n", "review_text": "", "extra_scores": {"ethical_concerns": "N/A", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "8"}, "extra_text": {}}, {"review_id": "2bqwzTbGWk", "reviewer": "Reviewer_ygsD", "summary": "This work builds on recent adversarial attacks on 3D point clouds and recent self-supervised methods on 3D ( 3D jigsaw[28] and 3D rotation prediction[27]) to evaluate the robustness of the 3D models that were trained in such SSL setup. Comprehensive experiments are conducted on 3 different benchmarks in 3D classification and one in 3D part segmentation that show a marginal improvement in robustness by adding SSL pre-training before performing a standard adversarial training. Finally, the authors perform an ensemble of different robust models to improve the robustness of 3D point cloud classification.  ", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Strengths** :\n\n- The work provides comprehensive 3D classification experiments on several 3D datasets, including the realistic ScanObjectNN. \n\n- The first work to investigate the effect of self-supervision on 3D deep learning robustness, as well as studying the robustness of 3D segmentation.\n\n\n**Weaknesses**:\n\n- The paper lacks technical novelty. All the SSL and adversarial attacks and defenses are developed in previous works, which makes the manuscript more or less an evaluation paper.  The proposed Joint SSL adversarial Training (AJT) is shown in the paper to be less effective than the naive SSL pre-training (APF). Performing an ensemble of robust models is not novel since it was shown before in [b]\n\n- One of the findings of the paper ( that DGCNN is more robust than other architectures) was already found in [73] in their comprehensive study of different 3D architectures, including DGCNN. \n\n- The work has a flawed experimentation setup. While the authors did show improvement in robustness, there are some caveats in the setup. The authors used a single $\\epsilon$ value for the PGD attack ( 0.005) based on a rejected ICLR’21 submission [21]. Published works (like [12,19,73]) usually report several $\\epsilon$ values for PGD attack or run on C&W formulation to make sure that the defense/attack methods work on generic attacks and not for that specific $\\epsilon$ ( that might be too small for the attack to work).\n\n- The uncertainty in the results is not quantified. It is not clear whether the small improvements over AT baseline ( that can be as low as 1%) are statistically significant. Reporting the variance of the results is highly appreciated in this scenario. \n\n\n\n**Minor issues** :\n\n- Some grammar mistakes need to be fixed ( line 117 e.g. )\n\n- The paper misses recent work on 3D attacks [a], and ensemble defenses [b].\n\n- The presentation of the paper can be improved. For example, Table 2  has 323 reported results, which is very difficult to follow.  Using plots, charts, and other visualizations can improve the presentation of the paper. The text size should be consistent in all of the figures and plots. Fig 3 has tiny legends for example.\n\n\n\n[a] Tsai et.al, Robust adversarial objects against deep learning models. AAAI Conference on Artificial Intelligence (2020)\n\n[b]Florina et.al,  Ensemble Adversarial Training: Attacks and Defenses, ICLR (2018)\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Adding a section about the vulnerabilities of 3D deep learning methods in safety-critical applications ( like self-driving cars systems) would be appreciated.  ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}]

Author rebuttal:

Meta-review: I recommend acceptance. This is solid experimental work on an interesting and relevant question: the adversarial robustness properties of the most popular 3D point cloud architectures. The question is relevant as a lot of follow up work and engineering efforts are built on these and similar architectures. Even though I accept some reviewer’s concerns that the paper might be thin on novelty, my opinion is that this solid experimental study and its findings will be of interest to the community. 
Decision: Accept (Poster)