O N T HE A DVERSARIAL R OBUSTNESS OF 3D P OINT C LOUD C LASSIFICATION
Anonymous authors
Paper under double-blind review
A BSTRACT
3D point clouds play pivotal roles in various safety-critical ﬁelds, such as autonomous driving, which desires the corresponding deep neural networks to be robust to adversarial perturbations.
Though a few defenses against adversarial point cloud classiﬁcation have been proposed, it remains unknown whether they can provide real robustness.
To this end, we perform the ﬁrst security analysis of state-of-the-art defenses and design adaptive attacks on them.
Our 100% adaptive attack success rates demonstrate that current defense designs are still vulnerable.
Since adversarial training (AT) is believed to be the most effective defense, we present the ﬁrst in-depth study showing how AT behaves in point cloud classiﬁcation and identify that the required symmetric function (pooling operation) is paramount to the model’s robustness under AT.
Through our systematic analysis, we ﬁnd that the default used ﬁxed pooling operations ( e.g., MAX pooling) generally weaken AT’s performance in point cloud classiﬁcation.
Still, sorting-based parametric pooling operations can signiﬁcantly improve the models’ robustness.
Based on the above insights, we further propose DeepSym , a deep symmetric pooling operation, to architecturally advance the adversarial robustness under AT to 47.0% without sacriﬁcing nominal accuracy, outperforming the original design and a strong baseline by 28.5% ( ∼ 2 . 6 × ) and 6.5%, respectively, in PointNet.
I NTRODUCTION
Despite the prominent achievements that deep neural networks (DNN) have reached in the past decade, adversarial attacks (Szegedy et al., 2013) are becoming the Achilles’ heel in modern deep learning deployments, where adversaries generate imperceptible perturbations to mislead the DNN models.
Numerous attacks have been deployed in various 2D vision tasks, such as classiﬁcation (Carlini & Wagner, 2017), object detection (Song et al., 2018), and segmentation (Xie et al., 2017).
Since adversarial robustness is a critical feature, tremendous efforts have been devoted to defending against 2D adversarial images (Guo et al., 2017; Papernot et al., 2016;
Madry et al., 2018).
However, Athalye et al.
(2018) suggest that most of the current countermeasures essentially try to obfuscate gradients, which give a false sense of security.
Besides, certiﬁed methods (Zhang et al., 2019) often provide a lower bound of robustness, which are not helpful in practice.
Therefore, adversarial training is widely believed as the most and only effective defense solution.
The emergence of 3D point cloud applications in safety-critical areas like autonomous driving raises public concerns about their security of DNN pipelines.
A few studies (Xiang et al., 2019; Cao et al., 2019; Sun et al., 2020) have demonstrated that various deep learning tasks on point clouds are indeed vulnerable to adversarial examples.
Among them, point cloud classiﬁcation models have laid solid foundations upon which other complex models are built (Lang et al., 2019; Yu et al., 2018a).
While it seems intuitive to extend convolutional neural networks (CNN) from 2D to 3D for point cloud classiﬁcation, it is actually not a trivial task.
The difﬁculty mainly inherits from that point cloud is an unordered set structure that CNN cannot handle.
Modern point cloud classiﬁcation models (Qi et al., 2017a; Zaheer et al., 2017) address this problem by leveraging a symmetric function , which is permutation-invariant to the order of points, to aggregate local features, as shown in Figure 2.
Recently, a number of countermeasures have been proposed to defend against 3D adversarial point clouds.
However, the failure of gradient obfuscation-based defenses in the 2D space motivates us to re-think whether current defense designs provide real robustness for 3D point cloud classiﬁcation.
Especially, DUP-Net (Zhou et al., 2019) and GvG-PointNet++ (Dong et al., 2020a) claim to improve the adversarial robustness signiﬁcantly.
However, we ﬁnd that both defenses belong to gradient obfuscation through our analysis, hence further design white-box adaptive attacks to break their robustness.
Unfortunately, our 100% attack success rates demonstrate that current defense designs are still vulnerable.
As mentioned above, adversarial training (AT) is considered the most effective defense strategy; we thus perform the ﬁrst rigorous study of how AT behaves in point cloud classiﬁcation by exploiting projected gradient descent (PGD) attacks (Madry et al., 2018).
We identify that the default used symmetric function weakens the effectiveness of AT.
Speciﬁcally, popular models ( e.g., PointNet) utilize ﬁxed pooling operations like MAX and SUM pooling as their symmetric functions to aggregate features.
Different from CNN-based models that usually apply pooling operations with a small sliding window ( e.g., 2 × 2 ), point cloud classiﬁcation models leverage such ﬁxed pooling operations to aggregate features from a large number of candidates ( e.g., 1024).
We ﬁnd that those ﬁxed pooling operations inherently lack ﬂexibility and learnability, which are not appreciated by AT.
Moreover, recent research has also presented parametric pooling operations in set learning (Wang et al., 2020;
Zhang et al., 2020), which also preserve permutation-invariance.
We take a step further to systematically analyze point cloud classiﬁcation models’ robustness with parametric pooling operations under AT.
Experimental results show that the sorting-based pooling design beneﬁts AT well, which vastly outperforms MAX pooling, for instance, in adversarial accuracy by 7.3% without hurting the nominal accuracy 1 .
Lastly, based on our experimental insights, we propose DeepSym , a sorting-based pooling operation that employs deep learnable layers, to architecturally advance the adversarial robustness of point cloud classiﬁcation models under AT.
Experimental results show that DeepSym reaches the best adversarial accuracy in all chosen backbones, which on average, is a 10.8% improvement compared to the default architectures.
We also explore the limits of DeepSym based on PointNet due to its broad adoption (Guo et al., 2020).
We obtain the best robustness on ModelNet40, which achieves the adversarial accuracy of 47.0%, signiﬁcantly outperforming the default MAX pooling design by 28.5% ( ∼ 2 . 6 × ).
In addition, we demonstrate that PointNet with DeepSym also reaches the best adversarial accuracy of 45.2% under the most efﬁcient AT on ModelNet10 (Wu et al., 2015), exceeding MAX pooling by 17.9% ( ∼ 1 . 7 × ).
B ACKGROUND AND R ELATED W ORK
3D point cloud classiﬁcation.
Early works attempt to classify point clouds by adapting deep learning models in the 2D space (Su et al., 2015; Yu et al., 2018b).
DeepSets (Zaheer et al., 2017) and
PointNet (Qi et al., 2017a) are the ﬁrst to achieve end-to-end learning on point cloud classiﬁcation and formulate a general speciﬁcation (Figure 2) for point cloud learning.
PointNet++ (Qi et al., 2017b) and DGCNN (Wang et al., 2019) build upon PointNet set abstraction to better learn local features.
Lately, DSS (Maron et al., 2020) generalizes DeepSets to enable complex functions in set learning.
Besides, ModelNet40 (Wu et al., 2015) is the most popular dataset for benchmarking point cloud classiﬁcation, which consists of 12,311 CAD models belonging to 40 categories.
The numerical range of the point cloud data is normalized to [ − 1 , 1] in ModelNet40.
Adversarial attacks and defenses on point clouds.
Xiang et al.
(2019) perform the ﬁrst study to extend C&W attack (Carlini & Wagner, 2017) to point cloud classiﬁcation.
Wen et al.
(2019) improve the loss function in C&W attack to realize attacks with smaller perturbations and Hamdi et al.
(2019) present black-box attacks on point cloud classiﬁcation.
Recently, Zhou et al.
and Dong et al.
(2020a) propose to defend against adversarial point clouds by input transformation and adversarial detection.
Besides, Liu et al.
(2019) conduct a preliminary investigation on extending countermeasures in the 2D space to defend against simple attacks like FGSM (Goodfellow et al., 2014) on point cloud data.
In this work, we ﬁrst design adaptive attacks to break existing defenses and analyze the adversarial robustness of point cloud classiﬁcation under adversarial training.
B REAKING THE R OBUSTNESS OF E XISTING D EFENSES 3. A DAPTIVE A TTACKS ON DUP-N ET
DUP-Net (ICCV’19) presents a denoiser layer and upsampler network structure to defend against adversarial point cloud classiﬁcation.
The denoiser layer g : X → X (cid:48) leverages k NN ( k -nearest neighbour) for outlier removal.
Speciﬁcally, the k NN of each point x i in point cloud X is deﬁned as knn ( x i , k ) so that the average distance d i of each point x i to its k NN is denoted as:
[Equation]
where n is the number of points.
The mean µ = 1 n (cid:80) ni
=1 d i and standard deviation σ = (cid:113) 1 n (cid:80) ni
=
( d i − µ ) 2 of all these distances are computed to determine a distance threshold as µ + α · σ to trim the point clouds, where α is a hyper-parameter.
As a result, the denoised point cloud is represented as X (cid:48)
= { x i | d i < µ + α · σ } .
The denoised point cloud X (cid:48) will be further fed into
PU-Net (Yu et al., 2018a), deﬁned as p : X (cid:48)
→ X (cid:48)(cid:48) , to upsample X (cid:48) to a ﬁxed number of points.
Combined with the classiﬁer f , the integrated DUP-Net can be noted as ( f ◦ p ◦ g )
( X ) .
The hypothesis is that the denoiser layer will eliminate the adversarial perturbations and the upsampler network will re-project the denoised off-manifold point cloud to the natural manifold.
Analysis.
The upsampler network p ( i.e., PU-Net) is differentiable and can be integrated with the classiﬁcation network f .
Therefore, f ◦ p is clearly vulnerable to gradient-based adaptive attacks.
Although the denoiser layer g is not differentiable, it can be treated as deterministic masking :
M ( x i ) =
1 d i <µ + α · σ so that the gradients can still ﬂow through the masked points.
By involving
M ( x i ) into the iterative optimization process: ∇ x i ( f ◦ p ◦ g )( X ) | x i =
ˆ x ≈ ∇ x i ( f ◦ p )( X ) | x i =
ˆ x ·M (ˆ x ) , similar to BPDA (Athalye et al., 2018), attackers may still ﬁnd adversarial examples.
Experimentation.
We leverage the open-sourced codebase 2 of DUP-Net for experimentation.
Speciﬁcally, a PointNet (Qi et al., 2017a) trained on ModelNet40 is used as the target classiﬁer f .
For the PU-Net, the upsampled number of points is 2048, and the upsampling ratio is 2.
For the adaptive attacks, we exploit targeted L 2 norm-based C&W attack and untargeted L ∞ norm-based
PGD attack with 200 iterations (PGD-200).
Detailed setups are elaborated in Appendix A.1.
[Table]
Discussion.
As shown in Table 1, adaptive C&W attacks achieve 100% success rates on DUP-Net.
Though the mean distance of adversarial examples targeting DUP-Net is larger than those targeting
PU-Net, they are almost indistinguishable by human perception, as visualized in Appendix A.2.
We ﬁnd that na¨ıve PGD attacks are also effective since the upsampler network is sensitive to L ∞ norm-based perturbations.
The design of DUP-Net is similar to ME-Net
(Yang et al., 2019) in the 2D space, which recently has been shown vulnerable to adaptive attacks (Tramer et al., 2020).
We hereby demonstrate that such input transformation-based defenses cannot offer real robustness to point cloud classiﬁcation, either.
3. A DAPTIVE A TTACKS ON G V G-P OINT N ET ++
GvG-PointNet++ (CVPR’20) introduces gather vectors in the 3D point clouds as an adversarial indicator.
The original PointNet++ aggregates local features
f i hierarchically to make ﬁnal classiﬁcation.
Gather vectors v i are learned from local features f i to indicate the global center c
i of a point cloud sample.
If the indicated global center c i is far away from the ground-truth global center c g , the corresponding local feature f i will be masked out:
[Equation]
[Figure]
where x ci is the geometry center of the local point set, r is the distance threshold to mask the local feature, and F g is the cleaned feature set for ﬁnal classiﬁcation.
To train GvG-PointNet++, it is necessary to optimize a surrogate loss to correctly learn the gather vectors besides the cross-entropy (xent) loss:
[Equation]
where n (cid:48) is the number of local features and λ is a hyper-parameter.
Thus, GvG-PointNet++ essentially applies self-attention to the local features and relies on it for robustness enhancement.
Analysis.
Dong et al.
(2020a) evaluate white-box adversaries on GvG-PointNet++ with na ¨ ıve Lnorm-based PGD attacks.
Speciﬁcally, only L xent is utilized in the adversarial optimization process so that the masking M i will hinder the gradient propagation.
However, since M i is learned from the network itself, it is highly possible to further break this obfuscation with L gather considered.
The adaptive attack can be then formulated as an optimization problem with the loss function:
[Equation]
where β is a hyper-parameter.
By maximizing L adv with L 2 norm-based PGD attacks, adversaries strive to enlarge the adversarial effect but also minimize the perturbations on gather vectors.
We also ﬁnd that GvG-PointNet++ is by design vulnerable to PGD attacks on L gather as such perturbations will potentially affect most gather vector predictions to make g i masked out so that insufﬁcient for ﬁnal classiﬁcation.
Experimentation.
We train GvG-PointNet++ based on single-scale grouped PointNet++ (Qi et al., 2017b) on ModelNet40 and set r = 0 .
08 and λ = 1 as suggested by Dong et al.
(2020a). The model is trained by Adam (Kingma & Ba, 2014) optimizer with 250 epochs using batch size
[Table]
and the initial learning rate is 0.01.
For the adaptive attack, we use 10-step binary search to ﬁnd a appropriate β .
The setup of L 2 norm-based PGD attacks is identical to Dong et al. (2020a)
[Table]
and we also leverage
L ∞ norm-based PGD-200 in the evaluation.
Detailed setups are elaborated in Appendix A.
[Table]
[Table]
Discussion.
As shown in Table 2, both adaptive PGD attacks achieve high success rates on GvGPointNet++.
we also observe that the L ∞ norm-based PGD attack is more effective on L gather since
L ∞ norm perturbations assign the same adversarial budget to each point, which can easily impact a large number of gather vector predictions.
However, it is hard for the L 2 norm-based PGD attack to inﬂuence so many gather vector predictions because it prefers to perturb key points rather than the whole point set.
GvG-PointNet++ leverages DNN to detect adversarial perturbations, which is similar to MagNet (Meng & Chen, 2017) in the 2D space.
We validate that adversarial detection also fails to provide real robustness under adaptive white-box adversaries in point cloud classiﬁcation.
A DVERSARIAL T RAINING W ITH D IFFERENT S YMMETRIC F UNCTIONS
We have so far demonstrated that state-of-the-art defenses against 3D adversarial point clouds are still vulnerable to adaptive attacks.
While gradient obfuscation cannot offer real adversarial robust-
…… …… …… …… …… …… …… …… …… …… …… …… …… …… …… …… …… …… …… …… ФФ m d in = dd m n classification scores … … … … … … … … … … … … …
ρ global features σ k σ [Figure]
ness, adversarial training (AT) is widely believed to be the most effective method.
In this section, we conduct the ﬁrst thorough study showing how AT performs in point cloud classiﬁcation.
4. A DVERSARIAL T RAINING P RELIMINARIES
Madry et al.
(2018) formulate AT as a paddle point problem in Equation 5, where D is the underlying data distribution, L ( · , · , · ) is the loss function, x is the training data with its label y , (cid:15) is the adversarial perturbation, and S denotes the boundary of such perturbations.
[Equation]
Adversarial training setups.
We choose PointNet (Qi et al., 2017a), DeepSets (Zaheer et al., 2017), and DSS (Maron et al., 2020) as the backbone networks.
As shown in Section 3 and demonstrated by Madry et al.
(2018), L ∞ norm-based PGD attack tends to be a universal ﬁrst-order adversary.
Thus, we select PGD-7 into the training recipe, and empirically set the maximum per-point perturbation (cid:15)
= 0 .
out of the point cloud range [ − 1 , 1] .
We follow the default PointNet training setting to train models on the ModelNet40 training set.
In the evaluation, we utilize PGD-200 to assess their robustness on the ModelNet40 validation set with the same adversarial budget (cid:15)
= 0 .
Meanwhile, we also report the nominal accuracy on the clean validation set.
Each PGD attack starts from a random point in the allowed perturbation space.
More details can be found in Appendix B.
4. A DVERSARIAL T RAINING W ITH F IXED P OOLING O PERATIONS
As shown in Figure 2, modern models fundamentally follow a general speciﬁcation ( σ ◦ ρ ◦ Φ)
( X ) for point cloud classiﬁcation.
Φ( · ) represents a set of permutation-equivariant layers to learn local features from each point.
ρ ( · ) is a column-wise symmetric (permutation-invariant) function to aggregate the learned local features into a global feature, and σ ( · ) are fully connected layers for ﬁnal classiﬁcation.
PointNet, DeepSets, and DSS leverage different Φ( · ) for local feature learning, but all depend on ﬁxed pooling operations as their ρ ( · ) .
Speciﬁcally, MAX pooling is by default used in DeepSets (for point cloud classiﬁcation) and PointNet (Zaheer et al., 2017;
Qi et al., 2017a), while DSS utilizes SUM pooling (Maron et al., 2020).
We also additionally select MEDIAN pooling due to its robust statistic feature (Huber, 2004).
Though models with ﬁxed pooling operations have achieved satisfactory accuracy under standard training, they face various difﬁculties under AT.
As shown in Table 3, models with MEDIAN pooling achieve better nominal accuracy among ﬁxed pooling operations, but much worse adversarial accuracy, while SUM pooling performs contrarily.
Most importantly, none of them reach a decent balance of nominal and adversarial accuracy
[Table]
[Table]
Analysis.
AT consists of two stages: 1) inner maximization to ﬁnd the worst adversarial examples and 2) outer minimization to update model parameters.
Fixed pooling operations essentially leverage
d m … … … … …
S U M … … n α
α
α
αα 1 α
α
α 2 αα 2 α
α 3 α 3 αα
α
α
α 4 αα
α
α 5 α 5 αα
α 6 α 6 α 6 αα 6 α 7 α 7 α 7 ααα n α n α n α n α n …… …… …… …… …… …… …… …… …… … … … … …
d m … …
Attention Attention
[Figure]
d m … … … … … n …… …… …… …… …… …… …… …… …… … … … … …
d m
C o l u m n - w i s e
S
o r ti ng … … … …
[Figure]
a single statistic to represent the distribution of a feature dimension (Murray & Perronnin, 2014). Although MEDIAN pooling, as a robust statistic, intuitively should enhance the robustness, we ﬁnd it actually hinders the inner maximization stage from making progress. We utilize L ∞ norm-based
PGD attack to maximize the xent loss of standard trained model with three ﬁxed pooling operations.
Figure 3 validates that MEDIAN pooling takes many more steps to maximize the loss.
Therefore,
MEDIAN pooling fails to ﬁnd the worst adversarial examples in the ﬁrst stage with limited steps.
Though MAX and SUM pooling are able to achieve higher loss value, they encounter challenges in the second stage.
MAX pooling backward propagates gradients to a single point at each dimension so that the rest n − 1 n features do not contribute to model learning.
Since n is oftentimes a large number ( e.g., 1024), the huge information loss and non-smoothness will fail AT (Xie et al., 2020).
While SUM pooling realizes a smoother backward propagation, it lacks discriminability because by applying the same weight to each element, the resulting representations are strongly biased by the adversarial perturbations.
Thus, with SUM pooling, the models cannot generalize well on clean data.
4. A DVERSARIAL T RAINING W ITH P ARAMETRIC P OOLING O PERATIONS
Recent studies have also presented trainable parametric pooling operations for different tasks in set learning, e.g., multiple instance learning, which are also qualiﬁed to be the symmetric function ρ ( · ) in point cloud classiﬁcation models.
Thus, we ﬁrst group them into two classes: 1) attentionbased and 2) sorting-based pooling, and further benchmark their robustness under AT in point cloud classiﬁcation.
It is worth noting that none of those parametric pooling operations are proposed to improve the adversarial robustness, and we are the ﬁrst to conduct such an in-depth analysis of how they behave as the symmetric function under AT in point cloud classiﬁcation.
4.3. A TTENTION - BASED P OOLING O PERATIONS
An attention module can be abstracted as mapping a query and a set of key-value pairs to an output, making the models learn and focus on the critical information (Bahdanau et al., 2014).
Figure 4(a) shows the design principle of attention-based pooling, which leverages a compatibility function to learn point-level importance.
The aggregated global feature is computed as a column-wise weighted sum of the local features.
Two attention-based pooling operations, ATT and ATT-GATE , are ﬁrst proposed for multiple instance learning (Ilse et al., 2018).
Let F = { f 1 , f 2 , . . .
, f n } be a set of features, ATT aggregates the global feature g by:
[Equation]
where w ∈ R L × 1 and V ∈ R L ×
d m are learnable parameters.
ATT-GATE improves the expressiveness of ATT by introducing another non-linear activation sigmoid( · ) and more trainable parameters into weight learning.
Furthermore, PMA (Lee et al., 2019) is proposed for general set learning, which leverages multi-head attention (Vaswani et al., 2017) for pooling.
We detail the design and our implementation of ATT , ATT-GATE , and PMA in Appendix B.3, and adversarially train the backbone models with these attention-based pooling operations.
4.3. S ORTING - BASED P OOLING O PERATIONS
Sorting has been recently considered in the set learning literature due to its permutation-invariant characteristic, as shown in Figure 4(b).
Let F ∈ R n × d m be the matrix version of the feature set F ,
FSPool (Zhang et al., 2020) aggregates F by feature-wise sorting in a descending order:
[Equation]
where W ∈ R n × d m are learnable parameters.
Therefore, the pooled representation is column-wise weighted sum of (cid:101) F .
SoftPool (Wang et al., 2020) re-organizes F so that its j -th dimension is sorted in a descending order, and picks the top k point-level embeddings F (cid:48)
j ∈ R k ×
d m to further form (cid:101) F =
[ F (cid:48) 1 , F (cid:48) 2 , . .
, F (cid:48)
d m ] .
Then, SoftPool applies CNN to each (cid:101)
F j → g j so that the pooled representation is g =
[ g 1 , g 2 , . . . , g d m ] .
Implementation details of SoftPool are elaborated in
Appendix B.3.
We also adversarially train the backbone models with FSPool and SoftPool .
[Table]
E XPERIMENTAL A NALYSIS
Table 4 shows the results of AT with different parametric pooling operations.
To meet the requirement of permutation-invariance, attention-based pooling is restricted to learn point-level importance.
For example, ATT applies the same weight to all dimensions of a point embedding.
As a result, attention barely improves the pooling operation’s expressiveness as it essentially re-projects the point cloud to a single dimension ( e.g., f i → a i in ATT ) and differentiates them based on it, which signiﬁcantly limits their discriminability.
Therefore, little useful information can be learned from the attention module, explaining why they perform similarly to SUM pooling that applies the same weight to each point under AT, as shown in Table 4.
Sorting-based pooling operations naturally maintain permutation-invariance as sort ↓ ( · ) re-organizes the unordered feature set F to an ordered feature map (cid:101) F .
Thus, FSPool and SoftPool are able to further apply feature-wise linear transformation and CNN.
The insight is that feature dimensions are mostly independent of each other, and each point expresses to a different extent in every dimension.
By employing feature-wise learnable parameters, the gradients also ﬂow smoother through sorting-based pooling operations.
Tablevalidates that sorting-based pooling operations achieve much better adversarial accuracy, e.g., on average, 7.3% better than MAX pooling while maintaining comparable nominal accuracy.
I MPROVING T HE A DVERSARIAL R OBUSTNESS W ITH D EEP S YM
In the above analysis, we have shed light on that sorting-based pooling operations can beneﬁt AT in point cloud classiﬁcation.
We hereby explore to further improve the sorting-based pooling design inspired by existing arts.
First, we notice that both FSPool and SoftPool apply sort ↓ ( · ) right after a ReLU function (Nair & Hinton, 2010).
However, ReLU leads to some neurons being zero (Goodfellow et al., 2016), which makes sort ↓ ( · ) unstable.
Second, recent studies have shown that AT appreciates deeper neural networks (Xie & Yuille, 2019).
Nevertheless, FSPool only employs one linear layer to aggregate features, and SoftPool requires d m to be a small number.
The reason is that scaling up the depth in these existing sorting-based pooling designs requires exponential growth of parameters, which will make the end-to-end learning intractable.
To address the above limitations, we propose a simple yet effective pooling operation, DeepSym , that embraces the beneﬁts of sorting-based pooling and also applies deep learnable layers to the pooling process.
Given a feature set after ReLU activation F ∈ R + n × d m , DeepSym ﬁrst applies another linear transformation to re-map F into R n × d m so that f (cid:48) i
= W · f i (cid:62) where W ∈ R d m × d m and F (cid:48)
= { f (cid:48) 1 , f (cid:48) 2 , . . .
, f (cid:48) n } .
Let F (cid:48) be the matrix version of F (cid:48) , DeepSym also sorts F (cid:48) in a descending order (Equation 7) to get (cid:102) F (cid:48) .
Afterwards, we apply column-wise shared MLP on (cid:102) F (cid:48) : g j = MLP( (cid:102) F (cid:48) : ,j ) , j = { 1 , 2 , . . . d m } (8)
[Figure]
to learn the global feature representation g .
Each layer of the MLP composes of a linear transformation, a batch normalization module (Ioffe & Szegedy, 2015), and a ReLU activation function.
Compared to FSPool that applies different linear transformations to different dimensions, DeepSym employs a shared MLP to different dimensions.
By doing so, DeepSym deepens the pooling process to be more capable of digesting the adversarial perturbations.
DeepSym can also address the problem of SoftPool that is only achievable with limited d m because the MLP is shared by all the feature channels so that it can scale up to a large number of d m with little complexity increases.
Moreover, DeepSym generalizes MAX and SUM pooling by speciﬁc weight vectors.
Therefore, it can also theoretically achieve universality with d m ≥ n (Wagstaff et al., 2019) while being more expressive in its representation and smoother in gradients propagation.
To deal with the variable-size point clouds, DeepSym adopts column-wise linear interpolation in (cid:102) F (cid:48) to form a continuous feature map and then re-samples it to be compatible with the trained MLP (Jaderberg et al., 2015). Last but not least, DeepSym is by design ﬂexible with the number of pooled features from each dimension. In the paper, we only allow DeepSym to output a single feature for a fair comparison with others. However, it is hard for other pooling operations to have this ability.
For example, it requires a linear complexity increase for FSPool to enable this capability.
We implement a 5-layer DeepSym with [512 , 128 , 32 , 8 , 1] hidden neurons on three backbone networks and adversarially train them on ModelNet40 the same way introduced in Section 4.1.
Tableshows that almost all models with DeepSym reach the best results in both nominal and adversarial accuracy, outperforming the default architecture by 10.8%, on average.
Taking PointNet as an example, DeepSym (33.6%) improves the adversarial accuracy by 17.5% ( ∼ 2 . 1 × ) compared to the original MAX pooling architecture.
Besides, DeepSym also achieves a 3.5% improvement in adversarial accuracy compared to FSPool and SoftPool .
Overall, we demonstrate that DeepSym can beneﬁt AT signiﬁcantly in point cloud classiﬁcation.
We further leverage various white- and black-box adversarial attacks to cross validate the robustness improvements of DeepSym on PointNet.
Speciﬁcally, we exploit well-known FGSM (Szegedy et al., 2013), BIM (Kurakin et al., 2016), and MIM (Dong et al., 2018) as the white-box attack methods.
We set the adversarial budget (cid:15) = 0 . 05 , and leverage 200 steps for the iterative attacks, as well.
For the black-box attacks, we choose two score-based methods: SPSA (Uesato et al., 2018) and NES (Ilyas et al., 2018), and a decision-based evolution attack (Dong et al., 2020b).
We still select (cid:15) = 0 .
and allow 2000 queries to ﬁnd each adversarial example. The detailed setups are elaborated in Appendix C.1. As shown in Table 5, PointNet with DeepSym consistently achieves the best adversarial accuracy under white-box attacks, except for FGSM. The reason is that
FGSM is a single-step method that has limited ability to ﬁnd adversarial examples.
Besides, we ﬁnd the black-box attacks are not as effective as the white-box attacks, which also demonstrate that adversarial training with DeepSym is able to improve the robustness of point cloud classiﬁcation without gradient obfuscation (Carlini et al., 2019).
Since DeepSym brings deep trainable layers into the original backbones, it is necessary to report its overhead.
We leverage TensorFlow (Abadi et al., 2016) and NVIDIA proﬁlers to measure the inference time, the number of trainable parameters, and GPU memory usage on PointNet.
Speciﬁcally, the inference time is averaged from 2468 objects in the validation set, and the GPU memory is measured on an RTX 2080 with batch size = 8.
As shown in Table 6, DeepSym indeed introduces more computation overhead by leveraging the shared MLP.
However, we believe the overhead is relatively small and acceptable, compared to its massive improvements on the adversarial robustness.
To fur-
[Table]
ther have a lateral comparison
[Table]
point cloud classiﬁcation backbones are much more light-weight than image classiﬁcation models.
For example, ResNet-50 (He et al., 2016) and VGG-16 (Simonyan &
Zisserman, 2014) have 23 and 138 million trainable parameters, respectively
[Table]
and take much longer time to do the inference.
The reason that models with SoftPool and PMA have fewer trainable parameters is that they limit the number of dimensions in the global feature by design
[Table]
[Table]
5. E XPLORING T HE L IMITS OF D EEP S YM
There is a trade-off between the training cost and adversarial robustness in AT.
Increasing the number of PGD attack steps can create harder adversarial examples (Madry et al., 2018) [Table] which could further improve the model’s robustness.
However, the training time also increases linearly with the number of attack iterations increasing.
Due to PointNet’s broad adoption (Guo et al., 2020), we here analyze how it performs under various AT settings. Speciﬁcally
[Table] we exploit the most efﬁcient AT with PGD1 on ModelNet10 (Wu et al., 2015) [Table] a dataset consisting of 10 categories with 4899 objects
[Table]
and a relatively expensive AT with PGD-20 on ModelNet40 to demonstrate the effectiveness of DeepSym .
Other training setups are identical to Section 4.1.
Figure 5 shows the results of the robustness of adversarially trained PointNet with various pooling operations under PGD-200.
We demonstrate that PointNet with DeepSym still reaches the best adversarial accuracy of 45.2% under AT with PGD-1 on ModelNet10, which outperforms the original
MAX pooling by 17.9% ( ∼ 1 . 7 × ) and SoftPool by 4.0%.
Surprisingly, PointNet with DeepSym also achieves the best nominal accuracy of 88.5%.
Moreover, DeepSym further advances itself under AT with PGD-20 on ModelNet40.
Figure 5(b) shows that PointNet with DeepSym reaches the best 47.0% adversarial accuracy, which are 28.5% ( ∼ 2 .
6 × ) and 6.5% improvements compared to MAX pooling and SoftPool , respectively while maintaining competent nominal accuracy.
We also report detailed evaluations using different PGD attack steps and budgets (cid:15) in Appendix C.1.
C ONCLUSION
In this work, we perform the ﬁrst rigorous study on the adversarial robustness of point cloud classiﬁcation.
We design adaptive attacks and demonstrate that state-of-the-art defenses against adversarial point clouds cannot provide real robustness.
Furthermore, we conduct a thorough analysis of how the required symmetric function affects the AT performance of point cloud classiﬁcation models.
We are the ﬁrst to identify that the ﬁxed pooling generally weakens the models’ robustness under
AT, and on the other hand, sorting-based parametric pooling beneﬁts AT well.
Lastly, we propose
DeepSym that further architecturally advances the adversarial accuracy of PointNet to 47.0% under AT, outperforming the original design and a strong baseline by 28.5% ( ∼ 2 . 6 × ) and 6.5%.
A A DAPTIVE A TTACK E XPERIMENTAL S ETUP AND V ISUALIZATIONS A. E XPERIMENTAL S ETUPS
Since DUP-Net is open-sourced, we target the publicly released PointNet and PU-Net models.
For the L 2 norm-based C&W attack, we set the loss function as:
[Equation]
where X ∈ R n × 3 is the matrix version of point cloud X , X (cid:48) is the optimized adversarial example,
Z ( X ) i is the i -th element of the output logits, and t (cid:48) is the target class.
We leverage 10-step binary search to ﬁnd the appropriate hyper-parameter λ from [10 , 80] .
As suggested by Xiang et al.
(2019), we choose 10 distinct classes and pick 25 objects in each class from the ModelNet40 validation set for evaluation.
The step size of the adversarial optimization is 0.01 and we allow at mostiterations of optimization in each binary search to ﬁnd the adversarial examples.
For the L ∞ norm-based PGD attack, we adopt the formulation in Madry et al.
[Equation]
(10) where X t is the adversarial example in the t -th attack iteration, Π is the projection function to project the adversarial example to the pre-deﬁned perturbation space S , which is the L ∞ norm ball in our setup, and α is the step size.
We select the boundary of allowed perturbations (cid:15) = { 0 . 01 , 0 .
out of the point cloud data range [ − 1 , 1] .
Since point cloud data is continuous, we set the step size α =
[Table]
For GvG-PointNet++ , we train it based on the single scale grouping (SSG)-PointNet++ backbone.
The backbone network has three PointNet set abstraction module to hierarchically aggregate local features, and we enable gather vectors in the last module, which contains 128 local features ( i.e., n (cid:48)
= 128 in Section 3.2) with 256 dimensions.
To learn the gather vectors, we apply three fully connected layers with 640, 640, and 3 hidden neurons respectively, as suggested by Dong et al.
(2020a). Since the data from ModelNet40 is normalized to [-1,1], the global object center is c g =
For the L ∞ norm-based PGD attack, we leverage the same setup as the attack on DUP-Net.
For the L 2 norm-based PGD attack, we follow the settings in Dong et al.
(2020a) to set the L 2 norm threshold (cid:15) =
δ √ n × d in , where δ is selected in { 0 . 08 , 0 .
32 } , n is the number of points, and d in is the dimension of input point cloud ( i.e., 3).
The attack iteration is set to 50, and the step size α = (cid:15) 50 .
A. V ISUALIZATIONS
We visualize some adversarial examples generated by adaptive attacks on PU-Net and DUP-Net in
Figure 6 and Figure 7.
It is expected that adversarial examples targeting DUP-Net are noisier than the ones targeting PU-Net as the former needs to break the denoiser layer.
However, as mentioned in Section 3.1, they are barely distinguishable from human perception.
We also visualize some adversarial examples generated by untargeted adaptive PGD attacks on GvG-PointNet++ in Figurewith different perturbation budgets (cid:15) .
B A DVERSARIAL T RAINING S ETUP B. PGD A TTACK IN A DVERSARIAL T RAINING
We also follow the formulation in Equation 18 to ﬁnd the worst adversarial examples.
Speciﬁcally, we empirically select (cid:15) = 0 .
into the training recipe as there is no quantitative study on how much humans can bear the point cloud perturbations.
Figure 8 shows that adversarial examples with (cid:15) = 0 .
05 are still recognizable by human perception.
Moreover, because point cloud data is continuous, we set the step size of PGD attacks as:
[Equation]
airplane bed bookshelf bottle chair monitor sofa table toilet vase v a s e t o il e t t a b l e s o f a m on it o r c h a i r bo ttl e book s h e l f b e d a i r p l a n e in both training and evaluation phases to make sure PGD attacks reach the allowed maximum perturbations.
[Equation]
PointNet, DeepSets, and DSS are the fundamental architectures in point cloud classiﬁcation.
Other models, such as PointNet++ and DGCNN, are built upon PointNet and DeepSets.
Moreover, complex models oftentimes apply non-differentiable layers like knn ( · ) into end-to-end learning, which will make the adversarial training ineffective.
In this work, we aim at exploring how the symmetric (permutation-invariant) function can beneﬁt adversarial training.
To this end, we choose PointNet,
DeepSets, and DSS as the backbone networks.
For the ModelNet40 dataset, we follow the default setting to split into 9,843 objects for training and 2,468 objects for validation (Wu et al., 2015).
We randomly sample 1024 points from each object to form its point cloud, if not otherwise stated.
PointNet.
We leverage the default architecture in PointNet codebase 3 and exclude the transformation nets ( i.e., T-Net) and dropout layers for simplicity and reproducibility.
PointNet leverages shared fully connected (FC) layers as the permutation-equivariant layer φ l : FC l ( F l : ,i )
→ F l +1: ,i and MAX pooling as the symmetric function ρ ( · ) .
[Figure]
vase v a s e t o il e t t a b l e s o f a m on it o r c h a i r bo ttl e book s h e l f b e d a i r p l a n e
DeepSets.
We leverage the default architecture in DeepSets codebase 4 .
Different from PointNet,
DeepSets ﬁrst applies a symmetric function to each feature map and aggregate it with the original feature map.
Afterwards, DeepSets also leverages FC layers to further process the features: φ l : FC l ( F l : ,i − ζ ( F l ))
→ F l +1: ,i , where ζ ( · ) is column-wise MAX pooling in the original implementation.
Similarly, MAX pooling is still used as ρ ( · ) in DeepSets.
DSS.
DSS generalizes DeepSets architecture and applies another FC layer to ζ ( F l ) in DeepSets so that φ l : FC l 1 ( F l : ,i ) +
FC l 2 ( ζ ( F l ))
→ F l +1: ,i .
Different from other two achitectures, DSS utilizes
SUM pooling as ρ ( · ) .
Since there is no available codebase at the time of writing, we implement DSS by ourselves.
We visualize the differences of φ ( · ) in Figure 9, and summarize the layer information in Table 7.
B. P ARAMETRIC P OOLING D ESIGN A ND I MPLEMENTATION
We have introduced ATT in Section 4.3.1.
In our implementation, we choose L = 512 so that
V ∈ R to train the backbone models.
ε=0. ε=0. ε=0. ε=0.
ε=0.
v a s e t o il e t t a b l e s o f a m on it o r c h
a
i r bo ttl e book s h e l f b e d a i r p l a n e
ATT-GATE is a variant of ATT with more learnable parameters:
[Equation]
(12) where U ,
V ∈ R L × M , sigm( · ) is the sigmoid activation function, and (cid:12) is an element-wise multiplication.
We also choose L = 512 in ATT-GATE to train the backbone models.
[Table]
) φ ( · ) in DeepSets.
[Figure] [Figure]
[Figure]
FC 2 ( ) FCd l d l+ … … … … … … …… …… …… ……
FC 2 ( ) ( ) FC 2 ( ) ( )
FC 2 ( ) + + + + … …
FC 1 FC
FC(
[Figure]
) φ ( · ) in DSS.
[Figure]
PMA (Lee et al., 2019) adopts multi-head attention into pooling on a learnable set of k seed vectors
S ∈ R k × d m Let F ∈ R n × d m be the matrix version of the set of features.
[Equation] [Equation]
(14) where H = X + Multihead
( X , Y , Y ; w ) (15) where FC( · ) is the fully connected layer and Multihead( · ) is the multi-head attention module (Vaswani et al., 2017).
We follow the implementation in the released codebase 5 to choose k = 1 , the number of head = 4, and the hidden neurons in FC( · ) = 128 to train the backbone models.
Since SoftPool (Wang et al., 2020) sorts the feature set in each dimension, it requires the number of dimensions d m to be relatively small.
We follow the description in their paper to choose d m =and k = 32 so that each F j (cid:48) ∈
R 32 × 8 .
We apply one convolutional layer to aggregate each F (cid:48)
j into g j ∈ R 1 ×
so that the ﬁnal g ∈ R 1 × 256 .
Therefore, for all backbone networks with SoftPool , we apply the last equivariant layer as φ : n × d m − 1 → n × 8 and ρ : n × 8 → 256 .
C D EEP S YM A BLATIONS
It is worth noting that DeepSym does not require the ﬁnal layer to have only one neuron.
However, to have a fair comparison with other pooling operations that aggregate into one feature from each dimension, our implementation of DeepSym also aggregates into one feature from each dimension.
C. E VALUATION D ETAILS
We also perform extensive evaluations using different PGD attack steps and budgets (cid:15) on PGD20 trained PointNet.
Figure 10 shows that PointNet with DeepSym consistently achieves the best adversarial accuracy.
We also validate MEDIAN pooling indeed hinders the gradient backward propagation.
The adversarial accuracy of PointNet with MEDIAN pooling consistently drops even after
PGD-1000.
However, the adversarial accuracy of PointNet with other pooling operations usually converges after PGD-200.
Figure 11 shows that DeepSym also outperforms other pooling operations under different adversarial budgets (cid:15) .
We leverage the default setup in FGSM, BIM, and MIM in our evaluation.
FGSM is a single-step attack method, which can be represented as: [Equation] The BIM attack is similar to PGD attacks described in Appendix A.1. The differences are 1) the attack starts from the original point cloud X and 2) the step size α = (cid:15)/T , where T is the number of attack steps.
The MIM attack introduces momentum terms into the adversarial optimization:
[Equation]
Similar to BIM, the attack starts from the original point cloud X and the step size α =
(cid:15)/T .
We set µ = 1 following the original setup (Dong et al., 2018).
Due to the computational resource constraints, we set the sample size = 32 and allow 2000 quires to ﬁnd each adversarial example in the score-based black-box attack (Uesato et al., 2018; Ilyas et al., 2018).
For the evolution attack, we use the default loss L as the ﬁtness score, and initialize 32 sets of perturbations from a Gaussian distribution N (0 , 1) .
4 sets of perturbations with top ﬁtness scores will remain for the next iteration, while others will be discarded.
We also allow 2000 generations of evolution to ﬁnd the adversarial example.
C. E VALUATION ON S CAN O BJECT NN
We also evaluate the adversarial robustness of different pooling operations on a new point cloud dataset, ScanObjectNN (Uy et al., 2019), which contains 2902 objects belonging to 15 categories.
We leverage the same adversarial training setup as ModelNet10 ( i.e., PGD-1).
Table 8 shows the results.
We ﬁnd that PointNet with DeepSym still achieves the best adversarial robustness.
Since the point clouds from ScanObjectNN are collected from real-world scenes, which suffers from occlusion and imperfection, both nominal and adversarial accuracy drops compared to the results
ModelNet40.
We ﬁnd that even some clean point clouds cannot be correctly recognized by human perception.
Therefore, the performance degradation is also expected and we believe the results are not as representative as ones on ModelNet40.
[Table] C. T-SNE V ISUALIZATIONS We visualize the global feature embeddings of adversarially trained PointNet under PGD-20 with different pooling operations in Figure 12 and their logits in Figure 13.
Since it is hard to pickdistinct colors, though we put all data from 40 classes into the T-SNE process, we only choosecategories from ModelNet40 to realize the visualizations.
[Figure] [Figure]