On The Adversarial Robustness of 3D Point Cloud Classification

1 INTRODUCTION . Despite the prominent achievements that deep neural networks ( DNN ) have reached in the past decade , adversarial attacks ( Szegedy et al. , 2013 ) are becoming the Achilles ’ heel in modern deep learning deployments , where adversaries generate imperceptible perturbations to mislead the DNN models . Numerous attacks have been deployed in various 2D vision tasks , such as classification ( Carlini & Wagner , 2017 ) , object detection ( Song et al. , 2018 ) , and segmentation ( Xie et al. , 2017 ) . Since adversarial robustness is a critical feature , tremendous efforts have been devoted to defending against 2D adversarial images ( Guo et al. , 2017 ; Papernot et al. , 2016 ; Madry et al. , 2018 ) . However , Athalye et al . ( 2018 ) suggest that most of the current countermeasures essentially try to obfuscate gradients , which give a false sense of security . Besides , certified methods ( Zhang et al. , 2019 ) often provide a lower bound of robustness , which are not helpful in practice . Therefore , adversarial training is widely believed as the most and only effective defense solution . The emergence of 3D point cloud applications in safety-critical areas like autonomous driving raises public concerns about their security of DNN pipelines . A few studies ( Xiang et al. , 2019 ; Cao et al. , 2019 ; Sun et al. , 2020 ) have demonstrated that various deep learning tasks on point clouds are indeed vulnerable to adversarial examples . Among them , point cloud classification models have laid solid foundations upon which other complex models are built ( Lang et al. , 2019 ; Yu et al. , 2018a ) . While it seems intuitive to extend convolutional neural networks ( CNN ) from 2D to 3D for point cloud classification , it is actually not a trivial task . The difficulty mainly inherits from that point cloud is an unordered set structure that CNN can not handle . Modern point cloud classification models ( Qi et al. , 2017a ; Zaheer et al. , 2017 ) address this problem by leveraging a symmetric function , which is permutation-invariant to the order of points , to aggregate local features , as shown in Figure 2 . Recently , a number of countermeasures have been proposed to defend against 3D adversarial point clouds . However , the failure of gradient obfuscation-based defenses in the 2D space motivates us to re-think whether current defense designs provide real robustness for 3D point cloud classification . Especially , DUP-Net ( Zhou et al. , 2019 ) and GvG-PointNet++ ( Dong et al. , 2020a ) claim to improve the adversarial robustness significantly . However , we find that both defenses belong to gradient obfuscation through our analysis , hence further design white-box adaptive attacks to break their robustness . Unfortunately , our 100 % attack success rates demonstrate that current defense designs are still vulnerable . As mentioned above , adversarial training ( AT ) is considered the most effective defense strategy ; we thus perform the first rigorous study of how AT behaves in point cloud classification by exploiting projected gradient descent ( PGD ) attacks ( Madry et al. , 2018 ) . We identify that the default used symmetric function weakens the effectiveness of AT . Specifically , popular models ( e.g. , PointNet ) utilize fixed pooling operations like MAX and SUM pooling as their symmetric functions to aggregate features . Different from CNN-based models that usually apply pooling operations with a small sliding window ( e.g. , 2× 2 ) , point cloud classification models leverage such fixed pooling operations to aggregate features from a large number of candidates ( e.g. , 1024 ) . We find that those fixed pooling operations inherently lack flexibility and learnability , which are not appreciated by AT . Moreover , recent research has also presented parametric pooling operations in set learning ( Wang et al. , 2020 ; Zhang et al. , 2020 ) , which also preserve permutation-invariance.We take a step further to systematically analyze point cloud classification models ’ robustness with parametric pooling operations under AT . Experimental results show that the sorting-based pooling design benefits AT well , which vastly outperforms MAX pooling , for instance , in adversarial accuracy by 7.3 % without hurting the nominal accuracy1 . Lastly , based on our experimental insights , we propose DeepSym , a sorting-based pooling operation that employs deep learnable layers , to architecturally advance the adversarial robustness of point cloud classification models under AT . Experimental results show that DeepSym reaches the best adversarial accuracy in all chosen backbones , which on average , is a 10.8 % improvement compared to the default architectures . We also explore the limits of DeepSym based on PointNet due to its broad adoption ( Guo et al. , 2020 ) . We obtain the best robustness on ModelNet40 , which achieves the adversarial accuracy of 47.0 % , significantly outperforming the default MAX pooling design by 28.5 % ( ∼ 2.6× ) . In addition , we demonstrate that PointNet with DeepSym also reaches the best adversarial accuracy of 45.2 % under the most efficient AT on ModelNet10 ( Wu et al. , 2015 ) , exceeding MAX pooling by 17.9 % ( ∼ 1.7× ) . 2 BACKGROUND AND RELATED WORK . 3D point cloud classification . Early works attempt to classify point clouds by adapting deep learning models in the 2D space ( Su et al. , 2015 ; Yu et al. , 2018b ) . DeepSets ( Zaheer et al. , 2017 ) and PointNet ( Qi et al. , 2017a ) are the first to achieve end-to-end learning on point cloud classification and formulate a general specification ( Figure 2 ) for point cloud learning . PointNet++ ( Qi et al. , 2017b ) and DGCNN ( Wang et al. , 2019 ) build upon PointNet set abstraction to better learn local features . Lately , DSS ( Maron et al. , 2020 ) generalizes DeepSets to enable complex functions in set learning . Besides , ModelNet40 ( Wu et al. , 2015 ) is the most popular dataset for benchmarking point cloud classification , which consists of 12,311 CAD models belonging to 40 categories . The numerical range of the point cloud data is normalized to [ −1 , 1 ] in ModelNet40 . Adversarial attacks and defenses on point clouds . Xiang et al . ( 2019 ) perform the first study to extend C & W attack ( Carlini & Wagner , 2017 ) to point cloud classification . Wen et al . ( 2019 ) improve the loss function in C & W attack to realize attacks with smaller perturbations and Hamdi et al . ( 2019 ) present black-box attacks on point cloud classification . Recently , Zhou et al . ( 2019 ) and Dong et al . ( 2020a ) propose to defend against adversarial point clouds by input transformation and adversarial detection . Besides , Liu et al . ( 2019 ) conduct a preliminary investigation on extending countermeasures in the 2D space to defend against simple attacks like FGSM ( Goodfellow et al. , 2014 ) on point cloud data . In this work , we first design adaptive attacks to break existing defenses and analyze the adversarial robustness of point cloud classification under adversarial training . 3 BREAKING THE ROBUSTNESS OF EXISTING DEFENSES . 3.1 ADAPTIVE ATTACKS ON DUP-NET . DUP-Net ( ICCV ’ 19 ) presents a denoiser layer and upsampler network structure to defend against adversarial point cloud classification . The denoiser layer g : X → X′ leverages kNN ( k-nearest 1In this paper , we use nominal and adversarial accuracy to denote the model ’ s accuracy on clean and adversarially perturbed data , respectively . neighbour ) for outlier removal . Specifically , the kNN of each point xi in point cloud X is defined as knn ( xi , k ) so that the average distance di of each point xi to its kNN is denoted as : di = 1 k ∑ xj∈knn ( xi , k ) ||xi − xj ||2 , i = { 1 , 2 , . . . , n } ( 1 ) where n is the number of points . The mean µ = 1n ∑n i=1 di and standard deviation σ =√ 1 n ∑n i=1 ( di − µ ) 2 of all these distances are computed to determine a distance threshold as µ+α·σ to trim the point clouds , where α is a hyper-parameter . As a result , the denoised point cloud is represented as X′ = { xi | di < µ + α · σ } . The denoised point cloud X′ will be further fed into PU-Net ( Yu et al. , 2018a ) , defined as p : X′ → X′′ , to upsample X′ to a fixed number of points . Combined with the classifier f , the integrated DUP-Net can be noted as ( f ◦ p ◦ g ) ( X ) . The hypothesis is that the denoiser layer will eliminate the adversarial perturbations and the upsampler network will re-project the denoised off-manifold point cloud to the natural manifold . Analysis . The upsampler network p ( i.e. , PU-Net ) is differentiable and can be integrated with the classification network f . Therefore , f ◦ p is clearly vulnerable to gradient-based adaptive attacks . Although the denoiser layer g is not differentiable , it can be treated as deterministic masking : M ( xi ) = 1di < µ+α·σ so that the gradients can still flow through the masked points . By involving M ( xi ) into the iterative optimization process : ∇xi ( f ◦p◦g ) ( X ) |xi=x̂ ≈ ∇xi ( f ◦p ) ( X ) |xi=x̂·M ( x̂ ) , similar to BPDA ( Athalye et al. , 2018 ) , attackers may still find adversarial examples . Experimentation . We leverage the open-sourced codebase2 of DUP-Net for experimentation . Specifically , a PointNet ( Qi et al. , 2017a ) trained on ModelNet40 is used as the target classifier f . For the PU-Net , the upsampled number of points is 2048 , and the upsampling ratio is 2 . For the adaptive attacks , we exploit targeted L2 norm-based C & W attack and untargeted L∞ norm-based PGD attack with 200 iterations ( PGD-200 ) . Detailed setups are elaborated in Appendix A.1 . Discussion . As shown in Table 1 , adaptive C & W attacks achieve 100 % success rates on DUP-Net . Though the mean distance of adversarial examples targeting DUP-Net is larger than those targeting PU-Net , they are almost indistinguishable by human perception , as visualized in Appendix A.2 . We find that naı̈ve PGD attacks are also effective since the upsampler network is sensitive to L∞ norm-based perturbations . The design of DUP-Net is similar to ME-Net ( Yang et al. , 2019 ) in the 2D space , which recently has been shown vulnerable to adaptive attacks ( Tramer et al. , 2020 ) . We hereby demonstrate that such input transformation-based defenses can not offer real robustness to point cloud classification , either . 3.2 ADAPTIVE ATTACKS ON GVG-POINTNET++ . GvG-PointNet++ ( CVPR ’ 20 ) introduces gather vectors in the 3D point clouds as an adversarial indicator . The original PointNet++ aggregates local features fi hierarchically to make final classification . Gather vectors vi are learned from local features fi to indicate the global center ci of a point cloud sample . If the indicated global center ci is far away from the ground-truth global center cg , the corresponding local feature fi will be masked out : ci = xci + vi ; Mi = 1||cg−ci|| < r ; Fg = { fi · Mi } ( 2 ) 2https : //github.com/RyanHangZhou/DUP-Net where xci is the geometry center of the local point set , r is the distance threshold to mask the local feature , and Fg is the cleaned feature set for final classification . To train GvG-PointNet++ , it is necessary to optimize a surrogate loss to correctly learn the gather vectors besides the cross-entropy ( xent ) loss : Ltotal = Lxent + λ · Lgather , Lgather = n′∑ i=1 ||ci − cg||1 ( 3 ) where n′ is the number of local features and λ is a hyper-parameter . Thus , GvG-PointNet++ essentially applies self-attention to the local features and relies on it for robustness enhancement . Analysis . Dong et al . ( 2020a ) evaluate white-box adversaries on GvG-PointNet++ with naı̈ve L2 norm-based PGD attacks . Specifically , only Lxent is utilized in the adversarial optimization process so that the maskingMi will hinder the gradient propagation . However , sinceMi is learned from the network itself , it is highly possible to further break this obfuscation with Lgather considered . The adaptive attack can be then formulated as an optimization problem with the loss function : Ladv = Lxent − β · Lgather ( 4 ) where β is a hyper-parameter . By maximizing Ladv with L2 norm-based PGD attacks , adversaries strive to enlarge the adversarial effect but also minimize the perturbations on gather vectors . We also find that GvG-PointNet++ is by design vulnerable to PGD attacks on Lgather as such perturbations will potentially affect most gather vector predictions to make gi masked out so that insufficient for final classification . Experimentation . We train GvG-PointNet++ based on single-scale grouped PointNet++ ( Qi et al. , 2017b ) on ModelNet40 and set r = 0.08 and λ = 1 as suggested by Dong et al . ( 2020a ) . The model is trained by Adam ( Kingma & Ba , 2014 ) optimizer with 250 epochs using batch size 16 , and the initial learning rate is 0.01 . For the adaptive attack , we use 10-step binary search to find a appropriate β . The setup of L2 norm-based PGD attacks is identical to Dong et al . ( 2020a ) , and we also leverage L∞ norm-based PGD-200 in the evaluation . Detailed setups are elaborated in Appendix A.1 . Discussion . As shown in Table 2 , both adaptive PGD attacks achieve high success rates on GvGPointNet++ . we also observe that the L∞ norm-based PGD attack is more effective on Lgather since L∞ norm perturbations assign the same adversarial budget to each point , which can easily impact a large number of gather vector predictions . However , it is hard for the L2 norm-based PGD attack to influence so many gather vector predictions because it prefers to perturb key points rather than the whole point set . GvG-PointNet++ leverages DNN to detect adversarial perturbations , which is similar to MagNet ( Meng & Chen , 2017 ) in the 2D space . We validate that adversarial detection also fails to provide real robustness under adaptive white-box adversaries in point cloud classification .

Summary: The paper addresses the problem of adversarial robustness in 3D point cloud representations. It claims that two of the previous defense designs do not prevent adaptive attacks. The authors then propose to use adversarial training (AT) to improve the robustness. It claims that the standard MAX pooling operation within PointNet-derivates contribute to the weaknesses. It then proposes a new pooling operation that improves the robustness under AT.