PointDP : D IFFUSION - DRIVEN P URIFICATION AGAINST 3D A DVERSARIAL P OINT C LOUDS
Anonymous authors
Paper under double-blind review
A BSTRACT
3D Point cloud is a critical data representation in many real-world applications, such as autonomous driving, robotics, and medical imaging.
Although the success of deep learning further accelerates the adoption of 3D point clouds in the physical world, deep learning is notoriously vulnerable to adversarial attacks.
Various defense solutions have been proposed to build robust models against adversarial attacks.
In this work, we identify that the state-of-the-art empirical defense, adversarial training, has a major limitation in 3D point cloud models due to gradient obfuscation, resulting in significant degradation of robustness against strong attacks.
To bridge the gap, we propose PointDP , a purification strategy that leverages diffusion models to defend against 3D adversarial attacks.
Since PointDP does not rely on predefined adversarial examples for training, it can defend against diverse threats.
We extensively evaluate PointDP on six representative 3D point cloud architectures and leverage sixteen strong and adaptive attacks to demonstrate its lower-bound robustness.
Our evaluation shows that PointDP achieves significantly better ( i.e., 12.6%-40.3%) adversarial robustness than state-of-the-art methods under strong attacks bounded by different ℓ p norms.
I NTRODUCTION
Point cloud data is emerging as one of the most broadly used representations in 3D computer vision.
It is a versatile data format available from various sensors like LiDAR and stereo cameras and computer-aided design (CAD) models, which depict physical objects by many coordinates in the 3D space.
Many deep learning-based 3D perception models have been proposed [59, 34, 43, 60, 41, 9] and thus realized several safety-critical applications ( e.g., autonomous driving) [81, 46, 45].
Although deep learning models [41, 42] have exhibited performance boost on many challenging tasks, extensive studies show that they are notoriously vulnerable to adversarial attacks [5, 49, 68], where attackers manipulate the input in an imperceptible manner, which will lead to incorrect predictions of the target model.
Because of the broad applications of 3D point clouds in safety-critical fields, it is imperative to study the adversarial robustness of point cloud recognition models.
The manipulation space for 2D adversarial attacks is to change pixel-level numeric values of the input images.
Unlike adversarial examples in 2D applications, the flexible representation of 3D point clouds results in an arguably larger attack surface.
For example, adversaries could shift and detach existing points [88], add new points into the pristine point cloud [50], or even generate totally new point clouds [89] to launch attacks.
Different strategies, including limits on the number of altered points and constraints on the maximal magnitude of shifted points [50] were proposed to make attacks less perceptible.
The flexibility of 3D point cloud data formats enables diverse attacks, thus hindering a practical and universal defense design.
Given the safety-critical property involved in 3D point cloud applications, various studies have been devoted to advancing the robustness of 3D point cloud recognition models.
DUP-Net [90] and
GvG-PointNet++ [14] pioneered to add statistical outlier removal (SOR) modules as pre-processing and in-network blocks, respectively, as mitigation strategies.
More lately, Sun et al.
[51] broke the robustness of DUP-Net and GvG-PointNet++ by specific adaptive attacks.
Adversarial training has been acknowledged as the most potent defense to deliver strong empirical robustness on PointNet,
DGCNN, and PCT [50].
Meanwhile, advanced purification strategies like IF-Defense [66] and
LPC [25] leverage more complex modules to cleanse the adversarial point clouds.
However, given that point cloud is a sparse and unstructured data format, it motivates us to re-think that whether the current adversarial training and purification-based methods are robust enough against stronger adversarial attacks?
In this work, our journey starts with revisiting the prior arts and exploring their truly adversarial robustness.
By designing various types of strong adaptive attacks, we, for the first time, demonstrate that standard adversarial training [33] suffers from gradient obfuscation in the point cloud recognition models as the unstructured point cloud data format requires unique architectural designs to digest.
We also extensively evaluate IF-Defense and LPC to show that their purification strategies are actually vulnerable to stronger attacks (§ 4.3).
Furthermore, we propose PointDP , an adversarial purification method that leverages a diffusion model as a pre-processing module to defend against 3D adversaries.
As shown in Figure 1, PointDP consists of two components (1) an off-the-shelf 3D point cloud diffusion model and (2) a classifier.
Given an input point cloud, PointDP take two steps: (i) adding noise to the input data gradually via the diffusion process of the diffusion model, (ii) purifying the noised data step by step to get the reversed sample via the reverse process of a diffusion model (§ 3.1), and (iii) feeding the reversed sample to the final classifier.
Since PointDP does not rely on any types of pre-defined adversarial examples for training, it can defend against diverse unseen threats.
We rigorously evaluate PointDP with six representative point cloud models and sixteen attacks, including PGD [50, 33], C&W [68, 8], and point cloud-specific attacks [88, 21] with ℓ 0 , ℓ 2 , and ℓ ∞ norms.
PointDP on average achieves 75.9% robust accuracy while maintaining similar clean accuracy to the original models, outperforming existing studies by a significant margin.
In a nutshell, our contributions are summarized as two-fold :
• We are the first to demonstrate that standard adversarial training [33, 50], the most longstanding defense in the 2D image recognition task, has a major limitation in its application in 3D point cloud models due to architecture designs.
We launch black-box attacks to validate our claim that degrades adversarially trained models’ robust accuracy to merely ∼ 10%, which is no longer useful for 3D point cloud recognition.
• We propose PointDP that leverages diffusion models to purify adversarial 3D point clouds.
We conduct extensive and rigorous evaluation on six representative models with numerous attacks to comprehensively understand the robustness of PointDP .
Our evaluation shows that PointDP outperforms previous state-of-the-arts purification methods, IF-Defense [66] and LPC [25], by 12.6% and 40.3% on average, respectively.
We also set up a rigorous protocol for 3D robustness evaluation.
R ELATED W ORK
In this section, we review the current progress of deep learning, adversarial attacks, and defenses for 3D point cloud recognition tasks.
2. D EEP L EARNING ON 3D P OINT C LOUD R ECOGNITION
2D computer vision has achieved stellar progress on architectural designs of convolutional neural networks [22], followed by vision transformers [15].
However, there is currently no consensus on the architecture of 3D perception models since there is no standard data format for 3D perception [53].
As raw data from both 3D scanners and triangular meshes can be efficiently transformed into point clouds, they are becoming the most often utilized data format in 3D perception.
3D networks at the early stage use dense voxel grids for perception [59, 34, 47, 54], which discretize a point cloud to voxel cells for classification, segmentation, and object detection.
PointNet pioneered to leverage global pooling help achieve memory-efficient permutation invariance in an end-to-end manner.
PointNet++ [42] and
DGCNN [61] followed up to add sophisticated local clustering operations to advance the performance.
Sparse tensors are the other direction in 3D network designs [19, 9] to use 3D convolutions to improve 3D perception performance.
PointCNN and RSCNN reformed the classic pyramid CNN to improve the local feature generation [26, 29].
PointConv and KPConv designed new convolution operation for point cloud learning [65, 55].
PointTransformer and PCT advanced self-attention blocks in the 3D space and achieved good performance [87, 20].
Various novel local clustering operations [69, 32] also show enhancements on the clean performance.
In this work, we focus on PointNet, PointNet++,
DGCNN, PCT, CurveNet, and PointMLP as our evaluation backbones since they are representative and widely used and achieve state-of-the-art results in point cloud recognition [1].
2. A DVERSARIAL A TTACKS AND D EFENSES
Adversarial attacks have become the main obstacle that hinder deep learning models from real-world deployments, especially in safety-critical applications [16, 49, 5, 86, 85].
There are a lot of adversarial attacks proposed in the 2D space to break the various vision models [8, 72, 79, 75, 23, 24, 73, 52].
To fill this gap between standard and robust accuracies, many mitigation solutions have been studied and presented to improve the robustness against adversarial attacks [80, 78, 4, 38, 35, 82, 71, 83, 70].
However, most of them including adding randomization [28, 13, 14], model distillation [38], adversarial detection [35], and input transformation [80, 78, 37, 4, 90] have been compromised by adaptive attacks [56, 2].
Adversarial training (AT) [33, 18, 64, 44], in contrast, delivered a more longstanding mitigation strategy [74, 76, 84].
However, the robust accuracy achieved by AT is still not satisfactory enough to be used in practice.
Most recently, Nie et al.
proposed DiffPure [36] that leverages diffusion models to defend against adversarial attacks, and following-up studies to extend it to certified defenses [7].
Adversarial attacks and defenses also extend to 3D point clouds.
Xiang et al.
[72] first demonstrated that point cloud recognition models are vulnerable to adversarial attacks.
They also introduced different threat models like point shifting and point adding attacks.
Wen et al.
[62] enhanced the loss function in C&W attack to achieve attacks with smaller perturbations and Hamdi et al.
[21] presented transferable black-box attacks on point cloud recognition.
[63] pioneered to study the point dropping attack under both withe- and black-box settings.
Zhou et al.
[90] and Dong et al.
[14] proposed to purify the adversarial point clouds by input transformation and adversarial detection.
However, these methods have been successfully by [51] through adaptive attacks.
Moreover, Liu et al.
[28] made a preliminary investigation on extending countermeasures in the 2D space to defend against simple attacks like FGSM [18] on point cloud data.
Sun et al.
[50] conducted a more thorough study on the application of self-supervised learning in adversarial training for 3D point clodu recognition.
Besides adversarial training, advanced purification methods IF-Defense [66] and LPC [25] were proposed to transform the adversarial examples to the clean manifold.
Certified defenses for point clouds have been focusing on the adversarial transformations and deformations [30, 10, 40].
In this work, we present PointDP , that utilizes 3D diffusion models to purify adversarial point clouds that delivers both state-of-the-art (SOTA) empirical and certified robustness.
We also demonstrate that standard adversarial training suffer from strong black-box attacks and SOTA purification methods ( i.e., IF-Defense and LPC) are vulnerable to PGD-styled adversaries (§ 4.3).
PointDP : D IFFUSION - DRIVEN P URIFICATION AGAINST 3D A DVERSARIES
We first introduce the preliminaries of diffusion models and then propose PointDP that first introduces noise to the adversarial 3D point clouds, followed by the forward process of diffusion models to get diffused point clouds.
Purified point clouds are recovered through the reverse process (§-3.2).
Next, we follow [36] to apply the adjoint method to backward propagate through SDE for efficient gradient evaluation with strong adaptive attacks (§ 3.3).
3. P RELIMINARIES
In this section, we briefly review the background of conditional diffusion models in 3D vision tasks.
Following [31], we use the discrete-time formulation of the forward and reverse processes.
Given a clean point cloud sampled from the unknown data distribution x 0 ∼ q ( x ) , the forward process of the diffusion model leverages a fixed Markov chain to gradually add Gaussian noise to the clean point cloud x 0 over a pre-defined N time steps, resulting in a number of noisy point clouds { x 1 , x 2 , · · · , x N } .
Mathematically, the forward process is defined as:
[Equation]
where β n is a scheduling function of the added Gaussian noise, satisfying 0 < β 1 , · · · , β N < 1 .
The reverse process, in contrast, is trained to recover the diffused point cloud in an iterative manner.
3D Point clouds have less semantics than 2D images due to the lack of texture information.
Therefore, point cloud diffusion models leverage a separate encoder e to as a latent feature z x = e ( x ) as a condition to help recover the clean point cloud:
[Equation]
where µ θ denotes the approximated mean value parameterized by a neural network.
The training objective is to learn the variational bound of the negative log-likelihood [31].
In practice, we jointly train the encoder e with the noise predictor ϵ θ ( x n , n, z ) .
Similar to the DDPM model [12], we can conduct the sampling by reparameterizing µ θ as
[Equation]
where α n = (cid:81)
ni
=1 (1 − β i ) .
It is worth noting that point cloud diffusion models have recently achieved SOTA performance on generating and autoencoding 3D point clouds, which provides us with opportunities for adversarial point cloud purification.
3. D ESIGN OF PointDP
Overview .
Figure 1 illustrates the pipeline of PointDP .
Different from Nie et al.
[36] use unconditional diffusion model to remove the adversarial effect for 2D images, we use the conditional diffusion models as mentioned in § 3.1.
Specifically, PointDP first adds pre-quantified Gaussian noise to the input data and then leverage a well-trained diffusion model to purify the noisy point cloud step by step to recover the clean point cloud.
The reversed point cloud will be finally fed into the recognition model for the classification task.
Note that we do not aim at designing new point cloud diffusion models, but instead propose a novel purification pipeline with rigorous evaluations as our main contributions.
Following [36], in order to backpropagate through the forward and reverse processes for computing gradients, we first convert the discrete-time formulation defined in Eqs.
(1) and (2) to its continuoustime counterpart, i.e. , the forward and reverse stochastic differential equations (SDEs) [48].
Let x a be an adversarial example w.r.t.
the pristine classifier f , we initialize the input of the forward diffusion process as x a , i.e., x 0 = x a .
Also, let x ( nN ) :
= x n , β ( nN ) :
= β n , α ( nN ) :
= α n , and t ∈ { 0 , 1 , · · · , N − 1 N } .
The forward diffusion process from t = 0 to t =
t ∗ ∈ (0 , 1) can be solved by:
[Equation]
where ϵ ∼ N (0 , I ) .
We leverage Eq.
2 to recover the clean point clouds.
Equivalently, the truncated reverse process can be also solved by the SDE solver in [36] (denoted as sdeint ):
[Equation]
where the six inputs are initial value, drift coefficient, diffusion coefficient, Wiener process, initial time, and end time [36], with the definitions:
[Equation]
and the score function s θ is derived from ϵ θ ( x n , n, z ) in Eq.
(3) by following:
[Equation]
Note that the hyper-parameter t ∗ and N trades off the denoising performance and efficiency.
We empirically choose t ∗ = 0 .
15 and N = 200 in our study, which has shown satisfactory results in our evaluation (§ 4).
We also conduct ablation studies on t in § 4.2.
3. A DPATIVE A TTACKS ON PointDP
PointDP is a pre-processing module that purifies the adversarial perturbations.
[2] have shown that input transformation-based methods can be broken by specifically designed attacks.
Therefore, it is essential to model the adaptive attacks on PointDP to demonstrate its lower-bound adversarial robustness.
We thus formulate two types of adaptive attacks on PointDP .
Attack on Latent Feature .
As PointDP utilizes conditional diffusion models for adversarial purification, the latent feature z is a good candidate for adversaries to launch attacks.
Concretely, adversaries can set the goal to maximize some distance metric D between the latent feature of the optimized adversarial examples and the oracle latent feature of clean inputs z oracle .
Without loss of generality, the adaptive attacks can be formulated as:
[Equation]
where x s denotes the adversarial examples from the s -th step, Proj is the function to project the adversarial examples to the pre-defined space S , and α is the attack step size.
We choose two distance metrics in our study, where the first one is the KL divergence [17] and the other is the the ℓ 1 norm distance.
In our evaluation (§ 4), we report the lowest accuracy achieved under attacks with two distance metrics.
Adaptive Attack .
We follow [36] to formulate the adaptive attack as an augmented SDE process.
We re-state the attack formulation as below.
For the SDE in Equation 5, the augmented SDE that computes the gradient
[Equation]
) of backward propagating through it is given by:
[Equation]
where ∂
L ∂ ˆ x (0) is the gradient of the objective L w.r.t.
the output ˆ x (0) of the SDE in Equatrion 5), and
[Equation]
where 1 and 0 denote the vectors of all ones and all zeros, respectively.
Nie et al.
[36] have demonstrated that such approximation aligns well with the true gradient value.
Therefore, we leverage this adaptive attack formulation for our evaluation.
E XPERIMENTS AND R ESULTS
In this section, we first introduce our experimental setups (§ 4.1).
We then present the standard robustness evaluation of PointDP (§ 4.2).
We next show that how the SOTA adversarial training and adversarial purification methods fail under various strong attacks (§ 4.3).
We finally conduct stress test on PointDP to show its actual robustness under various stronger adaptive attacks (§ 4.4).
4. E XPERIMENTAL S ETUPS
Datasets and Network Architectures .
We conduct all the experiments on the widely used ModelNet40 point cloud classification benchmark [67], consisting of 12,311 CAD models from 40 artificial object categories.
We adopt the official split with 9,843 samples for training and 2,468 for testing.
We also uniformly sample 1024 points from the surface of each object and normalize them into an edgelength-2 cube, following most of the prior arts [41].
As mentioned before, there are various backbones for 3D point cloud recognition in the literature.
To demonstrate the universality of PointDP , we select six representative model architectures including PointNet [41], PointNet++ [42], DGCNN [61],
PCT [20], CurveNet [69], and PointMLP [32].
These backbones either have representative designs
[Table]
Adversarial Attacks .
As briefly described in § 2.2, adversarial attacks could be roughly categorized into C&W- and PGD-styled attacks.
C&W attacks involves the perturbation magnitude into the objective term of the optimization procedure by Lagrange multiplier, while PGD attacks set the perturbation magnitude as a firm constraint in the optimization procedure.
Moreover, adversarial attacks by ℓ p norm as the distance metric for the perturbation.
Although a number of attacks measure
Chamfer and Handoff “distances” in 3D point cloud [68], they are not formal distance metrics as they do not satisfy the triangular inequality.
Therefore, we still leverage ℓ 2 and ℓ ∞ norm, following most defense studies in both 2D and 3D vision tasks [8, 50].
We also have designed adaptive attacks on our proposed method § 3.3.
Besides naive C&W and PGD attacks, we leverage specific attacks designed to break the robustness of point cloud recognition such as k NN [57] and AdvPC [21].
We also apply strong adaptive AutoAttack [11] ( i.e. , APGD) in our evaluation.
Moreover, we use SPSA [58] and
Nattack [27] as black-box adversaries, followed by the suggestion of Carlini et al.
We also leverage EOT-AutoAttack.
Point adding (PA) and dropping/dropping (PD) attacks are also evaluated in our study, followed by the setups in [50].
We set the attack steps to 200 to maximize the adversarial capability and follow the settings in [50] for other attack parameters by default.
Evaluation Metrics .
We leverage two main metrics to evaluate the performance of our defense proposal, which are standard and robust accuracy.
The standard accuracy measures the performance of the defense method on clean data, which is evaluated on the whole test set from ModelNet40.
The robust accuracy measures the performance on adversarial examples generated by different attacks.
Because of the high computational cost of applying adaptive and black-box attacks to our method
[Table]
we evaluate robust accuracy for our defense on a fixed subset of 128 point clouds randomly sampled from the test set.
Notably, robust accuracies of most baselines do not change much on the sampled subset, compared to the whole test set.
We evaluate the robust accuracy on the whole test set for other adversarial attacks with acceptable overhead ( e.g. , C&W and PGD attacks)
[Table] [Table]
Baseline .
Without any defense applied to the original recognition models, the robust accuracy is mostly 0% for all models under ℓ 2 and ℓ ∞ based attacks (see Appendix A).
DGCNN exceptionally achieves 64% on ℓ 2 -based PGD, AutoAttack, respectively, due to its dynamic clustering design, which adaptively discards outlier points.
PA and PD are two weaker attacks and Table 1 presents the robust accuracy against these two attacks.
4. E XPERIMENT R ESULTS OF PointDP
In this section, we first present the evaluation results of PointDP under attacks on the plain models.
We train the diffusion and 3D point cloud recognition models in a sequential order.
Table 2 presents the detailed results of PointDP against attacks on six models.
We find that PointDP overall achieves satisfactory results across all models and attacks.
The average robust accuracy against adversarial attacks is above 75%.
We observe a drop on the clean accuracy for the chosen models due to the imperfect reconstruction of diffusion models.
As mentioned before, diffusion models for 3D point cloud is a more difficult task than 2D image diffusion, which may lead to partial semantic loss.
The average drop of standard accuracy is 4.9%.
We find that DGCNN still achieves the best robustness combined with PointDP , which has a 79.9% of robust accuracy.
We further compare the performance of PointDP with adversarial IF- LPC [Table]
We also ablate the effect of diffusion steps in PointDP .
Figure 3 shows the averaged evaluation results of point shifting, adding, and dropping attacks with PGD adversary over the selected models.
Point shifting attack is much stronger than point adding and dropping attacks.
It is, thus, more sensitive to the diffusion steps in PointDP .
We find that the robust accuracy converges after the number of diffusion steps n ≥ 30 (or equivalently t ≥ 0 . 15 ).
Therefore, we choose to use t ∗ = 0 .
15 in the main evaluation of our study.
Adversarial purification inevitably introduces overhead during model inference, we benchmark the computation of PointDP and other baselines using an RTX3080 GPU and a batch size of 32.
Tablepresents the results, where PointDP achieves the most negligible cost than existing SOTA methods, which is a 27 × speed-up than IF-Defense.
4. F AILURE OF S TATE - OF - THE -A RT D EFENSES
In this section, we demonstrate how lately proposed defense solutions fail when encountered with stronger (adaptive) adversarial attacks on 3D point cloud recognition models.
[Figure]
Adversarial training (AT) has been applied to PointNet, DGCNN
[Figure]
and PCT with the help of self-supervised learning [50] that achieves satisfactory robustness.
Such observations are consistent with the performance of AT for 2D perception models.
However, we find that AT is, in fact, a weak defense solution in 3D perception models.
First, as acknowledged by [50], point cloud models ( e.g., PointNet++ and CurveNet) often leverage different sampling strategies to select anchor points, like furthest point sampling (FPS).
Such sampling involves high randomness.
AT either cannot converge with different random seeds in each iteration or overfits to a single random seed.
Therefore, AT cannot fit these models.
Moreover, we discover that the k NN layers will cause severe gradient obfuscation in point cloud models as well.
Different from standard training process that only needs the gradient of model parameters w.r.t.
the loss function ∂ L ∂ w , AT additionally requires the gradient flow to the input ∂ L ∂ x .
As shown in Line 5 from Figure 4, k NN essentially applies top- k for point selection.
Top-
k is a general case for max pooling that does not have trainable model parameters.
Therefore, it will not affect the standard training.
However, top- k is not differentiable w.r.t.
the input [Table] x .
Therefore
[Table]
the implementation simplifies the gradient backward propagation through the top- k function as an indexing function to make the chain propagation smooth:
[Equation]
However, such simplification still cannot resolve the differentibility issue of the top- k function [77].
Different from 2D model usually at most use one layer of max pooling, the heavy usage of k NN layers in DGCNN and PCT will drastically hinder the actual gradient flow.
As mentioned in § 4.1, we exploit black-box SPSA and Nattack to validate our findings.
Table 4 presents the results of AT.
SPSA and Nattack can greatly lower the average robust accuracy (7.8%) than white-box attacks (55.6%) on
DGCNN and PCT.
This phenomenon exactly reveals gradient obfuscation as white-box attacks rely on the backward propagated gradient to succeed.
The results demonstrated that the approximated gradients from black-box attacks are more accurate than the propagated ones.
PointNet, however, achieves better robustness under black-box attacks because it only has one max pooling layer and does not employ k NN layers.
[Table]
Existing purification-based defenses against 3D adversarial point clouds mainly leverage
C&W-styled attacks in their evaluation.
C&W attacks utilize the method of Lagrange multipliers to find tractable adversarial examples while minimizing the magnitudes of the perturbation.
From the perspective of adversary, such attacks are desirable due to their stealthiness, while this does not hold from a defensive view.
Defense methods should be evaluated against strong adaptive attacks [6].
DUP-Net [90] is a pioneer study that uses statistical outlier removal and a upsampler network for purification, but it was adaptively attacked by [51].
We thus present the evaluation results of DUP-Net in Appendix A. IF-Defense and
LPC are the SOTA adversarial purification methods for 3D point cloud models.
We leverage PGD and AdvPC attacks, which assign constant adversarial budget in the adversarial optimization stage.
We follow the original setups of IF-Defense and LPC in our study.
Such evaluation is stronger than C&W attacks, while we note that they are not strict adaptive attacks since the adversarial target is still the classifier itself.
Similar to PointDP , IF-Defense can be pre-pended to any point cloud classifier, but LPC uses a specific backbone.
Table 5 presents the detailed evaluation results of IF-Defense under various settings and attacks.
We find that PointDP achieves much better robustness than IF-Defense, which is on average an 12.6% improvements.
However, IF-Defense achieves slightly higher clean accuracy (4.9%).
This is because IF-Defense leverages SOR to smooth the point cloud [90].
However, such an operation has been demonstrated to be vulnerable [51].
With specific adaptive attacks, there will be a even larger drop of robust accuracy for IF-Defense.
Figure 2 shows the comparison among PointDP and existing methods.
PointDP overall achieves the best performance than prior arts, which are 12.6% and 40.3% improvements than IF-Defense and
LPC, respectively.
We find that even without adaptive attacks, adversaries with constant budgets can already hurt the robust accuracy by a significant gap.
This suggests that IF-Defense and LPC fail to deliver strong robustness to 3D point cloud recognition models.
Especially, LPC appears in the proceedings of CVPR 2022, but actually achieves trivial robustness, indicating that a rigorous evaluation protocol is highly required in this community.
4. D EFENSE AGAINST A DAPTIVE T HREATS
We have so far illustrated that state-of-the-art defenses can be easily broken by (adaptive) adversarial attacks and PointDP consistently achieves the best robustness.
In this section, we further extensively evaluate the robustness of PointDP on even stronger adaptive attacks to demonstrate the actual robustness realized by PointDP .
As mentions in § 4.1, we leverage two types of adaptive attacks in our study
[Table]
and Table 6 presents their results.
We also leverage black-box SPSA and Nattack to validate our results.
We find that BPDA-PGD the strongest adaptive attacks, which align well with previous study on 2D diffusion-driven purification [36].
Even though with strong adaptive attacks,
PointDP still achieves much better robustness.
Besides, black-box attacks are much less effective.
Although we admit that PointDP still relies on gradient obfuscation, the extremely high randomness will hinder the black-box adversaries finding correct gradients. We also ablate the effectiveness of
PointDP with larger attack budgets in Appendix A
[Table]
[Table]
D ISCUSSION AND C ONCLUSION
Adversarial robustness has been well-established in 2D vision tasks, where Carlini et al.
[6] and many other researchers have devoted significant efforts to setting up a rigorous evaluation protocol.
In this study
[Table]
we emphasize that this evaluation protocol should be strictly followed in the 3D point cloud robustness study as well.
Counter-intuitively, we have demonstrated that standard adversarial training (AT) is not a good candidate to deliver robustness against strong black-box adversaries because gradient obfuscation in 3D point cloud architectures will hinder the inner maximization stage from making real progress in AT.
We propose PointDP as an adversarial purification strategy to mitigate the robustness loss in the 3D space.
We want to clarify that almost all purification methods (including PointDP ) still depend on gradient obfuscation to mislead adaptive attackers.
However, we argue that proper usage of gradient obfuscation could still serve as a good defense, as long as the obfuscation is sophisticated enough.
The multi-step purification in diffusion models adds extremely high-level randomness that EOT [3] and BPDA [2] attacks are hard to model.
Therefore, we believe our extensive evaluation reveals the actual robustness of PointDP .
Our evaluation also unveils a concerning fact that existing defenses in the 3D domain could be easily broken by strong attacks.
Therefore, we hope our evaluation protocol sets a standard for robustness assessment in this community, i.e., a defense study should strictly follow a formal distance metric and leverage strong attacks including PGD, black-box, and adaptive attacks to evaluate its actual robustness.
Limitation .
Mitigation solutions to adversarial attacks are critical and essential for modern machine learning systems.
Given that 3D point cloud is heavily adopted in safety-critical applications, we believe our study is valuable in demonstrating the vulnerabilities of existing SOTA defenses.
PointDP also.
On the other hand, diffusion models needs multiple steps in the reverse process to recover the point cloud and hinder adaptive attacks, which will incur additional computational overhead.
PointDP also limits itself to empirical robustness without theoretical guarantees.
In this paper, we propose PointDP , an adversarial purification method against attacks on 3D point cloud recognition.
We showed that adversarial training and prior purification methods are vulnerable to strong attacks.
We then performed extensive rigorous evaluations to validate that PointDP outperforms existing SOTA methods by a significant margin (12.6%-40.3%) in robust accuracy.
E THICS S TATEMENT
As we continuously mentioned in our main paper and acknowledged in other studies [8, 6] adversarial robustness is critical to the real-world deployment of machine learning models, especially for safetyrelated applications.
Point cloud data is heavily used in many such applications like autonomous driving, robotics, and medical imaging.
Although various defenses were proposed in the literature, even in top-tier conferences like ICCV [90], CVPR [25] and NeurIPS [50] we find that they actually can be broken by carefully-designed strong attacks.
Therefore, the first contribution of study is very beneficial for the 3D point cloud community to illustrate how existing state-of-the-art fail to deliver real robustness, as we have raised the attention for the 3D point cloud community to focus on actual robustness under strongest adaptive attacks.
PointDP is also beneficial since we have leveraged the most rigorous evaluation protocol to test its robustness.
We follow the licenses of usage for all the public models and datasets in our study.
R EPRODUCIBILITY S TATEMENT
To ensure the reproducibility of our results, we have provided our codebase in the supplementary materials and all of our results are based on consistent random seed in our implementation.
A E VALUATION D ETAILS
As mentioned in § 4.1, the robust accuracies of the unprotected base models are mostly 0%.
Tablepresents the detailed results.
[Table]
[Table]
We also include [63] in our evaluation.
[63] proposed
ISO attack that iteratively drops the most salient points.
This setting is very similar to our point-dropping (PD) adversary evaluated in § 4.2.
The difference is that [63] leverages a heuristic way to determine critical points, but PD uses the gradient that backward propagates to each point to select the critical points.
[63] only works for PointNet because i) both [63] and PointNet are very first explorations in the area of 3D point cloud recognition and ii) PointNet utilizes global max pooling so that only the critical points will affect the prediction results.
We evaluate ISO under PointNet with an attack budget of 200 points; the results are shown in the Table 8.
We find that ISO is a weaker attack than PD as it by design restricts its attack capability, which is good for an attack paper.
However, it cannot show the worst-case robustness of a defense proposal.
We also evaluate DUP-Net with IF-Defense and PointDP under ℓ ∞ norm PGD attacks using different attack budgets.
As Table 9 presents
[Table]
DUP-Net is vulnerable to such attacks due to sensitivity of the upsampler network to ℓ ∞ norm noises [51].
The robust accuracy for LPC is 27.8% and 19.1% for ϵ = 0 .
075 and ϵ = 0 . 1 , respectively.
Even with these extremely large distortions, PointDP achieves the strongest robustness, outperforming existing SOTA by a very large margin
[Table]
[Table]