Isometric 3D Adversarial Examples in the Physical World
Abstract 3D deep learning models are shown to be as vulnerable to adversarial examplesas 2D models.
However, existing attack methods are still far from stealthy andsuffer from severe performance degradation in the physical world.
Although 3Ddata is highly structured, it is difficult to bound the perturbations with simplemetrics in the Euclidean space.
In this paper, we propose a novel ϵ -isometric ( ϵISO) attack to generate natural and robust 3D adversarial examples in the physicalworld by considering the geometric properties of 3D objects and the invariance tophysical transformations.
For naturalness, we constrain the adversarial exampleto be ϵ -isometric to the original one by adopting the Gaussian curvature as asurrogate metric guaranteed by a theoretical analysis.
For invariance to physicaltransformations, we propose a maxima over transformation (MaxOT) method thatactively searches for the most harmful transformations rather than random onesto make the generated adversarial example more robust in the physical world.
Experiments on typical point cloud recognition models validate that our approachcan significantly improve the attack success rate and naturalness of the generated3D adversarial examples than the state-of-the-art attack methods.
Introduction
Deep neural networks (DNNs) have achieved unprecedented performance on numerous tasks, including 2D image classification [33, 25, 27] and 3D point cloud recognition [48, 49, 68].
However,
DNNs are vulnerable to adversarial examples [58, 20] — inputs crafted by adding imperceptibleperturbations to original examples that can cause misclassification of the victim model.
Adversarialexamples are prevalent in various domains beyond images, including texts [28], speeches [84] and3D objects [73].
As deep 3D point cloud recognition has been adopted in safety-critical applications,such as autonomous driving [7, 85], robotics [65, 92], medical image processing [59], it has garneredincreasing attention to studying the adversarial robustness of 3D point cloud recognition models [8].
However, the existing adversarial attacks on point cloud recognition models are still far from stealthyand suffer from drastic performance degeneration in the physical world.
There is usually a trade-offbetween the stealthiness and the real-world attacking performance, making it challenging to achievethe best of both worlds.
Early methods [80, 73, 42] adopt gradient-based attacks to add, remove, andmodify points, but they are limited to digital-world attacks.
The KNN attack [63] and the GeoA 3attack [69] constrain the smoothness of the adversarial point clouds and reconstruct adversarialmeshes from the point clouds that can be 3D-printed in the physical world.
Although these worksdemonstrate successful physical attacks, point cloud reconstruction introduces large noises and errors,resulting in low attack success rates and unnaturalness of the adversarial objects in the physicalworld.
Mesh Attack [88] is recently proposed to perturb the mesh representation of 3D objects,which improves the success rate but often creates large distortions that can be easily detected byhumans as anomalies, as illustrated in Fig.
Overall, it is difficult to achieve both the naturalness
[Figure]
and effectiveness of 3D adversarial attacks in the physical world, which we think is largely due to thelack of an appropriate metric to characterize the naturalness of 3D data.
To address these issues, we propose an ϵ -isometric ( ϵ -ISO ) attack method to generate natural androbust 3D adversarial examples in the physical world against point cloud recognition models.
Theϵ -ISO attack improves the naturalness of the 3D adversarial example by constraining it to be ϵisometric (see Definition 1) to the original one, which guarantees the consistency between theintrinsic geometric properties of two 3D objects [11].
We theoretically demonstrate that Gaussiancurvature (see Definition 2) can be used to provide a sufficient condition to ensure that two surfacesare ϵ -isometric.
Due to the computable and differentiable nature of Gaussian curvature, we adopt itas a new regularization loss to practically generate natural 3D adversarial examples.
To improve therobustness of 3D adversarial examples under physical transformations, we further propose a maximaover transformation ( MaxOT ) method that actively searches for the most harmful transformationsrather than random ones [3] for optimization.
Armed with Bayesian optimization that provides betterinitialization of the transformations, MaxOT is able to find a set of diverse worst-case transformations,leading to improved performance of the 3D adversarial examples in the physical world.
We conduct extensive experiments to evaluate the performance of our method on attacking typicalpoint cloud recognition models [48, 49, 68].
Results demonstrate that, in comparison with the alternative state-of-the-art attack methods [63, 69, 88], ϵ -ISO attack achieves higher success rates, whilemaking the generated adversarial examples more natural and robust under physical transformations.
A physical-world experiment is conducted by 3D-printing the adversarial meshes and re-scanning theobjects for evaluation, which also validates the effectiveness of our method.
Related work
Deep learning on 3D point clouds.
Deep 3D point cloud recognition [48, 21, 74, 77, 82, 50, 61]has emerged in recent years with various applications in many fields, such as 3D object classification [55, 37, 83, 86], 3D scene segmentation [22, 67, 78, 26], and 3D object detection in autonomousdriving [7, 95, 79].
One of the pioneering works is PointNet [48], which directly applies a multilayerperceptron to learn point features and aggregates them in an efficient way using a max-pool module.
PointNet++ [49] and a large number of later works [14, 43, 81] are built on PointNet to furthercapture fine-grained local structure information from the neighborhood of each point.
Recently, someworks have focused on designing special convolutions on 3D domains [4, 39, 44, 62] or developinggraph neural networks [19, 52, 53, 68] to improve point cloud recognition.3D adversarial attacks.
Following the previous studies on adversarial machine learning in the 2Dimage domain [58, 20, 34, 2, 6, 13], many works [73, 41, 5] apply adversarial attacks to the 3D pointcloud domain.
Xiang et al.
[73] proposed point generation attacks by adding a limited number ofsynthetic points to the point cloud.
Recently, more studies [70, 91] use gradient-based attack methodsto identify key points from the point cloud for deletion.
More point perturbation attacks [24, 45,89, 12] learn to perturb the xyz coordinates of each point through a C & W framework [6] based onmetrics defined in the Euclidean space.
Zhao et al.
[90] attack by the isometric transformations in the
Euclidean space such as rotation.
It is worth noting that we consider isometric mappings between surfaces, which is essentially different from [90].
Later works [60, 1, 76, 87] further apply iterativegradient methods to achieve more advanced adversarial perturbations.
Besides, other works considergenerative models [94, 10], 3D data attacks [88, 64, 56, 57], attacks against LIDAR [36, 30, 35],backdoor attacks [38, 75], etc., in the 3D domain.
However, the existing attacks on 3D point cloudrecognition are still far from stealthy and the only three methods that consider the physical-worldattacks [63, 69, 88] are not very effective.
In this paper, we surpass the performance of previousmethods by proposing a novel ϵ -isometric ( ϵ -ISO) attack method to generate natural and robust 3Dadversarial examples in the physical world.
Methodology
We now formally present ϵ -ISO attack .
We first present the general problem formulation, andthen describe
how ϵ -ISO attack enhances the imperceptibility and robustness of the generated 3Dadversarial samples, respectively.
3.1 Problem formulation
To generate 3D adversarial objects in the physical world, it is more straightforward to perturb the meshrepresentation of 3D objects rather than point clouds [88] since the reconstruction process can incurlarge errors [63, 69].
A mesh M = ( V , F ) is an approximate shape representation of its underlyingsurface, where V :
= { v i } n v i =1 is the set of n v vertices of xyz coordinates, and F :=
{ z i } n f i =1 is the setof n f triangle faces represented by the indices of vertices.
We let S denote a differentiable samplingprocess such that P :=
S ( M ) ∈
X is the point cloud obtained by sampling on the mesh surface.
Welet y ∈ Y denote the corresponding ground-truth label of M as well as P .
In this paper, we focus on the challenging targeted attacks against deep 3D point cloud classificationmodels [48, 49, 68].
Given a point cloud classifier f : X → Y , the goal of the attack is to generatean adversarial mesh M adv =
( V adv , F ) for the original one M with vertex perturbations such thatthe sampled point cloud P adv :=
S ( M adv ) will be misclassified by f as the target class y ∗ ( ̸ = y ) .
The added perturbation ∆ := V adv − V can be understood as a vector describing the direction andmagnitude to shift the vertices.
In general, the perturbation should be small to make the adversarialmesh M adv inconspicuous under human inspection.
Thus, the optimization problem of generatingthe adversarial mesh can be generally formulated as
[Equation]
where L f is the loss that facilitates the misclassification of P adv to y ∗ , R is the regularization termthat minimizes a perceptibility distance between M adv and M , and β is a balancing hyperparameterbetween these two losses.
In this paper, we try to develop a stealthy and robust attack method byproposing a new regularization loss R based on Gaussian curvature with theoretical guarantees toremain the naturalness as well as a new attacking loss L f to enhance the robustness of the generated3D adversarial objects under physical transformations.
R and L f will be introduced in the following.
3.2 ϵ -ISO attack
Most of the existing 3D adversarial attacks only consider the constraints R defined in the Euclideanspace [16, 82, 23].
The generated adversarial examples have noticeable point outliers that causespikes to appear on the object’s surface, thus losing the naturalness.
Moreover, the outliers are moreeasily removed and defended against.
The main reason is that the existing methods do not considerthe geometric properties of the 3D objects.
In differential geometry, isometric mapping guaranteesthe consistency of the intrinsic geometric features of two objects [11].
Therefore, we propose aconstraint loss R based on ϵ -isometric mapping to restrict the naturalness of 3D adversarial objects.
We first give the definition of ϵ -isometric below.
Definition 1.
Let S and ˜ S denote two surfaces of R 3 .
A diffeomorphism φ : S → ˜ S is called anϵ -isometric mapping if there exists a constant n such that it takes any local curve C in S to a curve˜ C = φ ( C ) in ˜ S satisfying (1 − nϵ ) s ( C ) < s ( ˜ C ) <
(1 + nϵ ) s ( C ) where s ( · ) is the length.
Thesurfaces S and ˜ S are then said to be ϵ -isometric.
As shown in Fig.
2, Fig.
2(a) is the original mesh, Fig. 2(b) is the adversarial mesh generated by KNNattack and GeoA 3 attack, and Fig. 2(c) is the adversarial mesh generated by Mesh Attack.
These
[Figure]
three methods only consider the constraints defined in the Euclidean space, and the curve lengths ofthe generated adversarial samples differ greatly from those of the original samples, which are notϵ -isometric and lose naturalness.
Fig. 2(d) is the adversarial mesh generated by our proposed ϵ -ISOattack.
We consider the geometric features of 3D objects to generate natural adversarial examples byconstraining them to be ϵ -isometric to the original examples (i.e., the curve length of the resultingadversarial examples varies very little).
However, it is intractable to directly optimize the adversarialmesh to be ϵ -isometric as the original one.
Therefore, we introduce the definition of the Gaussiancurvature [11] of the surface.
Definition 2.
Let S be a surface of R 3 parameterized by r :=
r ( u, v ) =
[ x ( u, v ) , y ( u, v ) , z ( u, v )] ,where ( u, v ) ∈ R 2 .
We let r u , r v denote the partial derivatives of r w.r.t.
u and v , r uu , r uv , r vvdenote the second partial derivatives of r , and ∧ , ⟨· , ·⟩ denote the outer product and inner product,respectively.
The parametrization thus defines unit normal vectors n := r u ∧ r v | r u ∧ r v | of the surface S .
Wedenote the eigenvalues of the coefficient matrix of the Weingarten map (cid:20) L
M M
N (cid:21)
[Equation]
E
F F
G (cid:21)
[Equation]
as k 1 and k 2 , where E = ⟨ r u , r u ⟩ , F = ⟨ r u , r v ⟩ and G =
⟨ r v , r v ⟩ are coefficients of the firstfundamental form and L = ⟨ r uu , n ⟩ , M = ⟨ r uv , n ⟩ and N = ⟨ r vv , n ⟩ are coefficients of the secondfundamental form.
The Gaussian curvature is defined as K = k 1 k 2 =
LN − M 2 EG − F 2 .
Remark 1.
The Gaussian curvature intrinsically measures bending degree of the surface reflected bythe Gaussian mapping.
Let the area element of the surface S be dA = ⟨ r u ∧ r v , n ⟩ dudv and thearea element under the Gaussian mapping g : S ∈ R 3 → S 2 r ( u, v ) → n ( u, v ) be dA ′ = ⟨ n u ∧ n v , n ⟩ dudv .
From n u ∧ n v = K r u ∧ r v , we obtain
[Equation]
Eq.
(2) illustrates that the geometric meaning of Gaussian curvature is the ratio of the area of thedomain at the point P on the surface S and the area of the domain at the corresponding point underthe Gaussian mapping, i.e., bending degree of the surface reflected by the Gaussian mapping.
Based on Definition 1 and 2, we have the following theorem.
Theorem 1 (proof in Appendix A) .
Let S and ˜ S denote two surfaces of R 3 ; φ : S → ˜ S denote adiffeomorphism that takes a point v in S to point v ′ =
φ ( v ) in ˜ S ; and K ( · ) be the Gaussian curvatureof the points.
If | K ( v ) − K ( v ′ )
| < ϵ for any point v , then the surfaces S and ˜ S are ϵ -isometric.
Theorem 1 indicates that to make two surfaces ϵ -isometric, one can constrain their Gaussian curvatures.
Since the Gaussian curvature is computable and differentiable w.r.t.
vertices, we adopt it to constrainthe naturalness of 3D adversarial meshes as
[Equation]
where φ ( · ) is the corresponding mapping between vertices in V and V adv .
We follow the GaussBonnet formula [9] to calculate the Gaussian curvature of the vertices as
[Equation]
where A ( · ) is the area of the vertex neighborhood, i.e., the area of the polygonal region joined by theconsecutive midpoints of triangles incident on the vertex of interest, N ( v ) is the set of faces containingv , and θ i ( v ) is the interior angle of the face at vertex v .
Note that the value of (cid:80) i ∈ N ( v ) θ i ( v ) for aplane is 2 π and the Gaussian curvature is 0 .
The more curved the surface, the smaller the value of 161 (cid:80) i ∈ N ( v ) θ i ( v ) and the larger the Gaussian curvature.
In addition, we prevent the generated adversarial meshes from self-intersecting by using the Laplaceloss [17], denoted as R Lap ( M adv ) , which represents the distance between a vertex and its nearestneighbor’s center of mass, and the edge length loss [66], denoted as R edge ( M adv ) , which representsthe smoothness of the surface.
Thus, the overall regularization term can be expressed as:
[Equation]
where λ 1 , λ 2 and λ 3 are balancing hyperparameters.
3.3 Improving the robustness under physical transformations
Besides concerning the naturalness of 3D adversarial examples, we further enhance their robustnessunder physical transformations, such as 3D rotations, affine projections, cutouts, etc.
A commonmethod is the expectation over transformation (EOT) algorithm
[3], which optimizes the adversarialexample over the distribution of different transformations.
However, it is still challenging to maintainthe attacking performance under various physical transformations.
As shown in the experiments, afterusing the EOT algorithm, there are still some transformations that the generated adversarial examplesare not robust to, leading to a reduction of the attack success rate.
To address this issue, our key insight is to consider the worst-case transformations rather than their expectation, since if the adversarial examples are resistant to the most harmful physical transformations,they can also resist much weaker transformations, inspired by adversarial training [46].
Therefore,we propose a maxima over transformation (MaxOT) algorithm to actively search for physicaltransformations that maximize the misclassification loss.
The loss function L f is thus formulated as:
[Equation]
where T contains all possible transformations, T ∗ is a subset of transformations in T , and L ce is thecross-entropy loss.
Note that in Eq.
(6) we consider a subset of transformations T ∗ rather than asingle one because the loss landscape w.r.t.
transformations is largely non-convex and contains manylocal maxima [15].
Thus we aim to find a set of diverse worst-case transformations.
By integrating
Eq.
(6) into Eq.
(1), it forms a minimax optimization problem, where the inner maximization aimsto find physical transformations that maximize the cross-entropy loss, while the outer minimizationaims to optimize the adversarial example with the worst-case transformations.
3.3.1 Bayesian optimization
To solve problem (6), we search for the worst-case transformations one by one.
Given an initializedtransformation, we perform gradient-based optimization to update the transformation parameters(e.g., angles for rotations).
However, randomly selecting initialized transformations is ineffectivesince the random initialization may drop into regions of weak transformations, which limits theexploration of the space of all transformations.
To address this issue, we propose to adopt the
Bayesian optimization [18, 54] to better break the dilemma between exploration and exploitation tofind more appropriate initialized transformations.
Bayesian optimization is an efficient method for solving global optimization problems consistingof two key components: a surrogate model, such as a Gaussian process (GP) [51] or a Bayesianneural network (BNN) [32], which models the unknown objective; and an acquisition functionα ( · ) , which is maximized by balancing exploitation and exploration to recommend the next querylocation.
We choose GP as a surrogate model, which provides a Bayesian posterior distribution to
[Figure]
describe the objective function f ( t ; D n ) ∼ GP ( µ ( D n ) , Σ ( D n , D n )) , where f ( t ; D n ) is the modeling of the unknown function L ce ( t ( S ( M adv )) , y ∗ ) and D n = { t i , L ce ( t i ( S ( M adv )) , y ∗ ) } n i =1is the n samples observed so far.
µ and Σ are the mean and covariance functions, respectively. We use the expected improvement (EI) [29, 47] acquisition function α EI ( t ; D n ) =
E f ( t ; D n ) ∼ GP ( µ ( D n ) , Σ( D n ,D n )) (cid:2) max (cid:0) f ( t ) − f + Dn , 0 (cid:1)(cid:3) , which measures the expected improvementof each point with respect to the current best value, where f + Dn = max i ≤ n f ( t i ) is the best valueobserved so far. Then, the target function f is sampled by arg max t α EI ( t ; D n ) to better explore thespace of all transformations by selecting the next query point t n +1 in the region where the predictionis high (i.e., high µ ( D n ) ) and the model is very uncertain (i.e., high Σ ( D n , D n ) ).
As shown in the overall algorithm (in Appendix B), we update the Bayesian posterior distribution ofthe objective f using the observations obtained from the previous iterations in the gradient descentprocess.
Then, we maximize the EI acquisition function α EI ( t ; D n ) to find the initial transformation.
From the initialization, we optimize the transformation parameters by gradient-based method to solveproblem (6).
This process is repeated for the number of transformations in the MaxOT algorithm.
Experiments
4.1 Experimental setup
Dataset.
We use the ModelNet40 [71] dataset in our experiments.
This dataset contains 12,
CAD models with 40 common object semantic categories in the real world.
We use the officialsplit [48, 49], where 9,843 examples are used for training and the remaining 2,468 examples are usedfor testing.
We follow [88] and get closed manifolds.
For adversarial attacks, we follow [69] andtake all the instances of the 40 categories that are well classified in the ModelNet40 testing set.
Victim models.
Following [69, 88, 40], we select three commonly used point cloud classificationnetworks as the victim models, i.e., PointNet [48], PointNet++ [49], and DGCNN [68].
Evaluation metrics.
To quantitatively evaluate the effectiveness of our proposed ϵ -ISO attack, wemeasure the attack success rate (ASR).
Besides, to measure the imperceptibility of different attackmethods, we use the Chamfer distance D c [16] and Gaussian curvature distance D g (i.e., Eq.
(3)) asevaluation metrics.
We also report the attack success rates on several existing defenses [80, 93, 72]and under the physical world to further validate the superiority of our ϵ -ISO attack.
Implementation details.
We use Adam [31] to optimize the objective of our proposed ϵ -ISO attack.
We use a fixed learning schedule of 250 iterations, where the learning rate and momentum arerespectively set as 0.01 and 0.9.
We assign the weighting parameters λ 1 = 1 . 0 , λ 2 = 0 .
2 andλ 3 = 0 . 8 .
The balancing parameter β is initialized as 1,500 and automatically adjusted by conducting10 runs of binary search following [6].
We uniformly sample 1,024 points from the adversarial mesh.
4.2 Pseudo physical attack
In this section, we compare our proposed ϵ -ISO attack with existing methods [63, 69, 88] that generateadversarial objects under the setting of white-box targeted attack, including the KNN attack [63], [Table] [Figure]
GeoA 3 attack [69], and Mesh Attack [88].
In Table 1, we compare different methods under themeasures of attack success rate (ASR), Chamfer distance D c [16] and Gaussian curvature distance
D g .
Our proposed ϵ -ISO attack outperforms all existing methods in terms of attack success rate and isalmost optimal in terms of Chamfer distance D c and Gaussian curvature distance D g , the evaluationmetrics for measuring imperceptibility.
These comparisons confirm the effectiveness of our proposedregularization term based on Gaussian curvature, to simultaneously achieve the strongest adversarialattack and the most natural perturbations.
Fig.
3 shows the adversarial objects generated by different methods.
The KNN attack and GeoA 3attack produce dense irregular deformations on the mesh surface, losing smoothness and regularity.
The Mesh Attack creates severe deformations and spurs, which are easily perceived by humansvisually.
In addition, the severe irregularities and distortions generated by the Mesh Attack canlead to self-intersecting phenomena in the mesh of 3D objects, i.e., the black surfaces in Fig. 3.
These self-intersecting surfaces can prohibit 3D-printing the adversarial objects in the physical world.
By considering geometric features of the object instead of using metrics in the Euclidean space,our ϵ -ISO attack produces adversarial samples without dense deformations, bursts, distortions, andself-intersection phenomena, while are stealthy and natural.
4.3 Robustness under physical transformation
To further verify the superior robustness of our proposed MaxOT algorithm under transformations inthe physical world, we compare MaxOT to EOT.
In Table 2, we compare the different methods underrotations to calculate the attack success rate, Chamfer distance D c and Gaussian curvature distance
D g .
Our proposed MaxOT algorithm outperforms the EOT algorithm in terms of attack success rate.
This comparison confirms our insight that the resistance to the most harmful physical transformationsis better than random ones.
In Fig.
4, we give the success rate w.r.t.
iterations curves for EOT and MaxOT of different victimmodels.
Our proposed MaxOT algorithm is always ahead of the EOT algorithm at different number ofiterations.
Moreover, we note that our proposed MaxOT algorithm can lead to higher attack successrate with a small number of iterations than that of EOT with a large number of iterations.
Therefore,our proposed MaxOT algorithm is more efficient.
[Table]
[Table] [Table]
[Table]
4.4 Performance under defense
We further verify the effectiveness of our ϵ -ISO attack under the existing defense methods.
Toevaluate the adversarial robustness of various attacks, we use the following defense methods: Simple
Random Sampling (SRS) [80], Statistical Outlier Removal (SOR) [93], DUP-Net defense [93]
[Table]
and
IF-Defense [72].
We give the attack success rate of various attacks under each defense method in
Table 3.
Under simple defenses such as SRS and SOR, the attack success rate of Mesh Attack ishigher than our ϵ -ISO attack.
However, under more advanced and effective defenses such as DUP-Netdefence and IF-Defence, the attack success rate of our ϵ -ISO attack is higher than all other attacks.
This is because KNN attack and GeoA 3 attack generate adversarial objects through dense localdeformation, generating outliers that are easily detected by the defense methods.
The Mesh Attackgenerates adversarial objects through large anomalous deformation, resulting in fewer local outliersthat have advantages under simple statistical defenses such as SRS and SOR, but cannot effectivelyattack the DUP-Net defence and IF-Defence with better defense performance.
Our ϵ -ISO attackproduces adversarial samples without local outliers and without anomalous deformations, with betterattack performance, especially under DUP-Net defence and IF-Defence.
4.5 Physical attack
We randomly selecte 100 instances in Table 2, 50 of which are generated by the MaxOT algorithm andthe other 50 are the corresponding instances generated by the EOT algorithm.
The selected meshesare printed by the Stratsys J850 Prime 3D printer and scanned by the EinScan-SE 3D scanner.
Theattack success rate in the physical world is shown in Table 4, where our algorithm almost completelymaintains the effect of the pseudo physical world attack.
None of the previous algorithms couldmaintain the pseudo physical world attack well in the physical world because the dense deformationand self-intersection phenomena would interfere with the scanning and prevent the scanner fromcorrectly scanning the surface of the adversarial samples generated by their algorithms.
In contrast,the surface of the adversarial sample produced by our ϵ -ISO attack is smooth and natural and can bescanned correctly by the scanner, preserving the adversarial effect.
In addition, the improvement onthe attack success rate of our proposed MaxOT algorithm against the EOT algorithm is also reflectedin the physical world.
Our proposed MaxOT algorithm is necessary to improve the robustness ofthe adversarial sample under the transformation of the physical world.
The adversarial meshes, 3Dprinted physical meshes, and scanned point clouds are shown in Fig.
4.6 Ablation studies
To investigate the effect of the penalty parameter λ 1 of the Gaussian curvature consistency regularization term on the attack success rate and imperceptibility metrics, we adjust the value of λ 1 to performquantitative and qualitative experiments.
The plots in Fig.
6 show that when λ 1 is tuned high, theattack success rate decreases rapidly.
When λ 1 is adjusted lower, the values of the evaluation metricsincrease and deteriorate rapidly.
The qualitative visualization results in Appendix C demonstrate theirregularity of the 3D objects as λ 1 is turned down.
Our default value of λ 1 = 1 gives the best resultfor balancing the attack success rate and stealthiness.
To investigate the effectiveness of Bayesian
AdversarialMesh
[Figure]
[Figure]
optimization to find the initial transformations, we test the attack success rate for randomly selectingthe initial transformation in the MaxOT algorithm.
As shown in Appendix C, the improvement ofusing Bayesian optimization over random initialization in the MaxOT algorithm is significant.
Thisconfirms the importance of Bayesian optimization methods in our approach.
More ablation studiescan be found in Appendix C.
Conclusion
In this paper, we propose a novel ϵ -isometric ( ϵ -ISO) attack method to generate natural and robust 3Dadversarial examples in the physical world.
We improve the naturalness of 3D adversarial examplesby constraining it to be ϵ -isometric to the original one.
Through a theoretical analysis, we adoptthe Gaussian curvature as the surrogate metric.
We further propose a maxima over transformation(MaxOT) algorithm armed with Bayesian optimization that actively searches for the most harmfultransformations rather than random ones to make the generated adversarial example more robustin the physical world.
Extensive Experiments on typical point cloud recognition models validatethe effectiveness of our approach in terms of attack success rate and naturalness compared to thestate-of-the-art methods.
A potential negative societal impact of our work in that the maliciousadversaries may adopt our method to generate 3D adversarial objects in the physical world, whichcan cause severe security/safety consequences for real-world applications.
Thus it is imperative todevelop more robust 3D recognition models, which we leave to future work.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] We do not develop more robust
3D models in this work, which we leave to future work.
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] See Sec.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes]
(b) Did you include complete proofs of all theoretical results?
[Yes] See Appendix A.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main exper-
imental results (either in the supplemental material or as a URL)?
[Yes] The code is
included in the supplementary material.
(b) Did you specify all the training details (e.g., data splits, hyperparameters,
how they
were chosen)?
[Yes] See Sec. 4.1.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[No]
(d) Did you include the total amount of compute and the type of resources used (e
g., typeof GPUs, internal cluster, or cloud provider)?
[Yes]
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]