G EOMETRY - AWARE G ENERATION OF A DVERSARIAL AND C OOPERATIVE P OINT C LOUDS
Anonymous authors
Paper under double-blind review
A BSTRACT
Recent studies show that machine learning models are vulnerable to adversarial examples.
In 2D image domain, these examples are obtained by adding imperceptible noises to natural images.
This paper studies adversarial generation of point clouds by learning to deform those approximating object surfaces of certain categories.
As 2D manifolds embedded in the 3D Euclidean space, object surfaces enjoy the general properties of smoothness and fairness.
We thus argue that in order to achieve imperceptible surface shape deformations, adversarial point clouds should have the same properties with similar degrees of smoothness/fairness to the benign ones, while being close to the benign ones as well when measured under certain distance metrics of point clouds.
To this end, we propose a novel loss function to account for imperceptible, geometry-aware deformations of point clouds, and use the proposed loss in an adversarial objective to attack representative models of point set classiﬁers.
Experiments show that our proposed method achieves stronger attacks than existing methods, without introduction of noticeable outliers and surface irregularities.
In this work, we also investigate an opposite direction that learns to deform point clouds of object surfaces in the same geometry-aware, but cooperative manner.
Cooperatively generated point clouds are more favored by machine learning models in terms of improved classiﬁcation conﬁdence or accuracy.
We present experiments verifying that our proposed objective succeeds in learning cooperative shape deformations.
I NTRODUCTION
The existence of adversarial examples shows the vulnerability of machine learning models, and triggers a great amount of research attention paid to either attacking and defense studies for safetycritical issues, or robustness analysis on machine learning models themselves.
In existing literature, adversarial examples generally satisfy two key properties: visually imperceptible by humans and capable to mislead machine learning models.
For 2D adversarial images (Szegedy et al., 2013;
Carlini & Wagner, 2017; Madry et al., 2017), their imperceptible nature indicates less sensitivity by humans to the texture changes of the original images, which can be viewed as noises with small magnitudes anchored on RGB values of pixels.
Different from texture changes in adversarial images, adversarial point clouds as approximate shape representations of object surface demand imperceptible shape deformations.
Heuristic manners of existing several works are taken to allow shape deformations in arbitrary directions for adversarial attack effects.
Consequently, they produce adversarial examples that contain obvious point outliers (Xiang et al., 2019; Liu et al., 2019a), which can thus be easily defended by simple outlier removal method such as SOR (Zhou et al., 2018).
In this work, we aim to adversarially perturbing benign point clouds with awareness of the underlying geometric properties, which can be generally described as some sort of approximate geometric smoothness, e.g., point-wise normals (Wang et al., 2018), curvatures (Bae, 2006), or geodesics for pairs of points (He et al., 2019), in view of the nature of object surface as 2-manifold embedded in the 3D space.
To this end, we propose a novel geometry-aware loss in combination with the attack one.
Our proposed geometry-aware loss is composed of three terms: the Chamfer Distance (CD) and Hausdorff Distance (HD) terms prevent geometric/topological changes of global shape via constraining point-wise perturbations, while the term encouraging consistency of local curvatures between the adversarial and benign point clouds achieves smooth local surface deformations.
Extensive experiments on the ModelNet40 dataset (Wu et al., 2015) can verify the imperceptibility and effectiveness of geometry-aware adversaries.
Our success in generating less noticeable point clouds inspires us to think of an opposite direction: is it possible to perturb raw point clouds in a cooperative, geometry-aware manner such that the resulting imperceptible perturbations of point clouds can improve classiﬁcation performance with existing machine learning models?
To the best of our knowledge, very few existing works are pursuing this direction, at least in the domain of 3D shape analysis.
Technically, the objective of cooperative generation on point clouds is achieved by learning point perturbations in favor of either improved classiﬁcation conﬁdence or corrected class predictions.
In view of their geometryaware nature, our cooperative point clouds could be practically meaningful in terms of guiding design of certain objects such that they can be better perceived by in-built machine learning systems in physical world.
Experiments on the ModelNet40 again verify our motivation on cooperative generation of point clouds.
R ELATED W ORKS
A number of adversarial attack algorithms have been proposed on 3D semantic analysis (Xiao et al., 2019;
Liu et al., 2019a; Xiang et al., 2019; Yang et al., 2019; Zheng et al., 2018; Wicker & Kwiatkowska, 2019; Liu et al., 2019b).
Beyond (Xiao et al., 2019) manipulating both texture and shape on the mesh level to attack image classiﬁers and detectors, most of adversarial algorithms on point clouds attack 3D classiﬁers via points attachment (Xiang et al., 2019; Yang et al., 2019), point detachment (Yang et al., 2019; Zheng et al., 2018; Wicker & Kwiatkowska, 2019) or point-wise perturbation (Xiang et al., 2019; Liu et al., 2019a; Yang et al., 2019; Liu et al., 2019b).
Existing methods fail to exploit geometric properties during adversarial example generation, and thus have evident outliers and can be easily defended by SOR defense
(Zhou et al., 2018), which encourage our adversaries (see Figure 2 and Table 1).
A number of deep models have been proposed for point-based surface reconstruction such as Point
Set Generation (PSG) (Fan et al., 2017), the AtlasNet (Groueix et al., 2018b), and Deep Cascade
Generation (DCG) (Wang et al., 2019a) from 2D images.
These methods exploiting local mesh structure into point clouds generation have inspired our geometry-aware concept, but they are directly regressing from the latent vector encoded from images rather than point-wise deformations in the proposed method.
Data argumentation on point clouds such as jittering, rotation and random scale ﬁrstly introduced in (Qi et al., 2017a)
shares similar concept as our cooperative deformations, both of which generate new data to boost classiﬁcation.
However, the key differences between our cooperative learning and data argumentation lie in two folds: 1) optimization based vs. simple physical processing; and 2) deformations on the underlying shape vs. point density changes.
Simply put, the proposed cooperative deformation can be a promising learning paradigm, while data argumentation is an effective pre-processing step to mitigate sparse sample distributions.
M ETHODOLOGY
Our problem setting assumes the availability of a collection of point clouds in the input space X , which are approximate shape representations of object surfaces of certain categories.
Each point cloud P ∈ X contains an orderless set of n points { p i } ni =1 , with the corresponding label y ∈ Y of object category, where any p denotes the coordinates ( x, y, z ) in the 3D Euclidean space.
In this work, we focus on machine learning models of 3D point set classiﬁcation (Qi et al., 2017a;b; Wang et al., 2019b), which learn a classiﬁer f : X → Y and expect f ( P ) =
y for any input P with the true label y .
P is a discrete approximation of an object surface that satisﬁes the general properties of smoothness and fairness (Botsch et al., 2010), which concern with the continuity and variation of (partial) derivatives of a parametric surface function; or in a more intuitive way, the properties concern with the curvatures of local surface patches.
Our objective in this work is to obtain from P a deformed point cloud P (cid:48) via learning to perturb individual points { p i } n i =1 of P .
Since the deformation is expected to be imperceptible by humans, we argue that P (cid:48) should have the aforementioned surface properties with similar degrees of smoothness/fairness to local surface patches of P , while being globally close to P as well, measured by certain distance metrics of point sets; otherwise, humans would notice either the global, possibly topological changes of part conﬁguration of the object surface, or those of local surface details.
We present shortly in section 3.1 our technical solutions to the above objective of geometry-aware point perturbations , and discuss how such solutions can be used to generate either adversarial or cooperative point clouds in sections 3.2 and 3.3 respectively.
3. G EOMETRY - AWARE P OINT P ERTURBATIONS
In 2D image domain, pixel-wise l p -norms are usually used to constrain the noise magnitudes of adversarial examples (Carlini & Wagner, 2017; Madry et al., 2017).
Due to the sharply different data nature of point clouds, we consider the following learning criteria to perturb points { p i } ni = of P in a geometry-aware manner.
These criteria are to constrain the deformation magnitudes of the resulting P (cid:48) , while taking into account prevention of point outliers and smooth regularization of local point neighborhoods.
Combined use of these criteria leads to point cloud deformations that are less noticeable by humans, as veriﬁed in the comparative experiments in section 4.
Chamfer Distance Given two point sets P and P (cid:48) , the Chamfer distance computes
[Equation]
[Equation]
which shows that Chamfer distance is symmetric w.r.t.
P and P (cid:48) .
Although the Chamfer distance (1) is not a strict distance function, since the triangle inequality does not hold, it is popularly used in the recent literature of learning based 3D shape generation (Yang et al., 2018;
Groueix et al., 2018a; Fan et al., 2017).
It measures the distance between the two point sets by averaging over the individual deviation of any p ∈ P from P (cid:48) and that of any p (cid:48)
∈ P (cid:48) from P .
However, Chamfer distance is less effective in prevention of outlier points in P (cid:48) , since a small portion of outliers in P (cid:48) increases the distance (1) negligibly — one can intuitively think of outliers of a point cloud as those away from the object surface, represented by the point cloud, with relatively large distances.
This shortcoming of Chamfer distance motivates us to additionally use the Hausdorff distance as introduced below.
Hausdorff Distance
In this work, we use a non-symmetric Hausdorff distance between the two point sets P and P (cid:48) that computes
[Equation]
since only the deformation of P (cid:48) is concerned.
As (2) indicates, the Hausdorff distance ﬁnds the largest one among the smallest distances of individual p (cid:48) ∈ P (cid:48) from P .
It is thus sensitive to generation of outliers in the resulting P (cid:48) .
Distances computed by the functions (1) and (2) rely on those between the individual points { p (cid:48)
i } ni = and { p i } ni =1 , which does not involve geometries of local surface patches associated with these individual points; consequently, the resulting P (cid:48) could be close to P when measured by (1) and/or (2), but changes of geometric details at certain local patches could be clearly visible, causing failure to achieve the objective of imperceptible deformation.
Consistency of Local Curvatures Our way of achieving geometry-aware imperceptibility is to constrain the point cloud deformation such that local patches of the surface approximated by the resulting P (cid:48) have curvatures whose magnitudes are similar to those of the corresponding patches of the surface approximated by P .
Since computations in this work are conducted on the discrete surface approximation of point clouds, we propose discrete notions that approximately characterize curvatures of local surface patches.
More speciﬁcally, for any point p (cid:
) ∈ P (cid:48) , we ﬁnd its closest point p ∈ P by p = arg min p ∈P (cid:107) p (cid:48) − p (cid:107) 2 .
There exist local point neighborhoods N (cid:48) p (cid:48)
⊂ P (cid:48) and N p ⊂ P respectively associated with p (cid:48) and p , which are obtained in this work by searching k nearest neighbors, suggesting |N (cid:48) p (cid:48)
|
= |N p |
= k .
To capture the local geometry of N p , we rely on the following discrete notion
[Equation]
where n p denotes the unit normal vector of the surface at p .
The term (3) intuitively measures the averaged angles between the normal vector and the vector deﬁned by pointing p towards each q of its neighboring points.
Indeed, since the normal vector n p is orthogonal to the tangent plane of the surface at p , each inner product in (3) characterizes how the normals vary directionally in the local neighborhood N p , thus approximately measuring the local, directional curvature, and an average of |N p | inner products in (3) approximately measures the local, mean curvature.
Note that unit normal vector n p in (3) can be computed from N p via eigen-decomposition of the set N ( p ) (Hoppe et al., 1992).
We compute κ (cid:48)
( p (cid:48) ; P (cid:48) ) in the same way as (3), with a subtle difference that instead of computing n (cid:48) p (cid:48) from N (cid:48) p (cid:48) , we directly use n p , i.e., the unit normal vector of the point in
P that is closest to p (cid:48) , as a surrogate of n (cid:48) p (cid:48) , since normal vectors of P can be pre-computed and efﬁciently retrieved during the deformation learning.
Given κ (cid:48) ( p (cid:48) ; P (cid:48) , P ) and κ ( p ; P ) , we use the following criterion to encourage the consistency of local geometries between any p (cid:48) ∈ P (cid:48) and its closest point p ∈ P
[Equation]
(4) where we write κ (cid:48)
( p (cid:48) ; P (cid:48) , P ) since the normal vector involved in its computation is from the corresponding one of P .
Note that terms similar to (3) are also used in (Wang et al., 2018; Tang et al., 2019) for single-view surface reconstruction.
Our use of the term (3) in (4) is to encourage the consistency of local surface geometries between P (cid:48) and P , rather than to directly minimize (3) as in (Wang et al., 2018;
Tang et al., 2019).
The Combined Geometry-aware Objective We use the following combined objective to learn the deformed P (cid:48) by either adversarial or cooperative perturbations of individual points of P
[Equation]
where α and β are the weighting parameters.
3. G ENERATION OF A DVERSARIAL P OINT C LOUDS
Assume a point cloud classiﬁcation model f : X → Y .
An adversarial example of a point cloud
P is a crafted malicious input P (cid:48) to the model f ( · ) , with imperceptible deformation, such that
P (cid:48) is misclassiﬁed by the model.
Let the true label of P be y ∈ Y , it means that f ( P (cid:48) )
(cid:54) =
y .
Adversarial examples can be generated either by untargeted or targeted attacks.
Similar to (Xiang et al., 2019), we focus in this work on the more difﬁcult task of targeted attack for point cloud data, which generates P (cid:48)
such that it is classiﬁed as a speciﬁed class y (cid:48)
(cid:54) =
y , i.e., f ( P (cid:48) )
=
y (cid:48) .
Among various attack models proposed for 2D image classiﬁcation (Madry et al., 2017; Carlini &
Wagner, 2017), we adopt the state-of-the-art framework of C&W attack (Carlini & Wagner, 2017).
Its objective can be generally written as
[Equation]
where x is the benign signal (e.g., an image) and x (cid:48) is the adversarial example to be optimized.
The misclassiﬁcation loss C Adv ( x (cid:48) ) is regularized by a λ weighted, l p -norm based term that constrains the noise magnitude of adversarial x (cid:48) .
To specify C Adv ( x (cid:48) ) , we assume a classiﬁcation model f ( · ) be implemented as a deep network, and denote as g ( · ) the function that outputs the network logits, i.e., g ( · ) includes all layers of the network except the ﬁnal softmax.
Let the targeted label attacking x as y (cid:48) , C&W attack commonly uses a margin based loss function as
C Adv ( x (cid:48) )
=
max { max i (cid:54) =
y (cid:48)
g i ( x (cid:48) )
− g y (cid:48)
( x (cid:48) ) , 0 } .
In this work, we adopt the C&W attack framework, and propose to replace the term of l p -norm in (6) with our geometry-aware objective (5), in order to generate adversarial point clouds with imperceptible shape deformations, giving
[Equation]
The original choice of margin based C Adv ( P (cid:48) ) in C&W attack ceases pursuing more adversarial examples once max i (cid:54) =
y (cid:48)
g i ( P (cid:48) )
− g y (cid:48)
( P (cid:48) ) ≤ 0 , assuming that further optimization would reduce the imperceptibility of the resulting P (cid:48) .
Our proposed geometry-aware (5) allows us to take a more aggressive strategy, and we propose to use the following term as our misclassiﬁcation loss
[Equation]
Our proposed objective (7) with term (8) continues to pursue more adversarial, arguably less defendable, point clouds without introducing noticeable shape deformations, as empirically veriﬁed by our experiments in section 4.
3. G ENERATION OF C OOPERATIVE P OINT C LOUDS
The adversarial objective (7) motivates us to think of an opposite direction of geometry-aware point cloud deformations that are less noticable by humans.
Speciﬁcally, is it possible to perturb points of any P in a cooperative, geometry-aware manner such that the resulting P (cid:48) is more favored by machine learning models in terms of improved classiﬁcation conﬁdence/accuracy?
Technically, for a given classiﬁer f ( · ) , this seems to be achieved trivially by optimizing
[Equation]
with the cooperative loss term as
[Equation]
where we simply use the true label y of P to replace the targeted attack label y (cid:48) (cid:54)
=
y in (8).
Given knowledge of the true label y of P , the objective (9) seems only learn a deformed P (cid:48) that overﬁts the classiﬁer f ( · ) , which could be practically less meaningful.
In this work, we present the following interesting, but less explored investigation based on (9), which suggests that subtle deformations of shape instances of common object categories (e.g., those in ShapeNet (Chang et al., 2015)) could lead to practical meanings in terms of being better perceived by existing 3D point set classiﬁcation models (Qi et al., 2017a;b; Wang et al., 2019b).
Assume we are given a set of m point clouds {P i } mi =1 of different object categories.
We take the following procedure on {P i } mi =1 .
We divide {P i } mi =1 evenly into s subsets, with consideration of class balance.
We use the ﬁrst s − 1 subsets as training data to train a classiﬁer f s ( · ) , and use the objective (9) to cooperatively deform point clouds in the s th subset (the validation set) w.r.t.
f s ( · ) .
We perform s times of step 2 by using each of the s subsets as the validation set, obtaining f i ( · ) , i = 1 , . . .
, s , and the aggregated collection of deformed {P (cid:48)
i } mi =1 .
The above procedure can optionally be conducted for multiple times to further increase the degree of deformations, with awareness of geometric imperceptibility.
We use the obtained {P (cid:48) i } mi =1 in a standard training-and-test setting of point set classiﬁcation.
That is, we split {P (cid:48)
i } mi =1 as training and test data, whose indices are the same as those of the original {P i } mi =1 , and train a new classiﬁer ˆ f ( · ) to evaluate ˆ f ( · ) on the split test data.
The obtained {P (cid:48) i } mi =1 are indeed cooperative when performance of ˆ f ( · ) is improved over that of the original classiﬁer f ( · ) .
In fact, we have further conducted cross-model experiments by learning {P (cid:48) i } mi =1 via the above procedure with PointNet (Qi et al., 2017a), and train and evaluate ˆ f ( · ) of other representative point set classiﬁers (e.g., PointNet++ (Qi et al., 2017b) and DGCNN (Wang et al., 2019b)).
We have also conducted experiments by converting the obtained {P (cid:48)
i } mi =1 to their mesh representations (Edelsbrunner et al., 1983), and uniformly re-sample points from the meshes to form { (cid:102) P (cid:48)
i } mi =1 ; classiﬁcation performance again improves by training and testing on { (cid:102) P (cid:48)
i } mi =1 , conﬁrming that we are indeed cooperatively deforming the underlying object surfaces.
While our above investigations remain in the digital space, cooperative deformations of point clouds could be physically achieved either by changes of shape design for common rigid objects, or by designing special devices that actively control the reﬂections of laser pulses from LiDAR (Cao et al.,
Physical-world adversarial attacks (Athalye et al., 2017; Eykholt et al., 2018) are actively pursued recently in the 2D image domain.
Deformations of geometry-aware imperceptibility make sense here since we are not to change the geometric and/or topological surface structures of the objects; otherwise the deformations would have inﬂuence on human perception or functional attributes of the objects.
E XPERIMENTS
Dataset We use point clouds of object instances from ModelNet40 (Wu et al., 2015) to evaluate our proposed algorithms.
The dataset consists of 12 , 311 CAD models belonging to 40 semantic categories.
For each CAD model, 1 , 024 points are uniformly sampled from its surface as the working point clouds, which are re-scaled into a unit ball following (Qi et al., 2017b).
Models and Protocols We evaluate the adversarial and cooperative generation of point clouds based on three representative classiﬁers, namely PointNet (Qi et al., 2017a), PointNet++ (Qi et al., 2017b), and DGCNN (Wang et al., 2019b).
More details of the networks please refer to A.6. In our adversarial setting, we follow the ofﬁcial data split for 3D classiﬁcation (Qi et al., 2017a;b) and use 9 , 843 samples of point clouds for training classiﬁers that are to be attacked, and for testing, we follow (Xiang et al., 2019) and randomly select 25 samples from each testing set of the top-10 object classes (ordered by sample sizes of these classes, giving rise to airplane, bed, bookshelf, bottle, chair, monitor, sofa, table, toilet, vase ); all adversarial attack algorithms are evaluated with a white-box, targeted attack protocol.
In our cooperative setting, we set the subset number as s = 5 .
Evaluation Metrics We respectively use the attack success rate (i.e., misclassiﬁcation rate) and classification accuracy to evaluate the algorithmic effectiveness in our adversarial and cooperative settings.
For both metrics, the higher, the better.
Implementation Details We set k = 16 to deﬁne local point neighborhoods for computation of normals and approximate curvatures in (3).
We ﬁx α = 0 .
1 and β = 1 .
for our proposed geometryaware loss (5), and the trade-off parameter λ in (7) and (9) is optimized via 10 -step binary search.
We use Adam optimizer (Kingma & Ba, 2014) to train networks, and set its learning rate as 0 . 01 .
4. E VALUATION ON G ENERATION OF A DVERSARIAL P OINT C LOUDS
In this section, we evaluate the efﬁcacy of our proposed geometry-aware objective (7) for generation of adversarial point clouds.
We dub our method as GeoAwareAdv and compare with the method (Xiang et al., 2019), which is among the only few existing works (Liu et al., 2019a;b) addressing adversarial point clouds and takes the same white-box, targeted attack as our method does.
Objective of (Xiang et al., 2019) strictly follows the C&W attack (6) (Carlini & Wagner, 2017), with a margin-based misclassiﬁcation loss regularized by l 2 -norm constraining magnitude of point perturbations.
We also compare with a degenerate version of our method, dubbed GeoDegenerateAdv, which replaces the geometry-aware term (5) with a similar l 2 -norm as in (Xiang et al., 2019).
These experiments are conducted using PointNet (Qi et al., 2017a) as the model of classiﬁer.
Ablation Studies To investigate how different terms in our proposed geometry-aware loss (5), we conduct ablation studies by removing each of them from (5), and use the remaining ones into the adversarial objective (7) for point cloud generation.
Figure 1) shows that these terms all together contribute to smooth and outlier-free results.
Without using CD, the shape deformations go to unexpected twists.
The use of HD and curvature terms largely removes generation of outliers, with the later one further improving the surface smoothness.
Comparative Results We report comparative results of our GeoAwareAdv, GeoDegenerateAdv, and the method (Xiang et al., 2019) in Table 1 and Figure 2.
Table 1 compares the attack success rates under two defense methods of SOR (Zhou et al., 2018) and Random Romoval (RR).
SOR is the current state-of-the-art method to defend attacking of point clouds, which works by dropping certain points from a point cloud based on statistical analysis.
RR simply drops points from a point cloud randomly.
Based on our stronger adversarial term (8), both of our methods achieve better success rates than those of (Xiang et al., 2019) under different dropping settings of the defenses SOR and
RR.
More signiﬁcantly, the stronger attacking effects of our GeoAwareAdv are achieved without introduction of noticeable outliers and surface irregularities, as shown in Figure 2; in contrast, the use of simple l 2 norm in (Xiang et al., 2019) produces its adversarial results with clear outliers.
Advantages of our GeoAwareAdv are essentially due to the use of HD and curvature terms in our geometry-aware loss (5).
[Table]
4. E VALUATION ON G ENERATING G EOMETRY - AWARE C OOPERATIVE P OINT C LOUDS
The state-of-the-art Geo-CNN (Lan et al., 2019) achieves the classiﬁcation accuracy of 93 . [Figure]
[Table]
Comparative Evaluation Cooperative point clouds are generated based on the PointNet via optimizing objective function (9), and a variant can be re-sampled point clouds from its mesh surface.
A number of recent classiﬁers are trained and evaluated on vanilla and cooperative point clouds respectively, whose results are reported in Figure 3 and Table 2.
It can be observed from Table 2, both cooperative point clouds signiﬁcantly outperform the vanilla ones by a large margin, i.e., increase at least 6% on classiﬁcation accuracy.
Moreover, performance gap between two types of cooperative point clouds can be explained by both approximation errors of mesh reconstruction and the changes on point distribution, which encourage us to consider adversarial shape deformations on the mesh level.
Cross-model experiments We conduct one more cross-model experiment to test transferable characteristics of cooperative point clouds across classiﬁers, i.e., generating cooperative examples on
PointNet, while train and evaluate other classiﬁers.
Results in Table 2 show consistent improvement on classiﬁcation accuracy achieved for different neural classiﬁers, which reveal that cooperative shape deformations on point clouds can capture discriminative geometric patterns favor for semantic object classiﬁcation, which are independent on neural models.
C ONCLUSIONS
In this paper, we propose a compact geometry-aware loss to constrain point-wise imperceptible perturbations to achieve similar geometric smoothness and fairness of local patches and their global topological conﬁguration in deformed point clouds as the original ones.
Experiment results reveal the rationale of three components in the proposed geometry-aware loss for generating adversaries without evident outliers and shape irregularities, which can thus achieve adversarial effects even confronting the SOR defense.
Moreover, cooperative generation on point clouds also demonstrates its positive effects on improving classiﬁcation with aware of geometric properties.
A. V ISUALIZATION OF M ORE G EOMETRY - AWARE C OOPERATIVE E XAMPLES .
[Figure]
A. Q UANTITATIVE RESULTS FOR F IGURE 1.
It’s hard to deﬁne a metric evaluating geometrical smoothness of a point cloud directly due to its irregularity and lacking of geometric topological connection.
Hence, we introduce an approximation metric mainly focuses on measuring the strength of isolated noisy points and the roughness of local patches, which can be deﬁned as:
[Equation]
where estimated tangent plane of the k nearest neighbors N p of p is denoted by T N p , and D denotes the distance function between the points and the tangent plane, where we use l 2 norm in our settings.
Intuitively, we are calculating the residual of ﬁrst-order Taylor expansion of the points we estimate the ﬁrst-order approximation of a point p , i.e., the tangent plane T N p , and then measures how far its neighbor points N p are to this plane.
Note that we set the |N p | =
k to be small to reduce approximation error, where in the following settings k = 16 .
We evaluate S ( P ) on the experiments of Figure 1, and the quantitative results are shown in the following Table 3.
As can be seen in the table, with all the losses, the geometry-aware adversarial point clouds achieve the lowest S ( P ) , where the point clouds are smoothest.
And with many outliers, the adversarial point clouds without Hausdorff Distance get the highest S ( P ) .
The quantitative results of the point clouds without Chamfer Distance are a little higher than the point clouds without
Consistency of Local Curvatures, which is consistent with our perception the point clouds without
Consistency of Local Curvatures are also very smooth expect for some high frequency outliers near their surfaces.
[Table]
A. U SER S TUDY ON A MAZON M ECHANICAL T URK (AMT).
We conducted a user study on Amazon Mechanical Turk (AMT) in order to verify the imperceptible quality of our adversarial examples.
We upload the snapshots of three kinds of point clouds including the benign point clouds, the adversarial point clouds generated via the method of Xiang et al.
and our geometry-aware adversarial ones.
All the adversarial point clouds can successfully mislead PointNet.
Participants were asked to compare which one of the adversarial point clouds are more geometrically similar to the original one.
The order of two kinds of adversarial point clouds was randomized and all the images appeared in the middle of the screen on each trial.
Each participant could conduct at most 30 trials and each adversarial images can be shown todifferent participant at most.
In total, we conduct 1500 trials among 128 participants.
And our geometry-aware adversarial point cloud are considered to be closer to the original ones in 82 .
06% of the trials, which indicates that our geometry-aware point clouds are more imperceptible.
We think this experiment can further support our opinion that our geometry-aware adversarial examples are more imperceptible to humans.
And we will attach this experiments to the appendix of our revised paper.
A. I NTRODUCTION OF O UR P OINT C LOUD C LASSIFIER .
The PointNet (Qi et al., 2017a) and Pointnet++ (Qi et al., 2017b) are the ﬁrst attempts to explore deep point cloud classiﬁcation, with the permutation invariance of points in multi-layer perceptrons (MLPs) and a symmetric function for aggregating features.
Both methods can only implicitly model global semantic patterns from 3D geometry.
Another groups of algorithms design graph based convolution operation on irregular distributed structure of points such as DGCNN (Wang et al., 2019b).
In DGCNN, an edge convolution operation is proposed on a dynamic graph to discover local geometric manifolds.
We mainly focus on PointNet Qi et al.
(2017a)
as our baseline due to its simple structure and good performance.
The classier f can be formulated as f ( P ) =
γ (max p i ∈ P { h ( p i ) } ) , where γ and h are two learnable parameters of the neural network.
A. I NTRODUCTION OF D EFENSIVE ALGORITHM
We adopt two kinds of defense method including random removal method (RR for short) and statistic outlier removal method (SOR for short)Zhou et al.
as the defensive method.
In RR, we randomly select a subset of points from the point cloud and drop it.
And in SOR, we calculate the average distance d p of a point p to its k -nearest neighbors, which can be denoted by
[Equation]
And then we calculate the mean ¯ d and standard deviation σ d over all the distance d p of points in the point cloud p ∈ P .
We remove all the points that fall outside ¯ d ± a × σ d , where a is set to be 1 .in
Zhou et al.
According to our experiments, around 100 points will be dropped out ofpoints when a is set to be 1 . 1 .
However, in our settings, we drop a ﬁx number points with m -largest d p for fairer comparison between different methods.
We drop k = 1 , 2 , 4 , 8 , 16 , 32 , 64 , 128 , 256 respectively in Table 1.