Year: 2019

Abstract:
Recent studies show that machine learning models are vulnerable to adversarial examples. In 2D image domain, these examples are obtained by adding imperceptible noises to natural images. This paper studies adversarial generation of point clouds by learning to deform those approximating object surfaces of certain categories. As 2D manifolds embedded in the 3D Euclidean space, object surfaces enjoy the general properties of smoothness and fairness. We thus argue that in order to achieve imperceptible surface shape deformations, adversarial point clouds should have the same properties with similar degrees of smoothness/fairness to the benign ones, while being close to the benign ones as well when measured under certain distance metrics of point clouds. To this end, we propose a novel loss function to account for imperceptible, geometry-aware deformations of point clouds, and use the proposed loss in an adversarial objective to attack representative models of point set classifiers. Experiments show that our proposed method achieves stronger attacks than existing methods, without introduction of noticeable outliers and surface irregularities. In this work, we also investigate an opposite direction that learns to deform point clouds of object surfaces in the same geometry-aware, but cooperative manner. Cooperatively generated point clouds are more favored by machine learning models in terms of improved classification confidence or accuracy. We present experiments verifying that our proposed objective succeeds in learning cooperative shape deformations.

heading: 1 INTRODUCTION
text: The existence of adversarial examples shows the vulnerability of machine learning models, and triggers a great amount of research attention paid to either attacking and defense studies for safetycritical issues, or robustness analysis on machine learning models themselves. In existing literature, adversarial examples generally satisfy two key properties: visually imperceptible by humans and capable to mislead machine learning models. For 2D adversarial images (Szegedy et al., 2013; Carlini & Wagner, 2017; Madry et al., 2017), their imperceptible nature indicates less sensitivity by humans to the texture changes of the original images, which can be viewed as noises with small magnitudes anchored on RGB values of pixels.
Different from texture changes in adversarial images, adversarial point clouds as approximate shape representations of object surface demand imperceptible shape deformations. Heuristic manners of existing several works are taken to allow shape deformations in arbitrary directions for adversarial attack effects. Consequently, they produce adversarial examples that contain obvious point outliers (Xiang et al., 2019; Liu et al., 2019a), which can thus be easily defended by simple outlier removal method such as SOR (Zhou et al., 2018).
In this work, we aim to adversarially perturbing benign point clouds with awareness of the underlying geometric properties, which can be generally described as some sort of approximate geometric smoothness, e.g., point-wise normals (Wang et al., 2018), curvatures (Bae, 2006), or geodesics for pairs of points (He et al., 2019), in view of the nature of object surface as 2-manifold embedded in the 3D space. To this end, we propose a novel geometry-aware loss in combination with the attack one. Our proposed geometry-aware loss is composed of three terms: the Chamfer Distance (CD) and Hausdorff Distance (HD) terms prevent geometric/topological changes of global shape via constraining point-wise perturbations, while the term encouraging consistency of local curvatures
between the adversarial and benign point clouds achieves smooth local surface deformations. Extensive experiments on the ModelNet40 dataset (Wu et al., 2015) can verify the imperceptibility and effectiveness of geometry-aware adversaries.
Our success in generating less noticeable point clouds inspires us to think of an opposite direction: is it possible to perturb raw point clouds in a cooperative, geometry-aware manner such that the resulting imperceptible perturbations of point clouds can improve classification performance with existing machine learning models? To the best of our knowledge, very few existing works are pursuing this direction, at least in the domain of 3D shape analysis. Technically, the objective of cooperative generation on point clouds is achieved by learning point perturbations in favor of either improved classification confidence or corrected class predictions. In view of their geometryaware nature, our cooperative point clouds could be practically meaningful in terms of guiding design of certain objects such that they can be better perceived by in-built machine learning systems in physical world. Experiments on the ModelNet40 again verify our motivation on cooperative generation of point clouds.

heading: 2 RELATED WORKS
text: A number of adversarial attack algorithms have been proposed on 3D semantic analysis (Xiao et al., 2019; Liu et al., 2019a; Xiang et al., 2019; Yang et al., 2019; Zheng et al., 2018; Wicker & Kwiatkowska, 2019; Liu et al., 2019b). Beyond (Xiao et al., 2019) manipulating both texture and shape on the mesh level to attack image classifiers and detectors, most of adversarial algorithms on point clouds attack 3D classifiers via points attachment (Xiang et al., 2019; Yang et al., 2019), point detachment (Yang et al., 2019; Zheng et al., 2018; Wicker & Kwiatkowska, 2019) or point-wise perturbation (Xiang et al., 2019; Liu et al., 2019a; Yang et al., 2019; Liu et al., 2019b). Existing methods fail to exploit geometric properties during adversarial example generation, and thus have evident outliers and can be easily defended by SOR defense (Zhou et al., 2018), which encourage our adversaries (see Figure 2 and Table 1).
A number of deep models have been proposed for point-based surface reconstruction such as Point Set Generation (PSG) (Fan et al., 2017), the AtlasNet (Groueix et al., 2018b), and Deep Cascade Generation (DCG) (Wang et al., 2019a) from 2D images. These methods exploiting local mesh structure into point clouds generation have inspired our geometry-aware concept, but they are directly regressing from the latent vector encoded from images rather than point-wise deformations in the proposed method.
Data argumentation on point clouds such as jittering, rotation and random scale firstly introduced in (Qi et al., 2017a) shares similar concept as our cooperative deformations, both of which generate new data to boost classification. However, the key differences between our cooperative learning and data argumentation lie in two folds: 1) optimization based vs. simple physical processing; and 2) deformations on the underlying shape vs. point density changes. Simply put, the proposed cooperative deformation can be a promising learning paradigm, while data argumentation is an effective pre-processing step to mitigate sparse sample distributions.

heading: 3 METHODOLOGY
text: Our problem setting assumes the availability of a collection of point clouds in the input space X , which are approximate shape representations of object surfaces of certain categories. Each point cloud P ∈ X contains an orderless set of n points {pi}ni=1, with the corresponding label y ∈ Y of object category, where any p denotes the coordinates (x, y, z) in the 3D Euclidean space. In this work, we focus on machine learning models of 3D point set classification (Qi et al., 2017a;b; Wang et al., 2019b), which learn a classifier f : X → Y and expect f(P) = y for any input P with the true label y.
P is a discrete approximation of an object surface that satisfies the general properties of smoothness and fairness (Botsch et al., 2010), which concern with the continuity and variation of (partial) derivatives of a parametric surface function; or in a more intuitive way, the properties concern with the curvatures of local surface patches. Our objective in this work is to obtain from P a deformed point cloud P ′ via learning to perturb individual points {pi}ni=1 of P . Since the deformation is
expected to be imperceptible by humans, we argue that P ′ should have the aforementioned surface properties with similar degrees of smoothness/fairness to local surface patches of P , while being globally close to P as well, measured by certain distance metrics of point sets; otherwise, humans would notice either the global, possibly topological changes of part configuration of the object surface, or those of local surface details. We present shortly in section 3.1 our technical solutions to the above objective of geometry-aware point perturbations, and discuss how such solutions can be used to generate either adversarial or cooperative point clouds in sections 3.2 and 3.3 respectively.

heading: 3.1 GEOMETRY-AWARE POINT PERTURBATIONS
text: In 2D image domain, pixel-wise lp-norms are usually used to constrain the noise magnitudes of adversarial examples (Carlini & Wagner, 2017; Madry et al., 2017). Due to the sharply different data nature of point clouds, we consider the following learning criteria to perturb points {pi}ni=1 of P in a geometry-aware manner. These criteria are to constrain the deformation magnitudes of the resulting P ′, while taking into account prevention of point outliers and smooth regularization of local point neighborhoods. Combined use of these criteria leads to point cloud deformations that are less noticeable by humans, as verified in the comparative experiments in section 4.
Chamfer Distance Given two point sets P and P ′, the Chamfer distance computes
CChamfer(P ′,P) = 1
n ∑ p′∈P′ min p∈P ‖p′ − p‖22 + 1 n ∑ p∈P min p′∈P′ ‖p− p′‖22, (1)
which shows that Chamfer distance is symmetric w.r.t. P andP ′. Although the Chamfer distance (1) is not a strict distance function, since the triangle inequality does not hold, it is popularly used in the recent literature of learning based 3D shape generation (Yang et al., 2018; Groueix et al., 2018a; Fan et al., 2017). It measures the distance between the two point sets by averaging over the individual deviation of any p ∈ P from P ′ and that of any p′ ∈ P ′ from P . However, Chamfer distance is less effective in prevention of outlier points in P ′, since a small portion of outliers in P ′ increases the distance (1) negligibly — one can intuitively think of outliers of a point cloud as those away from the object surface, represented by the point cloud, with relatively large distances. This shortcoming of Chamfer distance motivates us to additionally use the Hausdorff distance as introduced below.
Hausdorff Distance In this work, we use a non-symmetric Hausdorff distance between the two point sets P and P ′ that computes
CHausdorff(P ′,P) = max p′∈P′ min p∈P ‖p′ − p‖22, (2)
since only the deformation of P ′ is concerned. As (2) indicates, the Hausdorff distance finds the largest one among the smallest distances of individual p′ ∈ P ′ from P . It is thus sensitive to generation of outliers in the resulting P ′. Distances computed by the functions (1) and (2) rely on those between the individual points {p′i}ni=1 and {pi}ni=1, which does not involve geometries of local surface patches associated with these individual points; consequently, the resulting P ′ could be close to P when measured by (1) and/or (2), but changes of geometric details at certain local patches could be clearly visible, causing failure to achieve the objective of imperceptible deformation.
Consistency of Local Curvatures Our way of achieving geometry-aware imperceptibility is to constrain the point cloud deformation such that local patches of the surface approximated by the resulting P ′ have curvatures whose magnitudes are similar to those of the corresponding patches of the surface approximated by P . Since computations in this work are conducted on the discrete surface approximation of point clouds, we propose discrete notions that approximately characterize curvatures of local surface patches.
More specifically, for any point p′ ∈ P ′, we find its closest point p ∈ P by p = arg minp∈P ‖p′ − p‖2. There exist local point neighborhoods N ′p′ ⊂ P ′ and Np ⊂ P respectively associated with p′ and p, which are obtained in this work by searching k nearest neighbors, suggesting |N ′p′ | = |Np| = k. To capture the local geometry of Np, we rely on the following discrete notion
κ(p;P) = 1 k ∑ q∈Np |〈(q − p)/‖q − p‖2,np〉| , (3)
where np denotes the unit normal vector of the surface at p. The term (3) intuitively measures the averaged angles between the normal vector and the vector defined by pointing p towards each q of its neighboring points. Indeed, since the normal vector np is orthogonal to the tangent plane of the surface at p, each inner product in (3) characterizes how the normals vary directionally in the local neighborhood Np, thus approximately measuring the local, directional curvature, and an average of |Np| inner products in (3) approximately measures the local, mean curvature. Note that unit normal vector np in (3) can be computed from Np via eigen-decomposition of the set N (p) (Hoppe et al., 1992). We compute κ′(p′;P ′) in the same way as (3), with a subtle difference that instead of computing n′p′ from N ′p′ , we directly use np, i.e., the unit normal vector of the point in P that is closest to p′, as a surrogate of n′p′ , since normal vectors of P can be pre-computed and efficiently retrieved during the deformation learning.
Given κ′(p′;P ′,P) and κ(p;P), we use the following criterion to encourage the consistency of local geometries between any p′ ∈ P ′ and its closest point p ∈ P
CCurvature(P ′,P) = 1
n ∑ p′∈P′ ‖κ′(p′;P ′,P)− κ(p;P)‖22 s.t. p = arg minp∈P ‖p ′ − p‖2.
(4)
where we write κ′(p′;P ′,P) since the normal vector involved in its computation is from the corresponding one of P . Note that terms similar to (3) are also used in (Wang et al., 2018; Tang et al., 2019) for single-view surface reconstruction. Our use of the term (3) in (4) is to encourage the consistency of local surface geometries between P ′ and P , rather than to directly minimize (3) as in (Wang et al., 2018; Tang et al., 2019).
The Combined Geometry-aware Objective We use the following combined objective to learn the deformed P ′ by either adversarial or cooperative perturbations of individual points of P
CGeometry(P ′,P) = CChamfer(P ′,P) + α · CHausdorff(P ′,P) + β · CCurvature(P ′,P), (5) where α and β are the weighting parameters.

heading: 3.2 GENERATION OF ADVERSARIAL POINT CLOUDS
text: Assume a point cloud classification model f : X → Y . An adversarial example of a point cloud P is a crafted malicious input P ′ to the model f(·), with imperceptible deformation, such that P ′ is misclassified by the model. Let the true label of P be y ∈ Y , it means that f(P ′) 6= y. Adversarial examples can be generated either by untargeted or targeted attacks. Similar to (Xiang et al., 2019), we focus in this work on the more difficult task of targeted attack for point cloud data, which generates P ′ such that it is classified as a specified class y′ 6= y, i.e., f(P ′) = y′. Among various attack models proposed for 2D image classification (Madry et al., 2017; Carlini & Wagner, 2017), we adopt the state-of-the-art framework of C&W attack (Carlini & Wagner, 2017). Its objective can be generally written as
min x′
CAdv(x ′) + λ · ‖x′ − x‖p, (6)
where x is the benign signal (e.g., an image) and x′ is the adversarial example to be optimized. The misclassification loss CAdv(x′) is regularized by a λ weighted, lp-norm based term that constrains the noise magnitude of adversarial x′. To specify CAdv(x′), we assume a classification model f(·) be implemented as a deep network, and denote as g(·) the function that outputs the network logits, i.e., g(·) includes all layers of the network except the final softmax. Let the targeted label attacking x as y′, C&W attack commonly uses a margin based loss function as CAdv(x
′) = max{maxi 6=y′ gi(x′)− gy′(x′), 0}. In this work, we adopt the C&W attack framework, and propose to replace the term of lp-norm in (6) with our geometry-aware objective (5), in order to generate adversarial point clouds with imperceptible shape deformations, giving
min P′
CAdv(P ′) + λ · CGeo(P ′,P). (7)
The original choice of margin based CAdv(P ′) in C&W attack ceases pursuing more adversarial examples once maxi 6=y′ gi(P ′) − gy′(P ′) ≤ 0, assuming that further optimization would reduce
the imperceptibility of the resulting P ′. Our proposed geometry-aware (5) allows us to take a more aggressive strategy, and we propose to use the following term as our misclassification loss
CAdv(P ′) = − log ( exp(gy′(P ′))/
∑ i exp(gi(P ′))
) . (8)
Our proposed objective (7) with term (8) continues to pursue more adversarial, arguably less defendable, point clouds without introducing noticeable shape deformations, as empirically verified by our experiments in section 4.

heading: 3.3 GENERATION OF COOPERATIVE POINT CLOUDS
text: The adversarial objective (7) motivates us to think of an opposite direction of geometry-aware point cloud deformations that are less noticable by humans. Specifically, is it possible to perturb points of any P in a cooperative, geometry-aware manner such that the resulting P ′ is more favored by machine learning models in terms of improved classification confidence/accuracy? Technically, for a given classifier f(·), this seems to be achieved trivially by optimizing
min P′
CCoop(P ′) + λ · CGeo(P ′,P), (9)
with the cooperative loss term as
CCoop(P ′) = − log ( exp(gy(P ′))/
∑ i exp(gi(P ′))
) , (10)
where we simply use the true label y of P to replace the targeted attack label y′ 6= y in (8). Given knowledge of the true label y of P , the objective (9) seems only learn a deformed P ′ that overfits the classifier f(·), which could be practically less meaningful. In this work, we present the following interesting, but less explored investigation based on (9), which suggests that subtle deformations of shape instances of common object categories (e.g., those in ShapeNet (Chang et al., 2015)) could lead to practical meanings in terms of being better perceived by existing 3D point set classification models (Qi et al., 2017a;b; Wang et al., 2019b).
Assume we are given a set of m point clouds {Pi}mi=1 of different object categories. We take the following procedure on {Pi}mi=1.
1. We divide {Pi}mi=1 evenly into s subsets, with consideration of class balance. 2. We use the first s−1 subsets as training data to train a classifier fs(·), and use the objective
(9) to cooperatively deform point clouds in the sth subset (the validation set) w.r.t. fs(·). 3. We perform s times of step 2 by using each of the s subsets as the validation set, obtaining f i(·), i = 1, . . . , s, and the aggregated collection of deformed {P ′i}mi=1.
The above procedure can optionally be conducted for multiple times to further increase the degree of deformations, with awareness of geometric imperceptibility. We use the obtained {P ′i}mi=1 in a standard training-and-test setting of point set classification. That is, we split {P ′i}mi=1 as training and test data, whose indices are the same as those of the original {Pi}mi=1, and train a new classifier f̂(·) to evaluate f̂(·) on the split test data.
The obtained {P ′i}mi=1 are indeed cooperative when performance of f̂(·) is improved over that of the original classifier f(·). In fact, we have further conducted cross-model experiments by learning {P ′i}mi=1 via the above procedure with PointNet (Qi et al., 2017a), and train and evaluate f̂(·) of other representative point set classifiers (e.g., PointNet++ (Qi et al., 2017b) and DGCNN (Wang et al., 2019b)). We have also conducted experiments by converting the obtained {P ′i}mi=1 to their mesh representations (Edelsbrunner et al., 1983), and uniformly re-sample points from the meshes to form {P̃ ′i}mi=1; classification performance again improves by training and testing on {P̃ ′i}mi=1, confirming that we are indeed cooperatively deforming the underlying object surfaces.
While our above investigations remain in the digital space, cooperative deformations of point clouds could be physically achieved either by changes of shape design for common rigid objects, or by designing special devices that actively control the reflections of laser pulses from LiDAR (Cao et al.,
2019). Physical-world adversarial attacks (Athalye et al., 2017; Eykholt et al., 2018) are actively pursued recently in the 2D image domain. Deformations of geometry-aware imperceptibility make sense here since we are not to change the geometric and/or topological surface structures of the objects; otherwise the deformations would have influence on human perception or functional attributes of the objects.

heading: 4 EXPERIMENTS
text: Dataset We use point clouds of object instances from ModelNet40 (Wu et al., 2015) to evaluate our proposed algorithms. The dataset consists of 12, 311 CAD models belonging to 40 semantic categories. For each CAD model, 1, 024 points are uniformly sampled from its surface as the working point clouds, which are re-scaled into a unit ball following (Qi et al., 2017b).
Models and Protocols We evaluate the adversarial and cooperative generation of point clouds based on three representative classifiers, namely PointNet (Qi et al., 2017a), PointNet++ (Qi et al., 2017b), and DGCNN (Wang et al., 2019b). More details of the networks please refer to A.6. In our adversarial setting, we follow the official data split for 3D classification (Qi et al., 2017a;b) and use 9, 843 samples of point clouds for training classifiers that are to be attacked, and for testing, we follow (Xiang et al., 2019) and randomly select 25 samples from each testing set of the top-10 object classes (ordered by sample sizes of these classes, giving rise to airplane, bed, bookshelf, bottle, chair, monitor, sofa, table, toilet, vase); all adversarial attack algorithms are evaluated with a white-box, targeted attack protocol. In our cooperative setting, we set the subset number as s = 5.
Evaluation Metrics We respectively use the attack success rate (i.e., misclassification rate) and classification accuracy to evaluate the algorithmic effectiveness in our adversarial and cooperative settings. For both metrics, the higher, the better.
Implementation Details We set k = 16 to define local point neighborhoods for computation of normals and approximate curvatures in (3). We fix α = 0.1 and β = 1.0 for our proposed geometryaware loss (5), and the trade-off parameter λ in (7) and (9) is optimized via 10-step binary search. We use Adam optimizer (Kingma & Ba, 2014) to train networks, and set its learning rate as 0.01.

heading: 4.1 EVALUATION ON GENERATION OF ADVERSARIAL POINT CLOUDS
text: In this section, we evaluate the efficacy of our proposed geometry-aware objective (7) for generation of adversarial point clouds. We dub our method as GeoAwareAdv and compare with the method (Xiang et al., 2019), which is among the only few existing works (Liu et al., 2019a;b) addressing adversarial point clouds and takes the same white-box, targeted attack as our method does. Objective of (Xiang et al., 2019) strictly follows the C&W attack (6) (Carlini & Wagner, 2017), with a margin-based misclassification loss regularized by l2-norm constraining magnitude of point perturbations. We also compare with a degenerate version of our method, dubbed GeoDegenerateAdv, which replaces the geometry-aware term (5) with a similar l2-norm as in (Xiang et al., 2019). These experiments are conducted using PointNet (Qi et al., 2017a) as the model of classifier.
Ablation Studies To investigate how different terms in our proposed geometry-aware loss (5), we conduct ablation studies by removing each of them from (5), and use the remaining ones into the adversarial objective (7) for point cloud generation. Figure 1) shows that these terms all together contribute to smooth and outlier-free results. Without using CD, the shape deformations go to unexpected twists. The use of HD and curvature terms largely removes generation of outliers, with the later one further improving the surface smoothness.
Comparative Results We report comparative results of our GeoAwareAdv, GeoDegenerateAdv, and the method (Xiang et al., 2019) in Table 1 and Figure 2. Table 1 compares the attack success rates under two defense methods of SOR (Zhou et al., 2018) and Random Romoval (RR). SOR is the current state-of-the-art method to defend attacking of point clouds, which works by dropping certain points from a point cloud based on statistical analysis. RR simply drops points from a point cloud randomly. Based on our stronger adversarial term (8), both of our methods achieve better success rates than those of (Xiang et al., 2019) under different dropping settings of the defenses SOR and RR. More significantly, the stronger attacking effects of our GeoAwareAdv are achieved without introduction of noticeable outliers and surface irregularities, as shown in Figure 2; in contrast, the
use of simple l2 norm in (Xiang et al., 2019) produces its adversarial results with clear outliers. Advantages of our GeoAwareAdv are essentially due to the use of HD and curvature terms in our geometry-aware loss (5).

heading: 4.2 EVALUATION ON GENERATING GEOMETRY-AWARE COOPERATIVE POINT CLOUDS
text: Comparative Evaluation Cooperative point clouds are generated based on the PointNet via optimizing objective function (9), and a variant can be re-sampled point clouds from its mesh surface. A number of recent classifiers are trained and evaluated on vanilla and cooperative point clouds respectively, whose results are reported in Figure 3 and Table 2. It can be observed from Table 2, both cooperative point clouds significantly outperform the vanilla ones by a large margin, i.e., increase at least 6% on classification accuracy. Moreover, performance gap between two types of cooperative point clouds can be explained by both approximation errors of mesh reconstruction and the changes on point distribution, which encourage us to consider adversarial shape deformations on the mesh level.
Cross-model experiments We conduct one more cross-model experiment to test transferable characteristics of cooperative point clouds across classifiers, i.e., generating cooperative examples on PointNet, while train and evaluate other classifiers. Results in Table 2 show consistent improvement on classification accuracy achieved for different neural classifiers, which reveal that cooperative shape deformations on point clouds can capture discriminative geometric patterns favor for semantic object classification, which are independent on neural models.

heading: 5 CONCLUSIONS
text: In this paper, we propose a compact geometry-aware loss to constrain point-wise imperceptible perturbations to achieve similar geometric smoothness and fairness of local patches and their global topological configuration in deformed point clouds as the original ones. Experiment results reveal the rationale of three components in the proposed geometry-aware loss for generating adversaries without evident outliers and shape irregularities, which can thus achieve adversarial effects even confronting the SOR defense. Moreover, cooperative generation on point clouds also demonstrates its positive effects on improving classification with aware of geometric properties.

heading: A APPENDIX
text: A.1 VISUALIZATION OF MORE GEOMETRY-AWARE ADVERSARIAL EXAMPLES.
A.2 VISUALIZATION OF MORE GEOMETRY-AWARE COOPERATIVE EXAMPLES.
A.3 ABLATION STUDY OF DIFFERENT α AND β .
A.4 QUANTITATIVE RESULTS FOR FIGURE 1.
It’s hard to define a metric evaluating geometrical smoothness of a point cloud directly due to its irregularity and lacking of geometric topological connection. Hence, we introduce an approximation metric mainly focuses on measuring the strength of isolated noisy points and the roughness of local patches, which can be defined as:
S(P) = max p∈P ∑ q∈Np D(q,TNp) (11)
where estimated tangent plane of the k nearest neighbors Np of p is denoted by TNp, and D denotes the distance function between the points and the tangent plane, where we use l2 norm in our settings. Intuitively, we are calculating the residual of first-order Taylor expansion of the points we estimate the first-order approximation of a point p, i.e., the tangent plane TNp, and then measures how far its neighbor pointsNp are to this plane. Note that we set the |Np| = k to be small to reduce approximation error, where in the following settings k = 16.
We evaluate S(P) on the experiments of Figure 1, and the quantitative results are shown in the following Table 3. As can be seen in the table, with all the losses, the geometry-aware adversarial point clouds achieve the lowest S(P), where the point clouds are smoothest. And with many outliers, the adversarial point clouds without Hausdorff Distance get the highest S(P). The quantitative results of the point clouds without Chamfer Distance are a little higher than the point clouds without Consistency of Local Curvatures, which is consistent with our perception the point clouds without Consistency of Local Curvatures are also very smooth expect for some high frequency outliers near their surfaces.
A.5 USER STUDY ON AMAZON MECHANICAL TURK (AMT).
We conducted a user study on Amazon Mechanical Turk (AMT) in order to verify the imperceptible quality of our adversarial examples. We upload the snapshots of three kinds of point clouds including the benign point clouds, the adversarial point clouds generated via the method of Xiang et al. (2019) and our geometry-aware adversarial ones. All the adversarial point clouds can successfully mislead PointNet. Participants were asked to compare which one of the adversarial point clouds are more geometrically similar to the original one. The order of two kinds of adversarial point clouds was randomized and all the images appeared in the middle of the screen on each trial. Each participant could conduct at most 30 trials and each adversarial images can be shown to 50 different participant at most. In total, we conduct 1500 trials among 128 participants. And our geometry-aware adversarial point cloud are considered to be closer to the original ones in 82.06% of the trials, which indicates that our geometry-aware point clouds are more imperceptible. We think this experiment can further support our opinion that our geometry-aware adversarial examples are more imperceptible to humans. And we will attach this experiments to the appendix of our revised paper.
A.6 INTRODUCTION OF OUR POINT CLOUD CLASSIFIER.
The PointNet (Qi et al., 2017a) and Pointnet++ (Qi et al., 2017b) are the first attempts to explore deep point cloud classification, with the permutation invariance of points in multi-layer perceptrons (MLPs) and a symmetric function for aggregating features. Both methods can only implicitly model global semantic patterns from 3D geometry. Another groups of algorithms design graph based convolution operation on irregular distributed structure of points such as DGCNN (Wang et al., 2019b). In DGCNN, an edge convolution operation is proposed on a dynamic graph to discover local geometric manifolds.
We mainly focus on PointNet Qi et al. (2017a) as our baseline due to its simple structure and good performance. The classier f can be formulated as f(P ) = γ(maxpi∈P {h(pi)}), where γ and h are two learnable parameters of the neural network.
A.7 INTRODUCTION OF DEFENSIVE ALGORITHM
We adopt two kinds of defense method including random removal method (RR for short) and statistic outlier removal method (SOR for short)Zhou et al. (2018) as the defensive method.
In RR, we randomly select a subset of points from the point cloud and drop it. And in SOR, we calculate the average distance dp of a point p to its k-nearest neighbors, which can be denoted by
dp = 1
k ∑ q∈Np ‖p− q‖2 (12)
And then we calculate the mean d̄ and standard deviation σd over all the distance dp of points in the point cloud p ∈ P . We remove all the points that fall outside d̄ ± a × σd, where a is set to be 1.1 in Zhou et al. (2018). According to our experiments, around 100 points will be dropped out of 1024 points when a is set to be 1.1.
However, in our settings, we drop a fix number points with m-largest dp for fairer comparison between different methods. We drop k = 1, 2, 4, 8, 16, 32, 64, 128, 256 respectively in Table 1.

Review summaries:
This paper proposes a novel loss function to account for imperceptible, geometry-aware deformations of point clouds. The loss is used in two cases: generating adversarial point clouds to attack representative models of point set classifiers, and generating cooperative point clouds to improve classification confidence or accuracy. The combined geometry-aware objective is well-introduced, which mainly contains Chamfer distance term, Hausdorff distance term and local curvatures consistency term. The authors apply the geometry-aware objective to generate adversarial point clouds by adopting the framework of C&W attack. For generating cooperative point clouds, the authors introduced a training procedure to reduce the overfitting of the deformed point clouds. Most of the experiments are well-conducted, and demonstrate the effectiveness of the proposed loss function. 

This paper describes a new targeted adversarial attack against 3D point cloud object classifiers that is robust to several countermeasures. The attack finds a point of the target class that is close to the original point cloud in terms of a more complicated metric that combines the Hausdorff distance, the Chamfer distance, and a curvature distance measure. The proposed attack is 100% successful against several different state of the art classifiers on a dataset of 1024-point clouds sampled from 25 instances of CAD models of each of 10 common objects without any countermeasures.  When the Random Removal countermeasure is used, the attack is still successful almost 50% of the time even when 256 points are removed as compared to two other attacks that are only ~17% successful. When the SOR countermeasure is used, the attack is 60% successful when 64 points are removed as compared to <1% for the comparison attacks. The attack can also be used in reverse for data augmentation in training and can cut error rates almost in half.