Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 8: Accept
review_content: This paper describes a new targeted adversarial attack against 3D point cloud object classifiers that is robust to several countermeasures. The attack finds a point of the target class that is close to the original point cloud in terms of a more complicated metric that combines the Hausdorff distance, the Chamfer distance, and a curvature distance measure. The proposed attack is 100% successful against several different state of the art classifiers on a dataset of 1024-point clouds sampled from 25 instances of CAD models of each of 10 common objects without any countermeasures.  When the Random Removal countermeasure is used, the attack is still successful almost 50% of the time even when 256 points are removed as compared to two other attacks that are only ~17% successful. When the SOR countermeasure is used, the attack is 60% successful when 64 points are removed as compared to <1% for the comparison attacks. The attack can also be used in reverse for data augmentation in training and can cut error rates almost in half.

Overall, this seems like a large improvement over current approaches. The paper does a good job of explaining its approach and motivation and does an excellent job of situating its contributions within the existing literature. The experiments show that the improvements in success are large over competing attacks, that they are robust to current countermeasures. They also show that when used "cooperatively" they can improve performance substantially.

One issue that could be easily addressed is that Table 2 is mentioned on page 5, but doesn't appear until the end of page 7. It is also mentioned before Table 1. So I would recommend changing it to Table 1 and introducing it before it is mentioned.

It would also be nice to have some basic descriptions of the point cloud classifiers and the SOR countermeasure. Space for this could be regained from some of the figures, which are informative, but over-represented.
review_title: Official Blind Review #1

final_score: rating: 3: Weak Reject
review_content: This paper looks at the task of (adversarially or cooperatively) perturbing a point cloud in context of a classification task. It follows the prevailing paradigm of changing the original input in the direction of a high positive/negative gradient while staying ‘close’ to the original input, and the key contribution here is to define a different notion of ‘closeness’.

While previous work (Xiang et. al., at least for ‘point addition attack’) used a combination of chamfer and Hausdorff distance as the notion of closeness, this paper additionally includes change in curvature (which is an intuitive term to include) when computing the adversarial/cooperative updates to the point cloud. The obtained results do visually look less perturbed compared to the previous approach, and the obtained adversarial shapes are more robust against two defenses studied.

Concerns/Questions:

1) If a point cloud P’ is only a (small) perturbation of a point cloud P, then the Chamfer distance / Hausdorff distance is essentially the  L_2 / L_infinity norm of their difference. While the use of curvature terms is different, I feel the claims of importance of using ‘distance metric of point clouds’ is not very well justified (as I’d expect essentially same results if these two terms were replaced by L2 and L_infinity norms instead). I think the use of these terms was more necessary in the work of Xiang et. al., as they allowed point addition, so the ‘norm of difference of point clouds’ is not well defined.

2) This paper uses a different (more aggressive) adversarial term (in Eqn. 8) compared to Xiang et. al., so it is not surprising that the results in Table 1 indicate more robustness to defenses.

3) In addition to the above comments about specifics, I feel this work’s contribution over prior work is not significant. While Xiang et. al. did use L2 norm for their perturbation case, they did investigate the Chamfer/Hausdorff distances for another scenario, and therefore the main contribution here is an additional loss term.

4) This is perhaps a hard concern to address, but simply showing some qualitative results to highlight that the changes are ‘imperceptible’ is not sufficient. Ideally, one should report a curve on ‘change perceptibility’ vs ‘attack success rate’ (though this would require some notion of perceptibility that was not used in optimization). Alternately, one could compare methods via A/B testing on mechanical turk, asking ‘Which are these two shapes are closer to the original one?’, and ablate for a certain level of confidence on the wrong class, which approach led to less changes. The current results simply show some examples, but provide no empirical way of judging which approach actually leads to more imperceptible changes.

Overall, though the results are perceptually encouraging, I have slight concerns the empirical results reported. However, the primary issue is that the contribution regarding the additional term, while intuitive, is not a significant one in its own right.

While the rating here only allows me to give a ‘3’ as a weak reject, I am perhaps a bit more towards borderline (though leaning towards reject) than that indicates.

review_title: Official Blind Review #2

final_score: rating: 3: Weak Reject
review_content: This paper proposes a novel loss function to account for imperceptible, geometry-aware deformations of point clouds. The loss is used in two cases: generating adversarial point clouds to attack representative models of point set classifiers, and generating cooperative point clouds to improve classification confidence or accuracy. The combined geometry-aware objective is well-introduced, which mainly contains Chamfer distance term, Hausdorff distance term and local curvatures consistency term. The authors apply the geometry-aware objective to generate adversarial point clouds by adopting the framework of C&W attack. For generating cooperative point clouds, the authors introduced a training procedure to reduce the overfitting of the deformed point clouds. Most of the experiments are well-conducted, and demonstrate the effectiveness of the proposed loss function. 

Overall, the paper is a well-written and could be an interesting contribution. However, I would like it better if it does not contain the part about the cooperative point clouds, which are not very motivated and the experiment settings are not very convincing. The algorithm in page 5 inherently used testing labels (suppose some testing data is in the training split of one of those cross-validation folds, training for the P_i' for the rest of the training data). I don't know why experiment results generated by this method would be of any meaning. I would be OK to accept this paper if the cooperative point cloud part is dropped.

- In page 5, the authors introduce a procedure to train the model. The first step would require take the “consideration of class balance”. It would be better to give some details on how the class balance is considered.

- In ablation study, the authors give some visualization results in Figure 1. However, it would be more interesting if the authors could give some quantitative results. 

- It would be interesting to show some ablation study on how to choose the  α, β and λ .

review_title: Official Blind Review #3

Meta-review: This paper offers an improved attack on 3-D point clouds. Unfortunately the clarity of the contribution is unclear and on balance insufficient for acceptance.
Decision: Reject