Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper proposes a white-box adversarial attack on 3D point cloud recognition models such that the corresponding attacks are robust, natural, and reproducible in the physical world. For naturalness, they propose a novel framework based on maintaining $\epsilon$-isometry between the original and adversarial point cloud model. To boost robustness and ensure reproducibility, they develop a new framework combining maxima over transformation with bayesian optimization. Experimental evaluation against existing attacks on state-of-the-art models shows that the proposed approach achieves higher attack success rate while remaining imperceptible. The authors further show reproducibility in the real world.
strengths_and_weaknesses: Strengths:
+ The paper considers a novel setting where the author's goal is to generate robust and natural adversarial examples for NNs working on  3-D point clouds that can also be realized in the physical world. 

+ The technical contributions are novel and tailored for creating physical, natural, and robust adversarial examples in the 3-D point cloud setting. 

+ Strong experimental results showing the value of the contributions including reproducibility in the real world.

Weaknesses:
- The technical details in Section 3.1 are not rigorous enough which makes the corresponding text hard to follow. Further, while the high-level idea in 3.3.1 is clear, the text is too dense. I suggest the authors improve their writing at these places for making their technical contributions more accessible. 

-  While input-specific white-box attacks show the non-robustness of NNs, they are usually not realistic. The authors do not mention cases where an attacker can gain white-box access to the point cloud model and how it knows what inputs will be processed by the model in advance and whether it can create physical examples fast enough before the input is processed by the model. For example, in the case of a self-driving car, the model will continuously process new inputs. Is your attack feasible in such settings? Therefore I am not sure if the proposed method reveals any vulnerability that can be exploited by a real-world adversary.

questions: * In Section 3.1, $\mathcal{X}$ is undefined. I assume it's the set of all meshes or point clouds? Line 101, how is $\Delta$ measured? Is the sampling function $\mathcal{S}$ fixed for a given mesh $\mathcal{M}$? Can you provide a mathematical definition of the length function "s" used in Definition 1? How do you get the equality on line 145? 

* Line 132, "Therefore, we introduce the definition of the Gaussian curvature [11] of the surface", is it your definition or taken from literature? Please be precise with the wording.

* In Definition 1, what is the range of values of n? For arbitrarily n, it looks like the difference in the lengths of curves on $S$ and $\hat{S}$ can be quite large?

* In Definition 2, can you explain how the parameters $(u,v)$ relate to the meshes?

*  Is Theorem 1 your contribution or adapted from prior work?

* Can you compare the runtime of the different baselines and your method?

* "The Mesh Attack generates adversarial objects through large anomalous deformation, resulting in fewer local outliers that have advantages under simple statistical defenses such as SRS and SOR, but cannot effectively attack the DUP-Net defense and IF-Defence with better defense performance. " While there is a significant improvement over the mesh attack under the IF-defence, improvement under the DUP-Net defense is marginal (<0.5%), so it would be misleading to claim that mesh attack is ineffective and your attack is effective in this case.

* Can your MaxOT + Bayesian optimization provide any formal guarantees on the robustness of your examples (e.g., as in https://arxiv.org/pdf/2007.12133.pdf)? Further, does your robustness enable you to be more effective against certified defenses based on randomized smoothing (e.g., https://arxiv.org/abs/2103.03046)?

* Why did you not analyze the effects of the hyperparameters $\lambda_2$ and $\lambda_3$?

* How do $\mathcal{D}_c$ and $\mathcal{D}_g$ relate to $\epsilon$ in your $\epsilon$-ISO attacks?  

* I observed you only consider rotations when considering robustness in experiments. Any particular reason for that? Why not consider the composition of translation, rotation, other transformations, etc.?
limitations: The authors have adequately addressed the limitations and potential negative impacts of their work.
review_title: null

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposes a novel $\epsilon$-ISO attack algorithm to generate natural and robust 3D adversarial samples. It is motivated by the fact that the distance metric on the Euclidean space does not consider geometric features and cannot accurately measure the similarity between 3D objects. The proposed method constrains the adversarial sample to be $\epsilon$-isometric to the original sample to ensure the geometric similarity and adopts the Gaussian curvature as the regularization. The theoretical analysis is provided to uncover the connection between isometry and Gaussian curvature. The authors further propose MaxOT to improve the robustness of 3D adversarial samples in the physical world. MaxOT is reasonable by actively searching for the most harmful transformations during optimization. The experiments validate the naturalness and effectiveness of the proposed 3D attack.
strengths_and_weaknesses: Strengths:
- First, the paper is easy to read. The algorithm and experiments are well organized and explained. The experiments are thorough to prove the effectiveness.
- Overall, I find the proposed approach novel. To the best of my knowledge, previous 3D adversarial attacks only considered the distance metric on Euclidean space, i.e. Hausdorff distance, Chamfer distance, etc., whereas this is the first work to consider the geometric properties on Non-euclidean space of 3D objects. This paper introduces $\epsilon$-isometric mapping to ensure the geometric similarity between the adversarial sample and the original one. It further connects $\epsilon$-isometry to the Gaussian curvature, which can be computed and optimized in generating 3D adversarial samples. Furthermore, the MaxOT algorithm is reasonable to improve the robustness of 3D adversarial samples.
- The theoretical analysis makes this paper more solid and convincing.
- The paper has conducted thorough evaluation comparisons between the proposed attack and other attacks. The performance improvement is obvious. The 3D adversarial objects look more natural according to the visualization. The effectiveness measured by the attack success rate is also validated.

Weaknesses:
- Table 2 and Figure 4 demonstrate that the proposed MaxOT outperforms EOT in terms of attack success rate and efficiency in the physical world. Is it due to the 3D objects generated by MaxOT are more perceptible?
- To further verify the effectiveness of the proposed method, the authors are encouraged to conduct experiments of black box attacks.
questions: ### Other comments:
I was inspired by the authors’ discussion of metrics on Non-euclidean space. As analyzed by the authors, the distance metric on Euclidean space is not a perfect representation of the similarity between 3D objects because it does not take geometric features into account. While reading, I kept thinking if the Fisher information metric on statistical manifold mapped from topological manifold [1] would be an interesting consideration.

[1] F. Li et al, "Toward a unified framework for point set registration," ICRA 2021


Overall, I think the strengths of this paper overweigh its weaknesses. The contributions of this paper could be significant for the venue. I’d like to see my concerns addressed by the authors and consider increasing the rating.

### Question:
- In fig4, the title of the y-axis should be the attack success rate.
- Line 16 of the appendix: The authors should further clarify the relationship between Gaussian curvature and the first fundamental form in the proof in the appendix to ensure that it can be clearly understood.
- Line 142: "bending degree" should be "the bending degree".
- Line 148: "bending degree" should be "the bending degree".
- Line 149: "Definition" should be "Definitions".
- Line 262: "higher" should be "a higher".
- table 3: "DUP-Net" should be "DUP-Net defense".
- Line 281: "selecte" should be "select".
- In fig6(a), the title of the y-axis should be the attack success rate.
limitations: /na
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper presents $\epsilon$-ISO  attack method to generate natural and robust 3D adversarial examples in the physical world. The authors present a new metric or definition on the naturalness of the 3D adversarial example. They also provide theoretical proof to demonstrate that Gaussian curvature could be used as a proxy for $\epsilon$-isometric. MaxOT is proposed as a method combined with Bayesian to optimize the 3D adversarial examples.   
strengths_and_weaknesses: Strengths:
1. The paper contains both formal and empirical results to demonstrate the effectiveness of the proposed method.
2. The organization of the paper is good, and the content is rich.
3. The authors leverage extensive evaluation to show that the proposed attack is effective and realistic in the real world. The experiments with 3D printed mesh are convincing. 

Weakness:
1. Although the authors successfully demonstrate the physical attack on 3D point cloud recognition task, it is still questionable how these attacks will have real-world impacts. Real-world application of point clouds mostly include object detection and segmentation. Delving into the recognition task and showing naturalness seem to be off the correct direction in this area.
2. I like the organization of this paper, but the content is too crowded and sometimes it is hard to follow every detail.
questions: See the first point in the weakness section.
limitations: The authors should discuss the potential societal impact of the proposed attack.
review_title: null

Meta-review: metareview: The authors propose a new method for constructing adversarial distortions of 3d objects. The method centers around the a new distance metric between 3D objects which the authors argue is better suited for natural looking perturbations relative to the euclidian metric. Reviewers overall felt the paper was strong, the results convincing and the method new and interesting. The primary concerns were raised by reviewer WZCq, who questioned whether or not the method can be expected to have practical impact. The AC agrees with these concerns, particularly given that most documented examples of attacks on real systems do not involve any notion of subtleness, nor do attackers seem motivated to constrain adversarial inputs to small perturbations of clean inputs [1]. However, given the strong technical contribution of the work and the overall strong reviews, the AC recommends accepting the work but encourages authors to consider revising the text noting that epsilon, (or small, or subtle) perturbations is in no means a strict constraint on the attacker action space in real world settings.

1. Gilmer et. al. "Motivating the rules of the game for adversarial example research", 2018.
confidence: Less certain
Decision: Accept