Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: 
This paper studies the noise injection as defense methods against adversarial perturbations. It presents several experiments on the relationship between clean and robust accuracy. Conclusions of this study are (1-1) several defense methods have the same underlying mechanism (noise injection) and behave similarly against adversarial perturbations, (1-2) all of the defense methods can be attacked by the same black-box attack, and (1-3) the reason of the correct label recovery by noise injections might be the input instability on adversarial inputs.

This paper should be rejected because (2-1) some conclusions are not well-supported by the experiments, and (2-2) the paper fails to demonstrate the novelties and their importance of some conclusions.

Major comments:
(3-1) Experiment 1 aims to highlight the similarity between the ``perturbations defenses''. However, the "similarity" is not defined, and the arguments are mostly subjective. For example, SVD and Gaussian noise appear to have different effectiveness (Figure 1). If the similarity means the observation that large distortions decrease the accuracy, it will not be surprising because all methods should achieve the chance rate accuracy when the distortions are extreme.
(3-2) What is the point of Experiment 2? The phenomenon that strong defense methods decrease clean accuracy has been observed in many defense papers. The noise injection-based defense methods will not be exceptions. Remarking the existence of the trade-off will not be a strong contribution. The focus of the discussion should be on how to take a good balance or improve both of them.
(3-3) To my best knowledge, experimentally showing that adversarial inputs have larger gradient norms (Section 4) is novel, and it is potentially interesting. I wonder how large the gradient norms will be for randomly perturbed images.

===== UPDATE=====
Thank you for the response. After reading the response and other reviews, I mostly agree with Reviewer 2 and keep my score.

Thank you for the clarification on (3-1) in the initial review. I think it made the contribution of the paper clearer. However, as I and Reviewer 2 commented in their initial review, I did not find the observation significant.
I also thank you for the additional experiments concerning (3-3). It addressed a concern that the larger gradient norms are not specific to adv. examples and appear everywhere around the natural images. However, concerns c) raised by Reviewer 2 seems critical. It is unclear whether the instability explains the observed phenomena and whether Experiment 3.3 is enough to confirm the existence of the instability. I think the paper is better to address the concerns before publication.
review_title: Official Blind Review #1

final_score: rating: 3: Weak Reject
review_content: The paper studies the robustness of adversarial attacks to transformations of their output. Specifically, for standard methods for crafting adversarial examples, the authors evaluate whether the crafted examples remain adversarial under (stochastic or deterministic) transformations such as Gaussian noise and SVD compression. The authors argue that different transformations have a similar impact on the perturbed inputs. Finally, they argue that the reason why these transformations can sometimes recover the correct label is due to the loss being more unstable at these points (from a first- and second-order pespective).

From a conceptual point of view, I did not find the paper particularly impactful. In particular, I can identify three claimed contributions:

a) The L2 distortion introduced by these transformation might have more impact than the specific transformation used (figure 1).
Firstly, I would argue that this effect is most prominent for small L2 distortions (distortions larger than 5/40 do exhibit noticeable differences). Secondly, I am not sure why one would expect these transformations to have fundamentally different impact. The adversarial attacks considered manipulate the input using first-order methods in input space. Since the transformations are model- and data-agnostic it seems expected that the primary mechanism behind their effect is the pixel-wise distortion of the image.

b) Adversarial perturbations that are robust to one type of transformation tend to also be robust to other transformations (table 2).
This is probably the most interesting observation of the paper, indicating that attackers can bypass several of these transformation defenses by only aiming to be robust to a subset of them. At the same time, I am not sure what the impact of this observation is given that we already know how to bypass most of these defenses anyway.

c) The instability of adversarial perturbations can be explained by a larger gradient norm and Hessian spectrum (figure 2, 3).
First of all, I do not understand how this is considered a potential explanation of empirical behavior. If gradient norm is indicative of instability, then _natural_ images would be unstable (since the gradient wrt the wrong class is large). Furthermore, instability does not explain why adding noise leads to the _correct_ class as opposed to a random class. Perhaps most importantly, from what I understand (also skimming the code), Figure 2 plots the gradient of the _cross-entropy loss_ with respect to each class. However, the norm of the gradient is directly affected by the softmax probability of each class. Hence, for natural images, the probability of the correct class is high leading to a small norm while for the adversarial class it is low leading to large norm. Based on this reasoning, this plot does convey information about the classifier's stability but rather about the softmax probabilites assigned to each class.

In summary, after reading the paper, I am not sure how our understanding of transformation robustness has changed or what insight we have gained that will help us design future attacks and defenses (especially given prior work on how most of these defenses can be bypassed). I thus recommend rejection.

Comments to authors:
-- Many of the attack details are missing (what is the epsilon allowed for PGD? what does the "c" parameter correspond to for CW attacks?) which prevented me from fully comprehending the experimental results. 
-- LBFGS is not a particular attack, it is a general optimization algorithm, https://en.wikipedia.org/wiki/Limited-memory_BFGS 
-- The fact that adversarial points are close to correctly classified points does not mean that a small amount of random noise will change the classifier prediction. In high dimension, the distance to a hyperplane can be epsilon but moving in a random direction requires movement of epsilon * sqrt(number of dimensions) .

==============

UPDATE: I appreciate the authors' response. As stated in my original review, I do recognize the performance of a non-adaptive adversary as a contribution. However, I still don't think that the impact of this finding is significant enough for publication at ICLR. I hence keep my original score.

Response to specific points:
-- Unfortunately, the authors did not address my concerns about the gradient of the loss. The experiments and reasoning of the paper should be sufficient without citing other work in the author response. Moreover, it is unclear if the papers referenced in the response are showing fundamental properties of adversarial examples or are specific to the attacks considered.  For instance, there has been work challenging these papers (https://arxiv.org/abs/1907.12138).

-- As mentioned in my original review, measuring the norm of the gradient of the softmax loss will inevitably take into account the predicted probability of each class. Hence in order to truly measure sensitivity it would be important to work with the logits instead of the softmax probabilities.


review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: The work is concerned with a very interesting question: "robustness of the adversarial attacks" and discusses the following: First, what are different ways of destabilizing a given adversarial attack (different perturbations) and how much each of them are effective. Secondly, what makes adversarial images particularly non-robust compared to natural images and is there empirical evidence for it? Thirdly, is it possible for an attacker to use this knowledge and use a rather universal model of perturbations to make its adversarial examples robust against such (deterministic or stochastic) perturbations?

The paper is quite well-justified and addresses a very interesting question by unifying the existing results of the literature in one place. First of all, comparing the accuracy-robustness trade-off of different methods are useful.  The idea of modeling perturbations as a general distorted communication channel to have a unification of these methods is very useful. The perturbation analyses along with the results in Figures 2, 3 are novel and can be useful for further theoretical investigation of the source of instability of adversarial images. To the best of my knowledge, the transferability of adaptive attacks against noisy channel defense methods has not been discussed in the literature. The paper is well-organized and well written (except for Section 5 both in sense of the writing and where it appears in the paper).

A few questions and suggestions:

* The discussion of results in Figure2 should be more extensive and more clear. 

* Why do you think the L2 of the gradient for adv images has such high variance for both orig and adv class? Is there a relationship between the magnitude and how effective is a perturbation-based defense?

* It should be made clear in the text that methods like randomized smoothing are designed for the purpose of 'certified' accuracy and being robust against attack sizes more than the certification threshold was not part of the method's contributions.

* The use of the term "compression" in Section 3 is not entirely correct as it is not "technically" correct to say that compression is what is necessarily happening in a general case of a deterministic C(x).

* Fig 1 should be larger.
review_title: Official Blind Review #3

Meta-review: This paper presents an analysis on different methods of noise injection in adversarial examples, using gaussian noise for example. There are important issues raised by reviewers 1 & 2 about some conclusions not being well supported by the experiments and the utility/importance of some conclusions. After a discussion among reviewers, as of now all 3 reviewers stand by the decision that substantial improvements, and analysis can be made in the paper. Thus, Im recommending a Rejection.
Decision: Reject