Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: The authors suggest the use of Random Noise Defense (RND) to protect deep learning models from black-box query-based attacks. The authors motivate their proposal via both theoretical and experimental validation -- where RND is a lightweight defense that augments random noise to input queries. The paper further proposes RND-GF that is trained to be robust (smoothened) to random noise -- in order to further improve the adversarial robustness.
main_review: ### Merits
1. The paper systematically studies decision and score based adversarial attacks, their theoretical implications, and empirical performances
2. RND-GF helps combine adversarial and noise robustness
3. This is one of the first works that studies the relation of random noise to black-box query complexity theoretically. 

### Qualms
1. **Adaptive nature of attacks:** In adaptive attacks, EOT is the only attack considered. In my opinion, the knowledge of v and mu (only the magnitudes and not the exact noise) is one of the minimum knowledge that should be provided to any adaptive attacker. Moreover, all the theorems are heavily dependent on the ratio of v/mu. Can you elaborate in greater detail, the motivation behind this choice -- and how do the results fair when the knowledge is available to the attacker.

2. **Scope of work:** The scope of the work, though important, is limited. It does not even consider all the types of black-box query-based attacks -- such as transfer adversaries; and is not applicable in the case of non-query attacks.

3. **Failure Rates:** From the empirical failure rates, it can be seen that at the optimal selection of mu by the attacker, the failure rates can be very high. Given this information (in an empirical setting) -- would you suggest that RND is still beneficial to model robustness? (Given that adding random noise has a small tradeoff on clean accuracy as well)
limitations_and_societal_impact: Yes
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper proposes a simple but theoretically supported defense method against query-based black-box attack, called as Random Noise Defense. The core idea is to add Gaussian noise to a given input image at each query so it can be considered as quite simple. However, this paper shows theoretical analyses of RND against zero-order optimization, adaptive and search-based attacks. The overall finding through each theoretical analysis is that the number of queries for successful attacks by an attacker changes with the ratio of nu/mu. To overcome trade-off between clean accuracy and larger nu, the authors propose Gaussian augmentation fine-tuning for the target model. From the extensive experiments using CIFAR-10 and ImageNet against various black-box attacks, the authors demonstrated that, not only the results of theoretical analysis are confirmed by experiments but RND achieves the superior performance. Also, they showed RND can be easily combined with other defense methods and it achieves the state-of-the-art defense performance when it is combined with the adversarially-trained model.
main_review: Originality

The proposed idea, adding Gaussian noise to an input image, has been used for adversarial defense, such as randomized smoothing. 
However, This paper has the following differences from it. Firstly, the proposed method is focused on adversarial defense against query-based black-box attack. Second, from a different perspective, this paper theoretically and experimentally shows the superiority of the proposed method. Therefore, this paper has its own contribution.

Quality

I didn't find any flaws in theoretical analysis, but I'm not sure because I am not a theoretical expert.
The authors demonstrated the strength and theoretical findings of the proposed method from the extensive and solid experiments well.
Also, they stated limitations and future works of their work.

Clarity

This paper is well written and easy to read.

Significance

The results of this paper is important because this paper proposes a simple defense method as well as a theoretical analysis of it.

limitations_and_societal_impact: Limitations

1. In my opinion, Assumption 1 is somewhat acceptable for a neural net-based classifier. However, I wonder if Assumption 2 is also generally accepted because it stands for a higher level of smooth functions.

2. Input transformation-based defense methods (e.g. R&P, JPEG and bit-red) are should be considered as a baseline because [1] reported that these methods relatively achieve the superior robustness than other types of defense in case of query-based black-box attacks.

3. I think that there is a methodical connection or similarity between the proposed method and randomized smoothing-based methods [13, 38, 39]. However, this paper seems to lack reference to that content. Therefore, it would be a more solid paper if the authors address differences in theoretical and methodological perspectives from existing similar methods.


[1] Benchmarking Adversarial Robustness on Image Classification, Y. Dong et. al., CVPR 2020


Societal Impact

There is no potential negative social impact in this paper.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes to use random Gaussian noise as defense for black-box attacks. Also, a Gaussian augmentation Fine-tuning is also proposed to enable adding larger noise.

main_review: Overall, this paper is well motivated. The analysis is straight and well-understood.

In table 2, can you explain why RND-GF is even better than GF in terms of clean acc?
limitations_and_societal_impact: Please see my main review
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This work proposes addition of Gaussian noise as a defense against black box attacks and analyses the convergence of query attacks with this defense enabled. The work is quite sound. 

main_review: I have few questions wrt experiments


- What’s the motivation to choose a different \nu for RND, RND-GF and RND-AT in Table 2?
- RND by itself seemed quite poor against NES and Square attack while RND-GF provides quite a boost. Any insights on why this is the case?
- There is a significant drop in clean accuracy with RND-AT, any suggestions on what hyperparameters could help tune this better?
- What would you prescribe as a method to choose noise hyperparameters?
- On a more general note, does the noise need to be Gaussian, or is there an intuition on the suitability of other quantisation noises?
- Tables with small font size were a bit challenging to parse
limitations_and_societal_impact: Adequately addressed
review_title: null

Meta-review: This paper shows that adding a small amount of random noise to the output of a neural network can prevent current black-box attacks. The theory is convincing, the experiments cover a wide range of attacks, and the reviewers are satisfied by the author response. Even though there are still some questions about the generality of the proposed defense and if it could be evaded by stronger attacks, the proposal is interesting and will motivate future attack research to improve on randomized defenses.
Decision: Accept (Poster)