Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: This paper presents a new adversarial training procedure that enhances existing $L_\infty$ AT methods by adding specific noise to the images, with the goal of improving robustness to multiple types of perturbation ($L_1, L_2, L_\infty$).

The proposed method improves upon existing AT methods in two ways: 
1. For the underlying $L_\infty$ procedure, the addition of noise adds considerable robustness to other norms, at the expense of a small reduction in robustness for $L_\infty$. 
2. In comparison of other approaches targeting robustness against union of perturbation, the proposed method reduces the runtime by employing a single attack AT, with a relatively cheap noise augmentation. While competing approaches have to compute multiple attacks, the proposed method only computes one attack and only occasionally updates the noise distribution.

The improvement in performance allows the method to be applied to large datasets (ImageNet), which is prohibitive under competing approaches.
main_review: Overall, the presentation of the results is well done. The figures and tables are easy to read and well-elaborated with useful visual aids, which I found very helpful in some instances. The experimental setup is well described and the choice and discussion of experiments contribute to a cohesive narrative. While there is a lack of theoretical results and the originality of the method is somewhat reduced, this is compensated by a solid experimental evaluation.

The efficacy of the proposed method is clearly demonstrated with the empirical results.There are relevant comparison with SOTA methods that tackle both reducing complexity of AT and/or improving robustness for multiple attacks. In particular, Table 3 makes a compelling argument for the trade-off between the robustness and the training time. It does a good job of highlighting the benefits of SNAP when the training time is restricted to less than 12 hours. 

The flexibility of choosing any base AT procedure and the infrequent updates of the noise distribution made it possible to obtain good robustness results on ImageNet, which is exceedingly costly with other methods.

**Weaknesses:**

In essence, SNAP is a combination of randomized smoothing and adversarial training, i.e. adversarial training with noise-augmented image, where the noise distribution is occasionally updated throughout the training. However, it took me a while to understand this. The clarity of the paper could be improved by describing the similarities (and differences) to a method such as SmoothAdv earlier in the paper. Also, due to the strong parallels between AT+SNAP and SmoothAdv, I expected to see it included in the results of Tables 1 and 2. While there is some discussion in the supplementary material, having this comparison in the mentioned tables could contribute to set SNAP apart from the other randomized smoothing + AT approach.

The spectral subspace analysis, while interesting, was a bit hard to interpret. For example, why the dimensionality of the projection implies that the noise vectors are aligned? Couldn't they still be pointing in different directions even though they lie in a small subspace? Also, in Figure 3a, the dimension seems to be very different for each type of noise. This section brings an interesting discussion about how AT affects the geometry of the perturbations, but it is not easy to interpret and understand the implications of the analysis.


limitations_and_societal_impact: The authors have adequately addressed the limitations and societal impact in the final discussion section.
review_title: null

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: This paper expands the capabilities of widely popular single-attack $\ell_{\infty}$ adversarial training (AT) frameworks to provide robustness to the union of $(\ell_1, \ell_2, \ell_{\infty})$ perturbations while preserving the training efficiency. The proposed method, named SNAP, could reduce the curvature of the decision boundary of networks. SNAP prepends a given deep net with a shaped noise augmentation layer whose distribution is learned along with network parameters using any standard single-attack AT. SNAP enhances adversarial accuracy of ResNet-18 on CIFAR-10 against the union of $(\ell_1, \ell+2, \ell_{\infty})$ perturbations for four single-attack $\ell_{\infty}$ AT frameworks, and establishes a benchmark for ResNet-50 and ResNet-101 on ImageNet.
main_review: 1. $V=[v_1,\dots,v_D]$ denotes a group of basis in $\mathbb{R}^D$. Is V the orthonormal basis? 

2. The authors claim that "the average squared `$\ell_2$ norm of the noise vector $n$ is held constant at $P_noise$ while adapting the noise variances in the individual dimensions to align the noise vectors with the adversarial perturbations on average." Why can such an alignment help to defend the union attack?

3. The proposed method sacrifices the performance of the clean data to improve robustness against special attacks. However, Eq. (1) can be treated as mitigating through randomization. Unfortunately, this mechanism has been broken by [1]. Therefore, the vulnerability of SNAP should be fully considered under various kinds of adversarial methods. 


[1] Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
limitations_and_societal_impact: Yes.
review_title: null

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: This paper produces SNAP -- that augments a noise layer to networks during standard adversarial training. On 9 different adv training methods for l_inf perturbations, the authors find that adding a noise layer helps improve robustness against the union of l1, l2, l_inf perturbations. The key hypothesis that the work is built on is that since AT reduced the curvature of the decision boundary, various attacks belonging to different perturbation types get closely aligned. Using this property, the authors attempt to modify the l_inf model boundary by adding a specifically chosen noise.
main_review: ### After rebuttal: Rating changed to 5

### Merits:
1. The method is a simple addition to any AT method to increases robustness 
2. The training time increases is only ~10% as opposed to past works that use 3x the time.
3. Reasonable accuracy improvements over standard training methods.


### Questions and Negative points
1. *Removal of Noise Layer at test time*: The authors say that using N = 8 restarts ensures that there is no gradient masking because they use EOT. My review heavily depends on the following result -- What happens when one removes the noise layer of SNAP+AT model at test time. According to the hypothesis, SNAP should only help in training, and not inference. In that case, we should not need the noise layer at inference. If the results hold, then it empirically justifies no gradient masking, however, if they don't -- it suggests that gradients are being masked and the evaluation was not sufficient.
2. *Hypothesis Graph*: Figure 3 appears problematic to me. Why should p_j^2 not be divided while graphing the expectation? It can mislead the reader into believing in higher alignment. I would like to view the graph after this change to make any inferences because it seems rather surprising that the adversarial attacks should align so well even in non-AT setting.
3. *Do adversarial vectors really align?*: It has been shown in past work that adversaries in different perturbation regions do not subsume each other: see table 3 in https://arxiv.org/pdf/1905.11213.pdf. The hypothesis that they should align seems surprising.
4. I find it difficult to understand that even if the Hypothesis in Section 3 holds, how does it motivate adding a random noise layer to "wiggle" the l_inf boundary? It seems that the two observations are disjoint. Can you help understand this link better?
5. *Evaluation*: Since Maini et. al. was first published in 2019, the standards of adversarial training evaluation have significantly changed. I request the authors to use attacks from AutoAttack library as standardized by RobustBench for l2, linf. Additionally please use, Fast Adaptive Boundary Attack (FAB) for all of l1,l2,l_inf to perform a thorough evaluation. These attacks are faster to run, and often help identify situations where gradients are masked.
6. *What about MNIST?*: It has been found in past work that hyperparameter tuning was especially tricky in case of the MNIST dataset. Does SNAP also successfully augment noise in case of AT methods on MNIST?

limitations_and_societal_impact: Yes
review_title: null

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: This paper aims to improve the robustness of networks trained using ell_infty norm-based adversarial training to the union of perturbations like ell_1, ell_2, and ell_infty. And the basic idea is to add a shaped augmentation layer that can inject noise during adversarial training.
main_review: Originality: This paper looks like a combination of adversarial training and randomized smoothing to me so I tends not to think it is novel enough.

Quality: The main claim of this paper is that by injecting noise during AT, the model can be robust to a union of perturbations without costing much extra time. However, from the results, the improvement looks mostly come from the ell_1 perturbation. It then makes sense why the authors found that Laplacian noise is found to be the best choice because ell_1 certified robsutness is achieved by randomized smoothing with Laplacian noise.

Clarity: The writing is clear.

Significance: Given the improvement is mostly from ell_1 robustness, the reviewer kind of doubts whether it is meaningful to consider the problem in the paper, i.e., being robust to the union of perturbations efficiently.

---
Update after rebuttal:
The results authors pointed in the paper together with the new results do address my concern on the source of improvement. Thus I would like to increase my score by 1.
limitations_and_societal_impact: Not applicable.
review_title: null

Meta-review: This is indeed a borderline paper. Based on the majority of opinions from reviewers and the suggested accept rate of program committee, a current rejection decision is made. 
Decision: Reject