Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: In this paper, the authors propose a novel meta-learning framework that explicitly learns to generate noise to improve model robustness (against multiple types of attacks). The results indicate that the proposed approach improves on the state-of-the-art.

Overall, the paper is well written. However some details are missing and this could make the paper hard to reproduce. The experiments could be expanded.

1) There is a significant amount of work about using generative models to build adversarial examples. The literature review only focuses on classical adversarial robustness and robustness against multiple adversaries. I'd recommend making a review of these approaches, even if they are orthogonal to the one proposed in this paper (e.g., [1,2,3])
2) In Eq. (6), what is \mathcal{B}(x, \epsilon). Since there is multiple threat models, I am assuming that it is selected at random between l_1, l_2 and l_inf (like SAT).
3) The number of inner steps T seems to be critical (as it will trade-off gradient precision with compute). However, I don't see any study on this in the paper. Also, it is not clear which value was used for the experiments.
4) Looking at Eq. (7), it seems like backpropagation through the T inner steps is necessary to compute the gradients w.r.t. \phi. This seems overly expensive and I find surprising that adv_avg and adv_max take so much longer to train.
5) Concerning Eq. (7), as a curiousity, have authors considered implicit differentiation [4] ?
6) The experiments are run using 30 epochs which is rather on slim side. E.g., RST_inf should reach about 59% robust accuracy with 200 epochs of training (with 30 epochs it only reaches 55%). I'm curious as to whether the comparison with the proposed approach is unfair (e.g., Adv_inf sees a single adv example per batch, whereas MNG-AC sees 2).
7) It's not entirely clear to me why beta negatively affects l_2 robustness. I'd assume that if the model was only trained against l_2, then there might be an optimal value for beta that is different that the one from Fig. 2. In general, it would be interesting to see on MNG-AC does if different subsets of threats are used.
8) The l_2 loss landscapes seem more noisy that what they should be. Also it's unclear why the axes are centered for l_inf and not for l_2 (explain how these are generated).
9) In Table 5, MNG-AC achieves 35.1% against all l_inf attacks, but only 33.7% against AutoAttack. Am I missing something?

Details:

A) It would helpful to the reader to have the epsilon values written on top of the different tables. The captions could be expanded to include more details.
B) Visuaization -> Visualization

[1] https://openreview.net/pdf?id=SJeQEp4YDH: GAT: Generative Adversarial Training for Adversarial Example Detection and Robust Classification
[2] https://arxiv.org/pdf/1801.02610: Generating Adversarial Examples with Adversarial Networks
[3] https://arxiv.org/pdf/1710.10766: PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examples
[4] https://arxiv.org/pdf/1911.02590: Optimizing Millions of Hyperparameters by Implicit Differentiation
review_title: Initial review

final_score: rating: 5: Marginally below acceptance threshold
review_content: Summary
=======
The authors propose a number of techniques to learn models which are adversarially robust to multiple perturbations. These involve a noise generator, a loss to enforce consistency, as well as a stochastic variant of adversarial training. With these changes, they are able to produce improvements to robust accuracy to multiple perturbation types. 


Overall, I get the idea and the empirical results seem promising. However, the structure and writing of the paper is at times rather confusing, and there are a lot of missing details. If the code were not supplied, it would be difficult in the current state to reproduce the method from the paper. Perhaps due to this, the specifics of the key component, the meta noise generator, are still rather opaque to me. Perhaps the authors can clarify, and I am happy to follow up afterwards. 

Comments for discussion
=======================
The majority of my confusion lies in section 4, for the specifics of the meta noise generator and parts of the algorithm in general. I am otherwise well acquainted with the relevant literature. 

1) Augmented examples (x_aug) are generated by adding noise from the MNG and projecting it onto some ball B. It is not clear to me what ball this is since the authors are considering multiple perturbations. Is it a random type? Or a joint projection? I assume it is at least one of the perturbations being considered, or is that incorrect? 

2) Similarly, in the algorithm, the authors generate adversarial examples (x_adv) by sampling a random attack. I could not find what set of attacks were being sampled from, or what the sampling distribution is (I checked the appendix as well). 

3) The generator is apparently updated to minimize the classifier loss on the adversarial examples as written in Equation (8). However, the adversarial examples are generated from some unspecified set of attacks, which implies that the set of attacks actually depends on the generator somehow. Is this supposed to be the classifier loss on the augmented samples? If not, then how do the adversarial examples depend on the generator? 

4) The consistency loss involves clean, adversarial, and augmented posterior distributions. There are no details on these distributions: are these simply the softmax of the logits? Or is a generative model that outputs a distribution being used? 

5) On a more fundamental level, what is the motivation behind training the generator to minimize the classifier loss? Why would we want to do this over random sampling? What's to prevent a degenerate solution of simply learning to produce a zero perturbation (and thus always producing clean examples, which can achieve low loss)? 


Minor comments
==============
I have checked the supplementary material and the authors have included the code for running their experiments. Ideally, this would also include pre-trained model weights. 

Update
======
After much effort, I can say that I understand the paper. The edits appear to have incorporated all the identified missing information. I have thus updated my confidence and slightly improved my score, however I am not confident that the current presentation of the approach will be understandable by a reader without contacting the authors, given that the difficulty I had in understanding the paper (and my initial confidence) stemmed primarily from missing information and poor presentation for the approach. Although the results do seem to improve upon past work, its impact will suffer if it is difficult to understand for a non-reviewer reader. I would be more confident if a fresh set of eyes could understand the details of the work without having to go to the authors to clarify so many details. 
review_title: Promising results, but method is not clear

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper addresses a timely issue in adversarial robustness - efficient training of robust models against multiple adversarial perturbations. The authors propose a combination of three techniques: stochastic adversarial training (SAT), meta noise generator (MNG), and adversarial consistency (AC) loss for efficient training, and evaluate the robustness using multiple L1, L2, and Linf norm-bounded attacks and three datasets (CIFAR-10, SVHN, and Tiny Imagenet). The results show improved multi-attack robustness over several baselines (including single-attack and multiple-attack models) and reduced training time. Ablation studies are also performed to illustrate the utility of each component of the proposed model. Overall, this paper provides very detailed evaluations involving multiple datasets, attacks, baselines, and robustness metrics. I find the results convincing and important, and also find sufficient novelty in the proposed training method.

The strengths (S) and weaknesses (W) of this submission are summarized below.

S1. The proposal of MNG and AC is effective and novel.
S2. The evaluation is thorough and convincing.
S3. The proposal improves both robustness and training efficiency in most cases.

W1. The adversarial consistency (AC) loss is never defined explicitly. Based on equation (5), it is hard to understand how AC "represents the Jensen-Shannon Divergence (JSD) among the posterior distributions" when considering three distributions, P_clean, P_adv, and P_aug. More clarification is needed.

W2. Although the results show improved multi-attack robustness, it will be great if the authors can add more intuition on why the proposed training method leads to performance improvement. Based on the ablation study,  it seems that the role of SAT and MNG is to reduce overfitting in robustness to encourage generalization, rather than optimization over the worst-case scenarios.

W3. The considered multi-attack setting is still limited to different Lp norm perturbation constraints. Although the authors showed improved robustness over unforeseen attacks, the authors should also discuss how the proposed method can generalize to different attacks beyond Lp norms.



review_title: Thorough Multi-attack Robustness Evaluation and Clever Adversarial Training 

final_score: rating: 6: Marginally above acceptance threshold
review_content: 1. Summary

The authors propose a new method to improve robustness to adversarial examples under various norms (L1, L2 and LInf). Their method combines adversarial training with an adversarial noise generator. They improve upon adversarial training in a multi norm setting by choosing one norm at random for each sample, instead of computing an adversarial for all norms, thus significantly reducing the training time. They additionally improve robustness by regularizing model features between the standard image, the adversarially perturbed image and a perturbation of the image created with an adversarial noise generator.


2. Strengths
+ The method is based on adversarial training. As far as I know and as the authors note this is the only method that reliably leads to more robust models.
+ The authors attack their models with  a range of attacks that to the best of my knowledge are state-of-the art.
+ The method apparently works in the multi norm setting.

3. Weaknesses 
- I was missing an intuitive description why the adversarial noise should improve robustness to adversarial attacks. I was only aware of it as a method to improve corruption robustness.
- I was not always sure if I got everything correctly in sections 4 and 5.3. I think I got it but I sometimes missed a figure. It may e.g. be helpful to include the losses in Figure 1 or make a separate figure. Especially why the MNG was trained the way it is was a bit unclear for me.


4. Recommendation

I think this paper is an accept but as I don't work with adversarial examples I am not at all confident in that assessment. From the discussions with people who work on adversarial examples new defenses are usually broken very quickly and there is a number of papers which break numerous defenses. The method is however based on adversarial training which to my knowledge is the only robust method so far and the used attacks seem valid. So I am definitely leaning towards accept but the opinion of a real expert would be highly appreciated as I feel not at all qualified to assess the validity of papers on adversarial examples.


5. Questions/Recommendations
- Is there a difference between the M(eta)NG and the A(dversarial)NG from Rusak et. al. 2020?


6. Additional feedback 
- None as the paper is pretty well written.
review_title: Review

Meta-review: This is a borderline case. The paper seems solid although some of the numbers are likely incorrect because in some results tables in the appendix the error taken over all attacks is higher than for the best individual attack (which should never happen).

The main contribution of this paper is to augment a standard adversarial loss (against attacks from different norms) with a “consistency” term (consistency between clean, adversarial and noise augmented samples). The relatively large jump in robustness compared to existing schemes that do adversarial training against multiple norms is a bit surprising. A possible explanation could be that the additional consistency term smoothes the landscape around the clean samples a little bit, which could help to find better adversarial examples. The latter would be very similar to a paper by Pushmeet and colleagues (https://arxiv.org/pdf/1907.02610.pdf) which is not cited, but definitely should. It might also be worthwhile to compare to this paper. 

Taken together, this work is interesting but not sufficiently convincing yet to belong to the top papers to be selected for publication at ICLR. 

Decision: Reject