Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: The paper looks at the idea of building models of natural variation of an input and then using these models to develop robust training algorithms that are less susceptible to outliers ( testable for worst-cases via adversarial attacks ). 

Strength:

+ The paper addresses a very important topic of adversarial robustness of DNN models and is accompanied by diligent evaluation over different datasets. 

Weakness:

- The paper adopts an approach which is very reminiscent of Ajil Jalal, Andrew Ilyas, Constantinos Daskalakis, and Alexandros G Dimakis. The robust manifold defense: Adversarial training using generative models. arXiv preprint arXiv:1712.09196, 2017. The key difference seems to be adoption of the idea of using auxiliary transformations (called natural perturbations) which is also very well-studied in literature, for e.g. see https://openaccess.thecvf.com/content_CVPR_2020/papers/Zhang_Auxiliary_Training_Towards_Accurate_and_Robust_Models_CVPR_2020_paper.pdf  So, the approach presented here is quite incremental for a premier venue such as ICLR.

- The attack in Jalal et. al. paper can be launched against this approach once the attacker also access to this generative/natural model (which would be easy to build for an attacker)

- The reviewer will strongly recommend reviewing the advices in https://arxiv.org/abs/1902.06705 for writing papers on defense and how to self-evaluate its robustness by suitably designing the attacker. If the defense approach uses some background/auxiliary knowledge, one must consider the attacker with this knowledge if it is accessible to the attacker. 

Questions to authors:

- Can authors explain why an attacker can't build similar natural model to defeat the proposed defense? 
- Any clarification on incremental novelty from  Jalal et. al. and Zhang et. al. (and references therein) would be also useful. 


After author's rebuttal:

"The other paper that the review points to ([4]) addresses a similar setting to our paper, but the approach is completely different. ... Moreover, [4] was published within three months of the ICLR submission deadline, meaning that it is essentially concurrent with our work."

Yes, the reviewer concurs that a work published so close to deadline should be treated as concurrent and will raise the score.
review_title: Unclear novelty and very similar to a broken robustness method ( defense )

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper proposes a model-based framework for improving the robustness of image classifiers to average-case corruptions of varying severity. The proposed framework can be thought of as adversarial training where the perturbation is replaced by a function that transforms the image according to a specific corruption. A nuisance parameter controls the instantiation and severity of the corruption that is applied to the input. The paper compares baselines to different versions of this general model-based framework with experiments on several datasets.

Pros:
- Focusing on average-case corruptions is an important and underexplored problem
- Using image translation to learn the corruptions is an interesting proposal

Cons:
- There isn't much novelty in the proposed technique. Adversarial training with 'natural' corruptions has been done several times before.
- There isn't any discussion of how the nuisance parameter is used. I suspect it isn't used for ImageNet-C experiments, though I may be wrong. In this case, the method becomes data augmentation with image translation, which is not very novel either.
- It sounds like the ImageNet-C experiments use the corruptions to train the image translation network. Even though these experiments use held-out images, the corruption types themselves are not being held out, which goes against the recommended methodology for that dataset.


Typo: "ImageNet-c" should be "ImageNet-C"

___________________________________________________________________________________

Update after author feedback:

I thank the authors for improving my understanding of the paper. I feel better about it after reading it again. One of the most interesting findings--that a translation network trained on one domain/set of classes generalizes to another--needs more discussion. Using this phenomenon to improve robustness is a good idea, and the MRT/MDA/MAT methods explored in this work are nice choices for this investigation. However, I agree with the other reviewers that it would be nice to include more baselines from other work where appropriate.

The results in Table 1 are also very interesting and deserve more discussion--perhaps an analysis of whether a translation network trained on weak corruptions can generalize to create something akin to the ground-truth stronger corruptions, as the results in the table for MRT imply.

Overall, I think this paper has some strong experiments and investigates a good idea, but the claims of a "paradigm shift" are overly grandiose, and some of the most interesting experiments could use more analysis. Additionally, the writing clarity and presentation could be improved. My initial understanding of the paper was flawed in some places, so I'll raise my score to 5.
review_title: Official Blind Review #3

final_score: rating: 5: Marginally below acceptance threshold
review_content: The paper extends current adversarial learning approaches beyond imperceptible L_p norm perturbations. The proposed approach can handle many models of natural variation, such as a change in brightness. The main idea behind the approach is to use unsupervised approaches such as GANs to model the natural variation. Given this model of natural variation, the paper replaces the adversarial learning objective of finding the worst example in an L_p norm ball around a data point to finding the worst example based on the model of natural variation. This is expensive, so the paper also proposes more computationally efficient approaches based on data augmentation. The experimental results demonstrate that the proposed approach performs well on a variety of tasks.

Strengths:

1. The question of training robust neural networks is clearly timely and well-motivated. A valid raised criticism of the current adversarial examples literature is that it is too tied to imperceptible or L_p norm perturbations, and this paper tries to go beyond them.
2. There is a bit of a lack of relevant methods to compare against, but the experiments still show that the proposed approach does well. I also appreciated the comparisons to domain adaption, and the multiple experimental setups generally.

Weaknesses:

1. Though the paper suggests that the approach is a “paradigm shift” in robust deep learning, it appears to be more or less an extension of the current adversarial examples literature. The adversarial training approach has been quite successful in defending against a fixed adversary model, from various L_p norms to geometric transformations. This paper is essentially defining a new adversary model, given by natural variations. In fact it is simpler than this, since the goal is to only do well on "random variations" in the predefined test set, i.e. there is no adversary which will try and attack the network by using the class of variations. There is a novelty here though that the variation model is learnt rather than pre-specified. But given that learning transformations such as change in brightness etc. is quite straightforward for unsupervised models such as GANs, this is not too much to ask for. Therefore, I find it a bit unsurprising that the proposed method works well, given the success of adversarial training and GANs.
2. Related to the above, in my view one main drawback of the current adversarial learning literature is that it is not difficult to get robustness to one pre-specified model of perturbation/variation, but in reality the space of possible variations is quite large. It is good that the proposed approach does seem to work on two simultaneous shifts, but the approach still seems restricted and not a substantial change in the current understanding. Note that some of the compared approaches are different in this regard: AugMix does not tailor the model to a particular variation (it explicitly excludes operations which overlap with ImageNet-C corruptions) and domain adaptation can handle quite large changes in the data distribution (which are not just transformations of the original data points). 
3. Perhaps less importantly, in the experiments the proposed MDA approach based on data augmentation does almost as well as the other ones, and data augmentation using unsupervised models is already studied in the literature.

Overall, this is not a bad paper and I am not completely opposed to acceptance, but I am not sure I can argue for it.

Other less important comments:

1. Since there is a lack of baselines to compare against, the paper could benefit from a few ablation studies. For example, is there any difference on using other unsupervised approaches? 
2. I think it would be better to have a concise list of a few major contributions in Section 1, rather than the current rather long list.
3. It would be nice to have representative images of the models of variation, perhaps in an appendix.  

------Updates after author response------

I thank the authors for the response and it helps clarify some points. However, I am still not unconvinced that the paper is a significant departure from current work on adversarial robustness (see weakness 1 and 2 above). I think it would be much more interesting if the approach could yield robustness against a wider/different class of shifts compared to what it was adversarially trained against. Therefore, I am keeping my score at 5 and will not advocate for acceptance, though I am not completely opposed to it.
review_title: Extends adversarial learning to natural variations using GANs, but appears to have somewhat limited contribution

final_score: rating: 5: Marginally below acceptance threshold
review_content: # Summary
This paper proposes a “paradigm shift” for augmenting datasets when training CNN-based image classifiers. On one side, traditional augmentations include blur, Gaussian noise, color distortions. On the other side, methods like adversarial training consider augmentations under norm bounds in the image space. The proposed method, instead, uses models of natural variation to augment the images. Increased out-of-domain accuracy is shown on ImageNet-C and several slices of the CURE-TSR dataset. 

Central question: How to shift from perturbation-based robustness analysis to model-based robustness analysis?

# Strong and Weak points

## Strong points 

  * This paper introduces a “paradigm shift” from perturbation based to model-based augmentation, when training CNN based image classifiers. The existing perturbations only reach so far, and using models of natural variation provides a new avenue for augmenting data sets. 
  * The method is presented in clear pseudo code, and detailed in the 36 page counting appendix. 

## Weak points

  * The paper claims a “paradigm shift”, but the proposed method of augmenting data has been used since 2012,  [1], and is used ever since [2]. Tables 2 and 3 provide key results for this method, but both were obtained with Model-based Data Augmentation, which is the common method before the "paradigm shift".
  * Results outlined in Section 5.2 and Table 2 concerns scenarios where the test-time perturbation is known at training time. Table 2 then compares MDA to ERM using this a-priori knowledge, and shows improved accuracy. However, the comparison is unbalanced, as ERM does not benefit from the prior knowledge. If Table 2 would be about comparing methods under the assumption that the test-time perturbation would be known at training time, then the ERM should be trained with the same data augmentation.
  * The paper misses comparisons against previous published work. Although Table 2 and 4 compare against another method, these results were not obtained by the respective authors, thus introducing a bias. The related work, Section 6, cites as many as 9 papers that addressed the same problem as this paper. The argument for the proposed method could be stronger when comparing to results from any of these papers. 
Moreover, Table 3 reports results on ImageNet-C, which multiple other research papers have published results on. See for example, [3] and [4]. Likewise, Table 4 reports results on MNIST-C, which have been reported in [5] and [6]. Omitting those comparisons hampers the argument for the proposed method. 

[1] Krizhevsky, 2012
[2] Szegedy, 2015
[3] https://paperswithcode.com/sota/domain-generalization-on-imagenet-c
[4] https://arxiv.org/pdf/1903.12261.pdf 
[5] https://proceedings.icml.cc/paper/2020/file/20546457187cf3d52ea86538403e47cc-Paper.pdf 
[6] https://arxiv.org/pdf/1906.02337.pdf 

# Statement

Recommendation: Reject
Reason:
Although this paper provides four tables of extensive evaluation of the method, none of the numbers were obtained from previous literature. The related work section cites 9 papers on the same problem, but none were compared with in the results section. (See concrete details above)

# Clarifying questions

  * How to quantify the increase in computational complexity for the inner maximization in Equation (2)? As I understand, even evaluating one point requires a forward pass of the variation-model and the classifier-model. In case of iterative optimization, this would also require two backward passes(see line 5 in Algorithm (1)). 
If the proposed method requires significantly more compute, the comparison should be made with a method that has access to the same amount of compute (for example a larger model, or longer trained model).
  * Line 6 of Algorithm (1). I miss the definition of set $\Delta$ and the projection operator in the text nor the appendix. Please explain?
  * Section 5.2 speaks of ImageNet “classes”. I am confused if these “classes” are referring to the ImageNet classes (1000 in total) or the severity levels in ImageNet-C (75 in total). Could you please clarify?
  Moreover, this argument implicitly assumes that the test-time perturbation is known at training-time. Could you name a scenario where this assumption is justified?
  * To what extent is the accuracy on the clean data preserved by the proposed method? Table 3, for example, shows an increase in top-1 accuracy on ImageNet data under perturbation. However, I wonder if this model preserves accuracy on the clean data. We know from existing literature [1] that a trade off exists for adversarial training between accuracy on perturbed and clean data. Providing the clean accuracy would provide evidence that this method compares favourably on this trade off. 

[1] Zhang, Hongyang, et al. "Theoretically principled trade-off between robustness and accuracy." arXiv preprint arXiv:1901.08573 (2019).

# Minor feedback

This minor feedback is not part of the assessment.

  * “arguably more common” -> “arguably more important”
  * “x with a corresponding label y” -> “x with a corresponding prediction y”
  * Abbreviation “MBRDL” appears in Section 5 without any explanation or definition. 
  * Abbreviation “CURE-TSR” appears in Section 5.0 without any explanation or definition.
  * The citation for CURE in Section 5.1 is wrong. The bibliography refers to “Temel, Dogancan, Min-Hung Chen, and Ghassan AlRegib. "Traffic sign detection under challenging conditions: A deeper look into performance variations and spectral characteristics." IEEE Transactions on Intelligent Transportation Systems (2019).”, but the CURE TSR dataset was published in “Temel, Dogancan, et al. "CURE-TSR: Challenging unreal and real environments for traffic sign recognition." arXiv preprint arXiv:1712.02463 (2017).”
  * Caption of Table 4: “ Then, we use this model to perform model-based training on a new dataset D1” -> “ Then, we use this model to perform model-based testing on a new dataset D2”? 


review_title: Review of "Model-Based Robust Deep Learning: Generalizing to Natural, Out-of-Distribution Data"

Meta-review: # Quality:
The technical contribution of the paper seems reasonable and there were only minor points being highlighted by the reviewers.

# Clarity:
The paper would benefit from being more polished. During the rebuttal, the authors suggested that several reviewers misunderstood the paper. This alone should encourage the authors to improve clarity.

# Originality:
Several reviewers presented concerns about the claims of the authors and the existence of connections to existing literature. Nonetheless, the proposed approach seems novel to the best of the reviewers and my knowledge.

# Significance of this work: 
The topic of the manuscript is relevant and impactful. However, several reviewers suggested to include additional baselines in the experiments to validate the goodness of the proposed approach.

# Overall:
The paper presents an interesting idea, with a high potential impact. Despite the interesting topic and some interesting insights, all the reviewers agree that the manuscript is not ready for publication just yet. I want to encourage the authors to keep improve it and resubmit it at the next conference.
Decision: Reject