Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: ## Paper summary
This paper addresses the problem of constraining adversarial image perturbations to be similar to some natural class of perturbations. This would allow natural perturbations to be treated with the same rigor (e.g. quantifiable attack and defense strengths) as standard adversarial perturbations. Standard adversarial perturbations are arbitrary perturbations of an image that are within a certain $\mathcal{l}_p$-ball around the datapoint in pixel space. They are otherwise unconstrained and therefore appear as unstructured noise. Instead, the paper proposes to train a conditional autoencoder to generate perturbed versions of clean images from pairs of clean and perturbed images. The autoencoder can then be used to generate new perturbations that are similar to the training data. Adversarial versions of these perturbations can then be defined in the latent space of the autoencoder, rather than in pixel space. The paper provides theoretical arguments that perturbations generated in this way are close to the perturbations used for training. Using these learned adversarial perturbations, models can then be trained that are robust to them.

## Arguments for acceptance
* The paper provides a method for generating adversarial perturbations that are similar to natural perturbations, such as lighting changes. This has great promise for studying and improving the robustness of vision models to natural perturbations.
* The paper provides a convincing motivation and thorough theoretical justification for the proposed method.
* The theoretical section of the paper is clear and well written.

## Arguments against acceptance
* The experimental section only mentions the bare minimum of methodological details and results, and instead defers most experiments to the extremely extensive appendix. For example, the experiments on the multi-illumination dataset, while very intriguing, are barely intelligible without reading the appendix. Perhaps a venue allowing longer articles (e.g. a journal) would have been more appropriate.
* The method is evaluated only on CIFAR and the multi-illumination dataset. It is unclear if the method scales to larger datasets such as ImageNet-C.

## Conclusion and suggestions
This paper provides a well motivated and generally applicable method for generating adversarial examples that are similar to natural image perturbations. This work will be of interest to many ICLR attendees, so I recommend acceptance.

Suggestions for improvement:
* Move material from the appendix to the experimental section, e.g. more detailed descriptions of the experimental methods.
* Repeat experiments on ImageNet-C.
* Discuss computational cost/scalability of the method in the main paper.
review_title: An interesting and well-motivated method for studying and improving robustness to natural perturbations.

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper aims to learn generative models, CVAE in particular, such that a fixed region of the latent space in CVAE corresponds to a possible perturbation set in adversarial robustness. The CVAE is learned from collected perturbation pairs, and it is expected to contain close approximations of the perturbed data and assign sufficient likelihood to perturbed data. 

Pros:

* This paper novelly proposes to learn perturbation sets with CVAE and addresses the problem of learning robust models without a predefined perturbation set.
* The paper defines measures for evaluating the quality of learned perturbation sets and theoretically proves learning a CVAE matches the quality measures. 
* The paper demonstrates a robustness improvement on OOD corruptions and learned perturbations, and also some collected perturbed data, with CVAE data augmentation or adversarial training. It is also shown that the approximation error of the learned CVAE appears to be small. 

Cons:

* In terms of robustness, the CVAE generator itself may not be adversarially robust and bring additional robustness concerns. For instance, is it possible to somehow attack the CVAE to find some small perturbations that can be missed by the learned CVAE? Or maybe it is possible to find a perturbed input whose perturbation set by CVAE is broken? I think it may also be important to take the robustness of CVAE itself into consideration when trying to use CVAE to model perturbation sets.
* In Table 3, CVAE training achieves much lower test set accuracy on perturbed data (41.7 v.s. 45.2), and it raises a concern whether the learned perturbation set is really good enough to match collected perturbed data. 

=================================================================

Post-rebuttal update: Thanks to the authors for their clarifications. I have read author response and understand that the cons I mentioned above might be orthogonal to the focus of this paper. Thus I have revised my rating to 6.

review_title: Novel method but the CVAE generator itself may have robustness concerns

final_score: rating: 6: Marginally above acceptance threshold
review_content: ===============================================================================================

Post-author response: I have read the response and am satisfied with the answer. I am leaning more towards accepting this paper.

====================================================================================================
This work approaches adversarial training (robustness) from a different approach by learning the uncertainty set, as opposed to studying a prespecified one such as those using divergences. This problem is largely unsolved and quite an important one since in most adversarial training settings, it is typically difficult to understand what uncertainty is (when using other forms of perturbations). The proposed method *learns* an uncertainty set using a combination of neural networks and data and principally motivated. Theoretical results are then given and particularly shown how it can be solved using Conditional Variational Autoencoders (CVAE) and the properties are then empirically studied.

Pros:
- A novel idea of learning uncertainty sets, which gives more interpretation than standard uncertainty sets.
- Derivation of CVAEs as useful for uncertainty sets.

Cons:
- Unclear how useful this would be for future developments.
- Theoretical results are slightly incomplete (minor).

I think this work makes a step in an important direction however I am somewhat reserved regarding future developments of such a method. In particular, I have two questions

(1) What relation, if any, does this method have to distributional robustness? I would imagine there is some link since the distribution sets become conditional distributions. There have been recent developments in this area such as in [1,2,3] which are not discussed, which are possibly highly relevant if there is a link to distributional robustness.

(2) while it is nice that uncertainty sets can be constructed with CVAEs, how can one satisfy Assumption (1)? Is there any particular structure in Assumption (1) that can be rewritten into a single objective? I believe this would strengthen at least the theoretical foundations of the results in this paper.

While I have these concerns, they may be answered in a more theoretically focused paper. I believe the problem is important and the proposed method is novel and therefore I am in favour of accepting this paper.

[1] Staib, M., & Jegelka, S. (2019). Distributionally robust optimization and generalization in kernel methods. In Advances in Neural Information Processing Systems (pp. 9134–9144).

[2] Blanchet, J., & Murthy, K. (2019). Quantifying distributional model risk via optimal transportMathematics of Operations Research, 44(2), 565–600.

[3] Husain, H. (2020). Distributional Robustness with IPMs and links to Regularization and GANsarXiv preprint arXiv:2006.04349.
review_title: Interesting approach with promising results

final_score: rating: 5: Marginally below acceptance threshold
review_content: **Summary.** In this work, the author(s) have presented an approach to identify valid perturbation operations that can be applied to the model inputs, which can be exploited to boost model robustness via purposefully corrupting the inputs during training. A weakly supervised setting has been assumed, such that pairs of valid perturbations (x, x') are available to support the identification. Technically, a conditional variational auto-encoder is trained to capture possible variations, where the perturbated inputs are used for reconstruction. The latent codes are treated as source space disruptions, concatenated with the original input x to reproduce x'. The author(s) provided empirical evidence to support their claim, along with some theoretical justifications. 

**Quality.** While the presentation is okay, I have concerns wrt the technical novelty and significance of this work. My detailed arguments are provided below. 

**Clarity.** The first three pages of this paper are very well written. The author(s) have clearly defined the problem setup, adequately reviewed relevant literature, and heuristically motivated the technical route. However, confusions arise when the author(s) try to provide some theoretical underpinnings. While I do not particularly mind if a paper has incredible empirical performance but offers very limited theoretical insights, I do find it annoying if some not-so-well-developed "theories" have been imposed upon the paper to boost technical significance. Notations of perturbated distributions are not clearly defined (Eqn (3)). Also, the necessary subset property (Defn. 1) and sufficient likelihood (Defn. 2) are heuristically defined, and subsequent development seems to inconsistently use these notations. For example, the reconstruction error in Thm 2 is actually the expected log-likelihood, rather than expected likelihood as defined in Defn. 2. It is also not clear what exactly does "Let r be the Mahalanobis distance which captures 1 − α of the probability mass for a k-dimensional standard multivariate normal for some 0 < α < 1. " mean. 

**Originality.** While the idea of applying CVAE to learn valid perturbations is new, the technical contributions seem very incremental. It feels like a direct application of CVAE to this problem, rather than develop specialized treatments to tailor the solution. 

**Significance.** This is an interesting proposal. But I encourage the author(s) to include additional comparisons wrt related techniques in order for the reviewers to better evaluate significance. For example, baselines such as AutoAug should be compared, as these methods also try to identify valid perturbations that do not affect the label. I would also love to see an adversarial variant of the proposed approach is compared as well. 

**Correctness.** The main reason I am leaning towards a rejection is that I have serious concerns about the correctness of the theories presented. The author(s) seem to imply the KL terms should go away in order to bound the approximation error, this is definitely not the case for VI. Such a phenomenon is more commonly known as KL-vanishing in VAE literature, which is typically associated with uninformative latents -- where VAE learning has failed. The limit of R also doe not make much sense. 

review_title: Interesting work, weak theoretical justification

Meta-review: The authors propose an approach to learn perturbation sets from data and go beyond the mathematically sound L_p adversarial perturbations towards more realistic real-world perturbations. To measure the quality of the learned perturbation set the authors put forward two specific criteria and prove that an approach based on conditional variational autoencoders (cVAE) can satisfy these criteria. In particular, given access to paired data (instance and its perturbation), the authors train a cVAE which can then be used to generate novel perturbations similar to the ones observed during training. Leveraging this generative model the authors train models which are robust to such perturbations while improving the generalisation performance on clean data.

The studied problem is of high significance and the proposed solution is sufficiently novel. The reviewers agree that the paper presents a significant step in the right direction and will be of interest to the ICLR community. The authors addressed all major concerns raised by the reviewers. In my opinion, given the inherent tradeoff between the two terms in Assumption 1, and the approximation gap due to the design choices of the particular cVAE, I feel that a hard problem was reduced to an almost equally hard problem. Nevertheless, the principled approach coupled with promising empirical results are sufficient to recommend acceptance. I strongly advise the authors to incorporate the remaining reviewer feedback and try to tone down claims such as “certifiably robust” given the issues pointed out above. 

Decision: Accept (Poster)