Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: ===== Summary =====
The paper presents a study about the representations learned by neural networks trained using robust optimization — a type of optimization that requires the model to be robust to small perturbations in the data. Specifically, the paper presents results of ResNet-50 trained on ImageNet with standard optimization and robust optimization. The paper draws three main insights from studying the learned representations of the standard and robust networks. First, the representation of the robust network is approximately invertible. In other words, when recovering an image by matching the representation of a random image to the representation of a target image by adding noise, the recovered images are semantically similar to the target image; the recovered images look similar to a human. Moreover, this is also demonstrated with images from outside of the distribution of the training data. Second, the representation of the robust network, unlike the representation of standard network, shows semantically meaningful high level features without any preprocessing or regularization. This leads to the final insight, feature manipulation is easier in robust networks. This is demonstrated by adding noise to an initial image in order to maximize the activation of a specific higher level feature and stopping early to preserve most of the other features of the original image. 

Contributions:
1. The paper demonstrates that robust optimization enforces a prior on the representation learned by neural networks that results in high correspondence between the high-level features of an image and its representation in the network, i.e., similar images share similar representations. 
2. The paper shows that the features learned by networks trained using robust optimization are semantically meaningful to humans without having to use any form of preprocessing. 
3. The paper demonstrates that robust networks facilitate feature manipulation by injecting noise that maximally activates one of the features in the representation. 

===== Decision =====
I consider that this paper should be accepted. The paper does not introduce any new algorithm or shows any theoretical results, but it is a great source of insight and intuition about robust optimization and deep learning. Moreover, the paper excels at the presentation and careful study of each of the main findings and it is well framed within the robust optimization literature. 

===== Comments and Questions =====

There is still a major question that the paper does not directly address, but that is very relevant to robust optimization. Given that robust optimization seems to result in better-behaved and semantically meaningful representations, as evidenced by the findings in the paper, why is it that the performance of the resulting networks, in terms of classification accuracy, is lower than the performance of standard networks (trained with standard optimization)? It seems counter-intuitive that the robust network have worse accuracy than the standard network  given that it is more robust to small perturbations. I am curious if we could obtain any insights about this issue based on what has already being done in the paper. For example, are there any salient features in the images that the standard network classifies correctly but the robust network does not? 

=== Minor Comments ===
1. I think the operator in Equations (1) and (4) should argmin since the noise is being added to x_1 in order to obtain x’_1.

2. What is the meaning of the error bars in Figure 4? I think this should be mentioned in the caption. 

review_title: Official Blind Review #2

final_score: rating: 3: Weak Reject
review_content: ###  Summary
1. The paper proposes robustness to small adversarial perturbations as a prior when learning representations. 
2. It demonstrates that representations that satisfy such a prior have non-trivial properties -- they are easier to visualize, are invertible (i.e. optimizing an input that produces the desired activation leads to reasonable images), and allows for direct manipulation of input features (by changing a feature in the representation space and then optimizing the image to satisfy this new representation.)

### Non-blind review 
THIS IS NOT A BLIND REVIEW
Reviewing this paper reminded me of a recent NeurIPS paper I read. 

I went back to that NeurIPS paper (to better compare the similarities and differences) only to found out:

1- The NeurIPS19 paper cites an earlier arxiv version of this paper as an inspiration for its approach. 
2- It is from the exact same authors. 

This, unfortunately, means I know who the authors are (however, there is no conflict of interest). 

More importantly, this paper is too similar to the NeurIPS paper and It's hard to review without taking into account the NeurIPS paper. In this review, I will treat the said NeurIPS19 paper as published work, and evaluate if this work adds more to the discourse. (I've refrained from naming the neurips paper so the anonymity is maintained for other reviewers; the authors, I presume, would immediately know which paper I'm referring to). 

### Decisions with reasons

Even though I think the idea introduced in this paper is interesting, I would argue for rejecting this paper for the simple reason: It doesn't add much to the existing discourse. 

Using the proposed framework (i.e. learning robust representations), it demonstrates two phenomena.

First, it shows that robust models allow feature inversion. Second, it shows that it's easily possible to directly visualize and manipulate features for such a model. (Both of these are achieved using the same idea: treating input to the model as parameterized, and optimizing for a target activation)

These are interesting observations and show that robust models learn features that rely on salient parts of the input image. However, the NeurIPS19 paper shows this even more cleary. 

As a result, I'm not convinced that demonstrating the same phenomena with different examples is sufficient for this to be a standalone paper. (Perhaps the two papers could have been one single paper). 

### Questions 

What is the rationale behind dividing examples showing robust models rely on salient parts of input into two papers? Is there a semantic meaning to the grouping i.e. showing feature inversion, feature manipulation, and visualization in one paper and Generation, inpainting, translation, etc in another?

If I understand correctly, all of these examples exist because the robustly learned representation relies on the salient parts of the input and not on the non-robust features. If that is the case, it makes more sense to show all of these examples in a single paper. 

### Update after Author's response
Since the authors have added and discussed the pertinent NeurIPS paper in this submission, I'm updating my score. 
I still think that the two papers are more similar than they might seem (See Re: Response 3 for more details). 


### Update 2 

I pointed out the similarities between the three contributions in this paper and the NeurIPS paper in "Re: Response #3" below. The authors replied to my concerns. I'm summarizing the author's position to my concerns followed by my response. 

#### Author's Position
The authors agreed that features manipulation and feature visualization is similar, but pointed out that the chain of dependency is this paper -> NeurIPS paper and not the other way around. They mentioned that the NeurIPS paper cites this paper and acknowledges this. Moreover, they argued that even if we consider NeurIPS paper to be prior work, feature visualization is explored in much more detail in this paper. 

#### Response 
I think the direction of the chain of dependency is not that important since neither paper clearly builds on top of the other. The NeurIPS paper is published work now, and it makes sense to consider it prior work (Especially since it is from the same authors). 

Moreover, during the NeurIPS review period, the authors did not cite this paper; they only added the citation in the camera-ready version. This means that during the NeurIPS review period, they did, in fact,  take credit for the ideas used in feature painting. (The authors mention that they somehow did not, and just stated the method and showed the pictorial result in the NeurIPS paper. However, I don't see how it is possible to present a method and a pictorial result without citing other work and not take credit for the method and result.) 

I would agree with the authors that this paper does go into more detail for feature visualization. More specifically, this paper also looks at visualizing individual features in the representation (The NeurIPS feature painting restricts the visualization using a mask) and demonstrates that the same feature can be used to visualize similar semantic concepts (such as red limbs) on multiple images. This is definitely interesting, but still very related to the feature-painting result. It would have made more sense to include these feature visualization results in the NeurIPS paper instead of adding them in a separate paper.

#### Author's Position 2
They disagreed that feature inversion (this paper) is similar to image generation (NeurIPS paper). I did acknowledge in my initial response that feature inversion is slightly more general than image generation, however, the authors suggest that they are completely different."

#### Response 
I think representation inversion is more similar to generation than it might seem. Representation for an in-distribution image would correspond to a class with high probability. Maximizing a class probability would indirectly optimize for a representation (Say R_0) that maximizes that class probability. Image generation, as presented in NeurIPS paper, can be seen as inverting R_0.
 
Moreover, the qualitative results for feature inversion, as presented in this paper, are not extra-ordinary. In the majority of the inverted images, I can not classify the inverted image correctly. That shows the model is still not paying attention to the correct aspects of the input to do classification. As a result, this paper certainly does not solve the feature inversion problem (Ideally, inverted features would highlight parts of the input necessary for making predictions and ignore other parts. Robust models, on the other hand, seem to be uniformly retaining all information of the image including the background and not highlighting the parts important for making predictions. As a result, many inverted images can not be classified by humans). 

#### My current position 
At the end of the day, both this and the NeurIPS paper are demonstration papers (They are empirically demonstrating an unintuitive phenomenon). Both papers are demonstrating that robust models learn features that correspond to salient parts of the input. Even though both papers are nice, either one is sufficient to demonstrate the phenomenon. For this to be a stand-alone paper, the authors would have to do more in my opinion. One option would be to explore and compare different forms of adversarial robustness as priors (The paper is called "Adversarial Robustness as a Prior for Learned Representations" and not "L2 Adversarial Robustness as a Prior for Learned Representations," after all). Another option would be to see if such representations are 'quantitatively' better in some settings (Such as for transfer learning). 

In its current form, I feel that the two papers are too similar to recommend acceptance. 
review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: Summary:
The paper shows that the learnt representations of robustly trained models align more closely with features that the human perceive as meaningful. They propose that robust optimization can be viewed as inducing a human prior over learnt features. Extensive experiments demonstrate that robust representations are approximately invertible, can be visualized yielding more human-interpretable features, and enable direct modes of input manipulations.

The paper indicate adversarial robustness as a promising avenue for improving learned representations in from several aspects. It is well written and contains extension experimental results. I'd suggest accepting the paper.

Questions and Comments:
- Is there a particular reason that $L_{2}$ norm is used throughout the paper? How is the performance if using other ones?
- Compared to the other methods introducing priors or additional components into the inversion process, how is the quantitative inversion quality and computational complexity of the proposed method?
- The paper claims that the representations are more perceptually meaningful than the others, which may need to be evaluated with broader human subjective.
- I think it should be argmin in Equation (1).
- Also Equation (4) is exactly the same as (1).
- Some symbols seem to be used somewhat interchangeably. E.g., x' in Equation (1) represents x+\delta, while x' in Equation (5) is \delta itself.
review_title: Official Blind Review #1

Meta-review: The paper proposes recasting robust optimization as regularizer for learning representations by neural networks, resulting e.g. in more semantically meaningful representations. 

The reviewers found that the claimed contributions were well supported by the experimental evidence. The reviewers noted a few minor points regarding clarity that seem to have been addressed. The problems addressed are very relevant to the ICLR community (representation learning and adversarial robustness).

However, the reviewers were not convinced by the novelty of the paper. A big part of the discussion focused on prior work by the authors that is to be published at NeurIPS. This paper was not referenced in the manuscript but does reduce the novelty of the present submission. In contrast to the current submission, that paper focuses on manipulating the learned manipulations to solve image generation tasks, whereas the current paper focuses on the underlying properties of the representation. Since the underlying phenomenon had been described in the earlier paper and the current submission does not introduce a new approach / algorithm, the paper was deemed to lack the novelty for acceptance to ICLR. 


Decision: Reject