Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 1: Reject
review_content: Summary
-------
This paper addresses the problem of performing robust transfer learning. A first contribution of the paper is to robust and classic training with respect to usual validation accuracy and robustness to adversarial attacks on the CIFAR task. Then, the same comparison is made on a transfer learning task. The transfer learning setting is then completed by studying transfer from ImageNet-based models with a particular attention to low-data regime and training deeper networks on top of the feature extractor. An analysis of robust features is provided and finally the authors studies the interest of  Learning without Forgetting strategies to provide robust transfer. The tendency s to obtain the Best performance from robust-trained source models having a good validation accuracy.


Overall
------
The paper presents  a study of robust transfer learning that can be interesting for practitioners to know the type of results that can be obtained by robust transfer learning. However, I feel that the results obtained are rather expected and the paper does not provide some interesting methodological contribution that could help to develop robust transfer training. 

Comments
---------

The results obtained in Section 3, 4 and 6 are rather expected and similar.  I think that the paper could benefit by reducing these 3 sections in only one section where the results obtained can be summarized in one big table and two or three figures for example - the complete set of results can then be reported in the supplementary section.

Then, if the contribution of the paper is to propose to focus on robust transfer learning including a Learning without Forgetting strategy, the authors should then focus more on this part and analyze better the behavior of learning.
In particular, the combination between distillation and robust training is certainly interesting, and trying to propose a methodological framework for doing robust training in this context would certainly result in a more significant contribution. How to constrain the feature extraction layers, how to make use of them with distillation and additionally what are the additional contraints/additions that can be made to learning problem (3) to improve robust transfer are some important questions. 

So far, the contribution appears to me rather limited for ICLR. If we restrict to the part related to experimental comparisons made, they are restricted to particular trainings and datasets with specific PGD attacks. The contribution would have been stronger is different types of adversarial attacks with different parameters have been studied and analyzed. 
review_title: Official Blind Review #3

final_score: rating: 8: Accept
review_content: The paper studies transfer learning from the point of view of adversarial robustness. The goal is, given a robust deep neural network classifier for a source domain, learn a robust classifier for a target domain as efficiently and with as few samples as possible. The authors empirically evaluate different strategies and compare with relevant baselines.

At a high level, the paper addresses an interesting problem. Robust models are quite computationally and sample intensive to train, so exploring pre-training is a reasonable way to deal with small datasets or computational constraints.

The authors perform a diverse set of experiments from which I identified the following individual contributions:

a) Retraining the last layer of the model on natural examples preserves robustness. Robustness degrades smoothly when pre-training progressively more layers.
This is an interesting contribution providing evidence that robust models do learn in fact robust input representations/features.

b) Transferring a learned robust representation from a source to a target domain preserves its robustness (training a linear layer on top of it leads to a robust classifier).
This provides further evidence that robust models learn _general purpose_ robust features of the input, while establishing robust pre-training as a valid strategy for cheaper robust models. The baselines considered are: 1) adversarial training on target domain which always underperforms the proposed method, 2) fine-tuning on adversarial samples from the target domain which performs better when there are a lot of samples from that domain and worse when there are only a few (this method is also more computationally expensive than transfer learning).

c) The "perceptually-aligned" saliency maps of Tsipras et al. 2018 are also a property of robust models obtained through transfer learning.
This illustrates that these saliency maps can also arise for out-of-distribution inputs and hence are likely to correspond to general, high-level features.

d) Fine-tuning all layers of the model while ensuring that the representations stay close to the original ones for natural examples can lead to transfer with improved validation accuracy and robustness (even when the source and target domains are the same).
This is an interesting improvement over the simpler transfer methods producing competitive results.

Overall, the paper contains an experimental study that, in my opinion, is thorough, presents interesting findings, and contains the necessary ablations. I believe that this paper would be of interest to the adversarial ML community and I hence recommend acceptance.

review_title: Official Blind Review #2

final_score: rating: 8: Accept
review_content: Paper summary: This paper explores the problem of robustly transfer learning using only standard training (as opposed to adversarial training (AT)) on the target domain. The authors start by highlighting that intermediate representations learned by adversarially trained networks are themselves fairly robust. Then they propose two strategies for robust transfer from a robust model trained on the source domain: (1) naturally fine-tuning the final linear layer on the target domain and (2) naturally fine-tuning all the layers using lifelong learning strategies. They study transfer between CIFAR10 and CIFAR100, as well as, from ImageNet to CIFAR10/100.

High-level comments: Overall, I find the paper interesting and well-written. Prior work from Hendrycks et al. showed that AT on the source domain followed by *adversarial* fine-tuning on the target domain attains better performance as compared to just AT on the target domain. The main contribution of this paper is to show that using instead careful *natural* fine-tuning on the target domain is sufficient to recover a reasonable amount of this robustness. 

Even though the clean/robust accuracy of the proposed approach is lower than just doing adversarial training/prior work from Hendrycks et al., I feel this paper could be useful to the community for two main reasons:

1. The authors perform a nice exploration of the thesis that robust models have robust representations, and how this connects to transfer learning. In particular, the experiments in Figure 1 (the effect of naturally re-training later layers of a robust network on its robustness) and Figure 5 (where the authors show that their naturally fine-tuned models have some of the unexpected benefits from Tsipras et al.) seem particularly interesting.

2. Despite its lower accuracy, this approach could be useful in settings where data is scarce or compute is expensive, and hence adversarial training on the target domain is not successful. 

Specific comments/questions:

i. Could the authors clarify what they mean by point 3 (re: validation accuracy drop) below Table 3? As far as I can tell, the drop in clean validation accuracy between Table 1 and Table 2 are similar for both the naturally and adversarially pre-trained models.

ii. In Table 2, it would also be interesting to see the performance when the source domain is CIFAR10 and the target domain is CIFAR100.

iii. For Table 2 is the eps=8? This should be mentioned in the caption as it is important to highlight that Tables 2 and 3 are not directly comparable.

iv. Why isn’t experiment in Figure 3 should be repeated for CIFAR10 as well? The authors should add this result to the paper, even if in the appendix. It is important to verify that this trend is not specific to CIFAR100 and holds across datasets (even though CIFAR10/100 are not too different).

v. The comment at the end of page 6 re: natural model is confusing (“Note, this seems to...perfectly”)---as far as I can tell, Figure 4 does not include the results of fine-tuning a naturally trained model.

vi. General comment motivated by the comment ("Note...perfectly") mentioned 5 above: For all the adversarial evaluation in the paper, the authors should also try CW attacks/black-box attacks to get a more confident estimate of their model’s robustness.

vii. The authors reference prior work on the tradeoff between robustness and accuracy and motivate Section 6 as an avenue to alleviate this trade-off for their model. However, I don’t see the lower performance of their model as an instance of this trade-off---the model in the paper performs worse in terms of both clean and adversarial accuracy. The approach proposed in Section 6 seems interesting, but more as an approach to improve the *overall* performance of the model. The authors mention this in retrospect, but I think the narrative of this section should be modified to make this clearer.

viii. In Table 5, in the experiments corresponding to CIFAR100+ -> CIFAR100, is the dataset split into two halves (for the source and target domains) or is the fine-tuning performed on the same data. In general, I find it odd that natural fine-tuning on the *same data* can improve both the clean and adversarial accuracy of the model (compared to the CIFAR100+ robust baseline). Is the robust model trained long enough/with enough hyperparameter search?

Overall, the exploration in the paper seems novel and could be useful to the community. Thus, I recommend acceptance.

review_title: Official Blind Review #1

Meta-review: This paper presents an empirical study towards understanding the transferability of robustness (of a deep model against adversarial examples) in the process of transfer learning across different tasks.

The paper received divergent reviews, and an in-depth discussion was raised among the reviewers.

+ Reviewers generally agree that the paper makes an interesting study to the robust ML community. The paper provides a nice exploration of the hypothesis that robust models learn robust intermediate representations, and leverages this insight to help in transferring robustness without adversarial training on every new target domain. 

- Reviewers also have concerns that, as an experimental paper, it should perform a larger study on different datasets and transfer problems to eliminate the bias to specific tasks, and explore the behavior when the task relatedness increases or decreases.

AC agrees with the reviewers and encourages the authors to incorporate these constructive suggestions in the revision, in particular, explore more tasks with different task relatedness.

I recommend acceptance, assuming the comments will be fully addressed.
Decision: Accept (Poster)