Adversarially-Trained Deep Nets Transfer Better: Illustration on Image Classification

1 INTRODUCTION . While deep neural networks ( DNNs ) achieve state-of-the-art performance in many fields , they are known to require large quantities of reasonably high-quality labeled data , which can often be expensive to obtain . As such , transfer learning has emerged as a powerful methodology that can significantly ease this burden by enabling the user to adapt a pre-trained DNN to a range of new situations and domains ( Bengio , 2012 ; Yosinski et al. , 2014 ) . Models that are pre-trained on ImageNet ( Deng et al. , 2009 ) have excellent transfer learning capabilities after fine-tuning only a few of the last layers ( Kornblith et al. , 2019 ) on the target domain . Early work in transfer learning was motivated by the observation that humans apply previously learned knowledge to solve new problems with ease ( Caruana , 1995 ) . With this motivation , learning aims to extract knowledge from one or more source tasks and apply the knowledge to a target task ( Pan & Yang , 2009 ) . The main benefits include a reduction in the number of required labeled data points in the target domain ( Gong et al. , 2012 ; Pan & Yang , 2009 ) and a reduction in training costs as compared to training a model from scratch . However , in practice , transfer learning remains an “ art ” that requires domain expertise to tune the many knobs of the transfer process . An important consideration , for example , is which concepts or features are transferable from the source domain to the target domain . The features which are unique to a domain can not be transferred , and so an important goal of transfer learning is to hunt for features shared across domains . It has recently been shown that adversarially-trained models ( henceforth denoted as robust models ) capture more robust features that are more aligned with human perception , compared to the seemingly patternless features ( to humans , at least ) of standard models ( Ilyas et al. , 2019 ) . Unfortunately , ∗Equal contribution these models typically have a lower generalization performance on the source domain , as compared to non-adversarially-trained ( henceforth denoted as natural , as in previous works ( Tsipras et al. , 2019 ; Shafahi et al. , 2019 ; Salman et al. , 2020 ) ) model . Hence , Ilyas et al . ( 2019 ) hypothesize that non-robust features that are lost during adversarially training may have a significant positive impact on generalization within a given dataset or domain . This inherently different feature representation between models constructed with adversarial training and models trained with standard methods would also explain why accuracy and robustness are at odds ( Tsipras et al. , 2019 ) . This leads to the question of whether models that use robust representations generalize better across domains . This is the main question we address . In this work , we demonstrate that robust models transfer better to new domains than natural models . To demonstrate this , we conduct an extensive number of transfer learning experiments across multiple domains ( i.e. , datasets ) , with various numbers of fine-tuned convolutional blocks and random subset sizes from the target dataset , where the critical variable is the constraint used to adversarially train the source model . ( Described in detail in Sections 3 and Appendix A.3 ) Importantly , note that we do not use an adversarial training procedure for the actual transfer learning process . Our findings indicate that robust models have outstanding transfer learning characteristics across all configurations , where we measure the performance in terms of model accuracy on target datasets for varying numbers of training images and epochs . Figure 1 provides a summary of our approach . Our focus in this work is to show that robust source models learn representations that transfer better to new datasets on image recognition tasks . While adversarial training was proposed to combat adversarial attacks , our experiments discover an unintended but useful application . Adversarial training retains the robust features that are independent of the idiosyncrasies present in the source training data . Thus , these models exhibit worse generalization performance on the source domain , but better performance when transferred . This observation is novel , and we undertake extensive empirical studies to make the following contributions : • We discover that adversarially-trained source models obtain higher test accuracy than natural source models after fine-tuning with fewer training examples on the target datasets and over fewer training epochs . • We notice that the similarity between the source and target datasets affects the optimal number of fine-tuned blocks and the robustness constraint . • We show that adversarial training biases the learned representations to retain shapes instead of textures , impacting the source models ’ transferability . • We interpret robust representations using influence functions and observe that adversarially-trained source models better capture class-level semantic properties of the images , consistent with human concept learning and understanding . 2 RELATED WORKS . ImageNet transfers . Our focus is on studying the transfer of all but the last few layers of trained DNNs and fine-tuning the last non-transferred layers . For ease of exposition , we restrict our attention to ImageNet models ( Deng et al. , 2009 ) . Kornblith et al . ( 2019 ) study the transfer of natural models to various datasets and is thus a prequel to our work . Yosinski et al . ( 2014 ) also study transferring natural models but focus on the importance of individual neurons on transfer learning . Recht et al . ( 2019 ) study the generalization of natural and robust models to additional data generated using a process similar to that of generating ImageNet . They conclude that models trained on ImageNet overfit the data . However , they study the models ’ generalization as-is without fine-tuning . Covariate shift . A significant challenge in transfer learning is handling the data distribution change across different domains , also called covariate shift . It ’ s widely recognized in successful domain adaptations ( Yosinski et al. , 2014 ; Glorot et al. , 2011 ) that the representations in earlier layers are more “ generic ” and hence more transferable than the ones in later layers . This hierarchical disentanglement is attributed to the properties of the data itself , so that the later layers are more closely associated with the data and do not transfer as well . This motivated studies for shallow transfer learning ( Yosinski et al. , 2014 ; Ghifary et al. , 2014 ) and more general studies to extract features that remain invariant across different data distributions ( Arjovsky et al. , 2019 ) . In Section 5 we see that adversarial training biases the learned representations to retain shapes instead of textures , which may be a more desirable invariant across the datasets . Transfering adversarially-trained models . There are mainly two works directly associated with ours . First , subsequent to this paper ’ s initial posting ( in a non-anonymized form in a public forum ) , Salman et al . ( 2020 ) posted a related paper . They arrived at broadly similar conclusions , confirming our main results that robust models transfer better ; and they do so by focusing on somewhat different experiments , e.g. , they focus on the effects of network architecture width , fixed feature transfer , and seeing if models without texture bias transfer better than robust models . Second , Shafahi et al . ( 2020 ) mainly find that models lose robustness as more layers are fine-tuned . It might seem to contradict our thesis that they also notice that an ImageNet robust model with a ‖δ‖∞ ≤ 5 constraint has lower accuracy on the target datasets , CIFAR-10 and CIFAR-100 , compared to a natural ImageNet model . However , we show that the robust model transfers better than the natural one when we use a ‖δ‖2 ≤ 3 constraint to adversarially train the source model . Example based interpretability . There has been significant interest in interpreting blackbox models using salient examples from the data . A line of research focuses on using influence functions ( Koh & Liang , 2017 ; Koh et al. , 2019 ; Khanna et al. , 2019 ) to choose the most indicative data points for a given prediction . In particular , Khanna et al . ( 2019 ) discuss the connection of influence functions with Fisher kernels ; and Kim et al . ( 2016 ) propose using criticisms in addition to representative examples . Complimentary lines of research focus on interpretability based on human-understandable concepts ( Bau et al. , 2017 ) and feature saliency metrics ( M. Ancona , 2017 ) . 3 BRIEF OVERVIEW OF THE ADVERSARIAL TRAINING PROCESS . Adversarial training modifies the objective of minimizing the average loss across all data points by first maximizing the loss produced by each image with a perturbation ( i.e. , a mask ) that may not exceed a specified magnitude . Here , we describe this process , similar to Madry et al . ( 2018 ) . Let ( xi , yi ) be m data points for i ∈ [ m ] , where xi ∈ Rd is the ith feature vector , and yi ∈ Y is the corresponding response value . Typically , we model the response as a parametric model hθ : Rd → Y with a corresponding loss function ` : Y ×Y → R≥0 . The objective is to minimize the loss ` ( ŷ , y ) , where ŷ = hθ ( x ) is the predicted response . Adversarial training replaces the above minimization problem of training the model by a minimax optimization problem to make the model resilient to arbitrary perturbations of inputs . The goal of adversarial training is to solve a problem of the form min θ 1 m m∑ i=1 max ‖δi‖p≤ ` ( hθ ( xi + δi ) , yi ) . ( 1 ) That is , the goal is to find the parameters θ of the model hθ that minimize the average maximum loss obtained by perturbing every input xi with a δi constrained such that its ` p norm does not exceed some non-negative . If = 0 , then δi = 0 , in which case there is no perturbation to the input , which is what we call natural training . As increases , the magnitude of the perturbation also increases . For more details on how we solve this problem , and a few examples , see Appendix A.2 . 4 TRANSFERRING ADVERSARIALLY-TRAINED MODELS . In this study , we train four ResNet50 source models on ImageNet . We train one of them naturally ( non-adversarially ) , and train each of the remaining three adversarially with one of the following constraints : ( i ) ‖δ‖2 ≤ 3 , ( ii ) ‖δ‖∞ ≤ 4255 , ( iii ) ‖δ‖∞ ≤ 8 255 . Next , we fine-tune some convolutional blocks in the source models to each of the six target datasets separately using a subset of the training data . We repeat each of these trials for various seed values and report the mean and 95 % confidence interval . Altogether , we have a comprehensive and replicable experimental setup that considers four ImageNet source models , four fine-tuning configurations , six target datasets , ten random subset sizes , and an average of fifteen random seeds for a total of 14,400 fine-tuned models . For more details , see Appendix A.3 and A.4 . Adversarially-trained models transfer better and faster . For ease of comparison , we select the robust and natural models that transfer with the highest test accuracy across all datasets ( finetuning three convolutional blocks and the robust model using the ‖δ‖2 ≤ 3 constraint ) , as shown in Figures 2 and 3 . See Appendix A.5 for additional results . Figure 2 ( b ) shows that the test accuracy delta between robust and natural models is above zero for all six target datasets . Thus , robust models obtain higher test accuracy on the target dataset than the natural model , especially with less training data in the target domain . Robust models also learn faster , as shown by the positive test accuracy delta in Figure 3 ( b ) for all target datasets after only 11 and 21 fine-tuning epochs . See Appendix A.6 for additional information on different random subset sizes . Fine-tuning cost is the same for both robust and natural models , but training the source model is considerably more expensive . For more detail on computational complexity see A.8 . Also , our code is available at https : //github.com/utrerf/robust transfer learning.git Best results achieved with ` 2 constraint and fine-tuning one to three convolutional blocks . Robust models achieve the highest test accuracy on the target datasets when an optimal number of convolutional blocks are fine-tuned , and when these models are trained with an appropriate constraint type . In particular , fine-tuning zero ( only the fully-connected layer ) or nine convolutional blocks leads to lower test accuracy than fine-tuning one or three blocks , as shown in Figure 4 ( a ) for all six target datasets . The natural model and the other two robust models exhibit the same behavior , as shown in Appendix A.7 . To analyze the best constraint type , we select the fine-tuning configuration that yields the highest test accuracy on the target datasets ( fine-tuning three convolutional blocks ) . We see that the ` 2 constraint outperforms the ` ∞ constraint , as shown by the positive accuracy delta between the ` 2 and ` ∞ models in Figures 5 ( d ) and ( e ) , respectively . Similarity effect on transfer learning configurations . Besides noticing that robust models achieved better performance on the target dataset than natural models , we also observe trends in how well they transfer to different datasets . When transferring from ImageNet , we find that CIFAR-10 and CIFAR-100 have interesting transfer properties , compared to the other datasets . In particular , even though all other datasets transfer better when fine-tuning one or three blocks , it seems that models transfer better to CIFAR-10 and CIFAR-100 when fewer blocks are fine-tuned , as shown in Figure 4 ( b ) . This suggests that because these datasets are close to ImageNet , fine-tuning of early blocks is unnecessary ( Yosinski et al. , 2014 ) . Along similar lines , it is better to use a smaller for CIFAR-10 and CIFAR-100 datasets than the other datasets when transferring from ImageNet , as seen from Figure 5 ( c ) . This is because a larger perturbation would destroy low-level features , learned from ImageNet , which are useful to discriminate between labels in CIFAR-10 and CIFAR100 . Finally , for datasets that are most distinct from ImageNet ( SVHN and KMNIST ) , we find that robustness yields the largest benefit to classification accuracy and learning speed , as seen in Figure 2 ( b ) and Figure 3 ( b ) , respectively . These discrepancies are even more noticeable when smaller fractions of the target dataset are used .

Summary: This paper tries to investigate and understand if and how adversarial training helps the models trained on the source domain transfer easier and faster to target domains. With extensive different configurations (such as fine-tuning strategies) in experiments, the authors show that robust models transfer better than natural models with less training data from the target domain. Also they demonstrate the intuition behind through experiments, such as capturing shapes than textures or using influence functions.