Adversarially robust transfer learning

1 INTRODUCTION . Deep neural networks achieve human-like accuracy on a range of tasks when sufficient training data and computing power is available . However , when large datasets are unavailable for training , or pracitioners require a low-cost training strategy , transfer learning methods are often used . This process starts with a source network ( pre-trained on a task for which large datasets are available ) , which is then re-purposed to act on the target problem , usually with minimal re-training on a small dataset ( Yosinski et al. , 2014 ; Pan & Yang , 2009 ) . While transfer learning greatly accelerates the training pipeline and reduces data requirements in the target domain , it does not address the important issue of model robustness . It is well-known that naturally trained models often completely fail under adversarial inputs ( Biggio et al. , 2013 ; Szegedy et al. , 2013 ) . As a result , researchers and practitioners often resort to adversarial training , in which adversarial examples are crafted on-the-fly during network training and injected into the training set . This process greatly exacerbates the problems that transfer learning seeks to avoid . The high cost of creating adversarial examples increases training time ( often by an order of magnitude or more ) . Furthermore , robustness is known to suffer when training on a small dataset ( Schmidt et al. , 2018 ) . To make things worse , high-capacity models are often needed to achieve good robustness ( Madry et al. , 2017 ; Kurakin et al. , 2016 ; Shafahi et al. , 2019b ) , but these models may over-fit badly on small datasets . CONTRIBUTIONS The purpose of this paper is to study the adversarial robustness of models produced by transfer learning . We begin by observing that robust networks contain robust feature extractors , which are resistant to adversarial perturbations in different domains . Such robust features can be used ∗equal contribution †University of Maryland ‡Cornell University as a basis for semi-supervised transfer learning , which only requires re-training the last layer of a network . To demonstrate the power of robust transfer learning , we transfer a robust ImageNet source model onto the CIFAR domain , achieving both high accuracy and robustness in the new domain without adversarial training . We use visualization methods to explore properties of robust feature extractors . Then , we consider the case of transfer of learning by “ fine-tuning. ” In this case , the source network is re-trained end-to-end using a small number of epochs on the target domain . Unfortunately , this end-to-end process does not always retain the robustness of the source domain ; the network “ forgets ” the robust feature representations learned on the source task . To address this problem , we use recently proposed lifelong learning methods that prevent the network from forgetting the robustness it once learned . Using our proposed methods , we construct robust models that generalize well . In particular , we improve the generalization of a robust CIFAR-100 model by roughly 2 % while preserving its robustness . 2 BACKGROUND . Adversarial examples fall within the category of evasion attacks—test-time attacks in which a perturbation is added to a natural image before inference . Adversarial attacks are most often crafted using a differentiable loss function that measures the performance of a classifier on a chosen image . In the case of norm-constrained attacks ( which form the basis of most standard benchmark problems ) , the adversary solves max δ l ( x+ δ , y , θ ) s.t . ‖δ‖p ≤ , ( 1 ) where θ are the ( already trained and frozen ) parameters of classifier c ( x , θ ) → ŷ that maps an image to a class , l is the proxy loss used for classification ( often cross-entropy ) , δ is the image perturbation , ( x , y ) is the natural image and its true class , and ||.||p is some ` p-norm1 . The optimization problem in Eq . 1 aims to find a bounded perturbation that maximizes the cross-entropy loss given the correct label . There are many variants of this process , including DeepFool ( Moosavi-Dezfooli et al. , 2016 ) , L-BFGS ( Szegedy et al. , 2013 ) , and CW ( Carlini & Wagner , 2017 ) . Many researchers have studied methods for building a robust network which have been later shown to be ineffective when attacked with stronger adversaries ( Athalye et al. , 2018 ) . Adversarial training ( Szegedy et al. , 2013 ) is one of the defenses that was not broken by Athalye et al . ( 2018 ) . While adversarial training using a weak adversary such as the FGSM attack ( Goodfellow et al. , 2015 ) can be broken even by single step attacks which add a simple random step prior to the FGSM step ( Tramèr et al. , 2017 ) , adversarial training using a strong attack has successfully improved robustness . Madry et al . ( 2017 ) showed that a PGD attack ( which is a BIM attack ( Kurakin et al. , 2016 ) with an initial random step and projection ) is a strong enough attack to achieve promising adversarial training results . We will refer to this training method as PGD adversarial training . PGD adversarial training achieves good robustness on bounded attacks for MNIST ( LeCun et al. , 1998 ) and acceptable robustness on CIFAR-10 ( Krizhevsky & Hinton , 2009 ) classifiers . Tsipras et al . ( 2018 ) show that adversarial training with strong PGD adversaries has many benefits in addition to robustness . They also state that while adversarial training may improve generalization in regimes where training data is limited ( especially on MNIST ) , it may be at odds with generalization in regimes where data is available . This trade-off was also recently studied by Zhang et al . ( 2019 ) , Su et al . ( 2018 ) , and Shafahi et al . ( 2019a ) . While , to the best of our knowlegde , the transferability of robustness has not been studied in depth , Hendrycks et al . ( 2019 ) studied the case of adversarially training models that were pre-trained on different domains . Our work is fundamentally different in that we seek to transfer robustness without resorting to costly and data-hungry adversarial training . We train the target model on natural examples only , which allows us to directly study how well robustness transfers . Additionally , this allows us to have better generalization and achieve higher accuracy on validation examples . While as Hendrycks et al . ( 2019 ) state , fine-tuning on adversarial examples built for the target domain can improve robustness of relatively large datasets such as CIFAR-10 and CIFAR-100 compared to adversarial training from scratch on the target domain , we show that in the regimes of limited data ( where transfer learning is more common ) , adversarially robust transfer learning can lead to better results measured in terms of both robustness and clean validation accuracy . 1By default we will use the ` ∞-norm in this paper . 3 THE ROBUSTNESS OF DEEP FEATURES . In this section , we explore the robustness of different network layers , and demonstrate that robust networks rely on robust deep features . To do so , we start from robust classifiers ( c ( θr ) ) for the CIFAR-100 and CIFAR-10 datasets ( Krizhevsky & Hinton , 2009 ) , and update θ by training on natural examples . In each experiment , we re-initialize the last k layers/blocks of the network , and re-train just those layers . We start by re-initializing just the last layer , then the last two , and so on until we re-initialize all the layers . We use the adversarially trained Wide-ResNet 32-10 ( Zagoruyko & Komodakis , 2016 ) for CIFAR10 from Madry et al . ( 2017 ) as our robust model for CIFAR-10 . We also adversarially train our own robust classifier for CIFAR-100 using the code from Madry et al . ( 2017 ) . To keep things consistent , we use the same hyper-parameters used by Madry et al . ( 2017 ) for adversarially training CIFAR-10 to adversarially train the CIFAR-100 model.2 The performance of the CIFAR-10 and CIFAR-100 models on natural and adversarial examples are summarized in Table 1 . To measure robustness , we evaluate the models on adversarial examples built using PGD attacks . We break the WRN 32-10 model into 17 blocks , which are depicted in Fig . 2 . In each experiment , we first re-initialize the k deepest blocks ( blocks 1 through k ) and then train the parameters of those blocks on natural images3 . We train for 20,000 iterations using Momentum SGD and a learning rate of 0.001 . We then incrementally unfreeze and train more blocks . For each experiment , we evaluate the newly trained model ’ s accuracy on validation adversarial examples built with a 20-step PGD ` ∞ attack with = 8 . Fig . 1 shows that robustness does not drop if only the final layers of the networks are re-trained on natural examples . In fact , there is a slight increase in robustness compared to the baseline PGD7 adversarially trained models when we just retrain the last batch-normalization block and fully connected block . As we unfreeze and train more blocks , the network ’ s robustness suddenly drops . This leads us to believe that a hardened network ’ s robustness is mainly due to robust deep feature representations and robustness is preserved if we re-train on top of deep features . Now that we have identified feature extractors as a source of robustness , it is natural to investigate whether robustness is preserved when transfer learning using robust feature extractors . We will 2We adv . train the WRN 32-10 on CIFAR-100 using a 7-step ` ∞ PGD attack with step-size=2 and = 8 . We train for 80,000 iterations with a batch-size of 128 . 3In this experiment , we use standard data augmentation techniques . study two different approaches for transferring robustness across datasets : one in which only the last layer is re-trained , and one with end-to-end re-training . 4 TRANSFER LEARNING : RECYCLING FEATURE EXTRACTORS . We study how robustness transfers when the feature extractor layers of the source network are frozen , and we retrain only the last fully connected layer ( i.e . the classification layer ) for the new task . Formally , the transfer learning objective is : min w l ( z ( x , θ∗ ) , y , w ) ( 2 ) where z is the deep feature extractor function with pre-trained and now “ frozen ” parameters θ∗ , and w represents the trainable parameters of the last fully connected layer . To investigate how well robustness transfers , we use two source models : one that is hardened by adversarial training and another that is naturally trained . We use models trained on CIFAR-100 as source models and perform transfer learning from CIFAR100 to CIFAR-10 . The results are summarized in Table 2 . Compared to adversarial/natural training the target model , transferring from a source model seems to result in a drop in natural accuracy ( compare first row of Table 1 to the first row of Table 2 ) . This difference is wider when the source and target data distributions are dissimilar ( Yosinski et al. , 2014 ) . To evaluate our method on two datasets with more similar attributes , we randomly partition CIFAR100 into two disjoint subsets where each subset contains images corresponding to 50 classes . Table 2 shows the accuracy of transferring from one of the disjoint sets to the other ( second row ) and to the same set ( third row ) . We can compare results of transfer learning with adversarial training on CIFAR-100 by averaging the results in the second and third rows of Table 2 to get the accuracy across all 100 classes of CIFAR-100.4 By doing so , we see that the accuracy of the transferred classifier matches that of the adversarially trained one , even though no adversarial training took place in the target domain . For completeness , we have also included experiments where we use CIFAR-10 as the source and CIFAR-100 as the target domain . We make the following observations from the transfer-learning results in Table 2 . 1 ) robustness transfers : when the source model used for transfer learning is robust , the target model is also robust ( although less so than the source ) , 2 ) robustness transfers between models that are more similar : If 4The robust CIFAR-100 classifier has 59.87 % validation accuracy and 22.76 % accuracy on PGD-20 adversarial examples . The average validation accuracy of the two half-CIFAR-100 classifiers on validation examples is 64.96 % +58.48 % 2 = 61.72 % while the average robustness is 25.16 % +15.86 % 2 = 20.51 % . the source and target models are trained on datasets which have similar distributions ( and number of classes ) , robustness transfers better , and 3 ) validation accuracy is worst if we use a robust model as the source compared to using a conventionally trained source model : if the source model is naturally trained , the natural validation accuracy is better , although the target model is then vulnerable to adversarial perturbations .

Summary: This paper addresses the problem of performing robust transfer learning. A first contribution of the paper is to robust and classic training with respect to usual validation accuracy and robustness to adversarial attacks on the CIFAR task. Then, the same comparison is made on a transfer learning task. The transfer learning setting is then completed by studying transfer from ImageNet-based models with a particular attention to low-data regime and training deeper networks on top of the feature extractor. An analysis of robust features is provided and finally the authors studies the interest of  Learning without Forgetting strategies to provide robust transfer. The tendency s to obtain the Best performance from robust-trained source models having a good validation accuracy.