Adversarially robust transfer learning

1 INTRODUCTION . Deep neural networks achieve human-like accuracy on a range of tasks when sufficient training data and computing power is available . However , when large datasets are unavailable for training , or pracitioners require a low-cost training strategy , transfer learning methods are often used . This process starts with a source network ( pre-trained on a task for which large datasets are available ) , which is then re-purposed to act on the target problem , usually with minimal re-training on a small dataset ( Yosinski et al. , 2014 ; Pan & Yang , 2009 ) . While transfer learning greatly accelerates the training pipeline and reduces data requirements in the target domain , it does not address the important issue of model robustness . It is well-known that naturally trained models often completely fail under adversarial inputs ( Biggio et al. , 2013 ; Szegedy et al. , 2013 ) . As a result , researchers and practitioners often resort to adversarial training , in which adversarial examples are crafted on-the-fly during network training and injected into the training set . This process greatly exacerbates the problems that transfer learning seeks to avoid . The high cost of creating adversarial examples increases training time ( often by an order of magnitude or more ) . Furthermore , robustness is known to suffer when training on a small dataset ( Schmidt et al. , 2018 ) . To make things worse , high-capacity models are often needed to achieve good robustness ( Madry et al. , 2017 ; Kurakin et al. , 2016 ; Shafahi et al. , 2019b ) , but these models may over-fit badly on small datasets . CONTRIBUTIONS The purpose of this paper is to study the adversarial robustness of models produced by transfer learning . We begin by observing that robust networks contain robust feature extractors , which are resistant to adversarial perturbations in different domains . Such robust features can be used ∗equal contribution †University of Maryland ‡Cornell University as a basis for semi-supervised transfer learning , which only requires re-training the last layer of a network . To demonstrate the power of robust transfer learning , we transfer a robust ImageNet source model onto the CIFAR domain , achieving both high accuracy and robustness in the new domain without adversarial training . We use visualization methods to explore properties of robust feature extractors . Then , we consider the case of transfer of learning by “ fine-tuning. ” In this case , the source network is re-trained end-to-end using a small number of epochs on the target domain . Unfortunately , this end-to-end process does not always retain the robustness of the source domain ; the network “ forgets ” the robust feature representations learned on the source task . To address this problem , we use recently proposed lifelong learning methods that prevent the network from forgetting the robustness it once learned . Using our proposed methods , we construct robust models that generalize well . In particular , we improve the generalization of a robust CIFAR-100 model by roughly 2 % while preserving its robustness . 2 BACKGROUND . Adversarial examples fall within the category of evasion attacks—test-time attacks in which a perturbation is added to a natural image before inference . Adversarial attacks are most often crafted using a differentiable loss function that measures the performance of a classifier on a chosen image . In the case of norm-constrained attacks ( which form the basis of most standard benchmark problems ) , the adversary solves max δ l ( x+ δ , y , θ ) s.t . ‖δ‖p ≤ , ( 1 ) where θ are the ( already trained and frozen ) parameters of classifier c ( x , θ ) → ŷ that maps an image to a class , l is the proxy loss used for classification ( often cross-entropy ) , δ is the image perturbation , ( x , y ) is the natural image and its true class , and ||.||p is some ` p-norm1 . The optimization problem in Eq . 1 aims to find a bounded perturbation that maximizes the cross-entropy loss given the correct label . There are many variants of this process , including DeepFool ( Moosavi-Dezfooli et al. , 2016 ) , L-BFGS ( Szegedy et al. , 2013 ) , and CW ( Carlini & Wagner , 2017 ) . Many researchers have studied methods for building a robust network which have been later shown to be ineffective when attacked with stronger adversaries ( Athalye et al. , 2018 ) . Adversarial training ( Szegedy et al. , 2013 ) is one of the defenses that was not broken by Athalye et al . ( 2018 ) . While adversarial training using a weak adversary such as the FGSM attack ( Goodfellow et al. , 2015 ) can be broken even by single step attacks which add a simple random step prior to the FGSM step ( Tramèr et al. , 2017 ) , adversarial training using a strong attack has successfully improved robustness . Madry et al . ( 2017 ) showed that a PGD attack ( which is a BIM attack ( Kurakin et al. , 2016 ) with an initial random step and projection ) is a strong enough attack to achieve promising adversarial training results . We will refer to this training method as PGD adversarial training . PGD adversarial training achieves good robustness on bounded attacks for MNIST ( LeCun et al. , 1998 ) and acceptable robustness on CIFAR-10 ( Krizhevsky & Hinton , 2009 ) classifiers . Tsipras et al . ( 2018 ) show that adversarial training with strong PGD adversaries has many benefits in addition to robustness . They also state that while adversarial training may improve generalization in regimes where training data is limited ( especially on MNIST ) , it may be at odds with generalization in regimes where data is available . This trade-off was also recently studied by Zhang et al . ( 2019 ) , Su et al . ( 2018 ) , and Shafahi et al . ( 2019a ) . While , to the best of our knowlegde , the transferability of robustness has not been studied in depth , Hendrycks et al . ( 2019 ) studied the case of adversarially training models that were pre-trained on different domains . Our work is fundamentally different in that we seek to transfer robustness without resorting to costly and data-hungry adversarial training . We train the target model on natural examples only , which allows us to directly study how well robustness transfers . Additionally , this allows us to have better generalization and achieve higher accuracy on validation examples . While as Hendrycks et al . ( 2019 ) state , fine-tuning on adversarial examples built for the target domain can improve robustness of relatively large datasets such as CIFAR-10 and CIFAR-100 compared to adversarial training from scratch on the target domain , we show that in the regimes of limited data ( where transfer learning is more common ) , adversarially robust transfer learning can lead to better results measured in terms of both robustness and clean validation accuracy . 1By default we will use the ` ∞-norm in this paper . 3 THE ROBUSTNESS OF DEEP FEATURES . In this section , we explore the robustness of different network layers , and demonstrate that robust networks rely on robust deep features . To do so , we start from robust classifiers ( c ( θr ) ) for the CIFAR-100 and CIFAR-10 datasets ( Krizhevsky & Hinton , 2009 ) , and update θ by training on natural examples . In each experiment , we re-initialize the last k layers/blocks of the network , and re-train just those layers . We start by re-initializing just the last layer , then the last two , and so on until we re-initialize all the layers . We use the adversarially trained Wide-ResNet 32-10 ( Zagoruyko & Komodakis , 2016 ) for CIFAR10 from Madry et al . ( 2017 ) as our robust model for CIFAR-10 . We also adversarially train our own robust classifier for CIFAR-100 using the code from Madry et al . ( 2017 ) . To keep things consistent , we use the same hyper-parameters used by Madry et al . ( 2017 ) for adversarially training CIFAR-10 to adversarially train the CIFAR-100 model.2 The performance of the CIFAR-10 and CIFAR-100 models on natural and adversarial examples are summarized in Table 1 . To measure robustness , we evaluate the models on adversarial examples built using PGD attacks . We break the WRN 32-10 model into 17 blocks , which are depicted in Fig . 2 . In each experiment , we first re-initialize the k deepest blocks ( blocks 1 through k ) and then train the parameters of those blocks on natural images3 . We train for 20,000 iterations using Momentum SGD and a learning rate of 0.001 . We then incrementally unfreeze and train more blocks . For each experiment , we evaluate the newly trained model ’ s accuracy on validation adversarial examples built with a 20-step PGD ` ∞ attack with = 8 . Fig . 1 shows that robustness does not drop if only the final layers of the networks are re-trained on natural examples . In fact , there is a slight increase in robustness compared to the baseline PGD7 adversarially trained models when we just retrain the last batch-normalization block and fully connected block . As we unfreeze and train more blocks , the network ’ s robustness suddenly drops . This leads us to believe that a hardened network ’ s robustness is mainly due to robust deep feature representations and robustness is preserved if we re-train on top of deep features . Now that we have identified feature extractors as a source of robustness , it is natural to investigate whether robustness is preserved when transfer learning using robust feature extractors . We will 2We adv . train the WRN 32-10 on CIFAR-100 using a 7-step ` ∞ PGD attack with step-size=2 and = 8 . We train for 80,000 iterations with a batch-size of 128 . 3In this experiment , we use standard data augmentation techniques . study two different approaches for transferring robustness across datasets : one in which only the last layer is re-trained , and one with end-to-end re-training . 4 TRANSFER LEARNING : RECYCLING FEATURE EXTRACTORS . We study how robustness transfers when the feature extractor layers of the source network are frozen , and we retrain only the last fully connected layer ( i.e . the classification layer ) for the new task . Formally , the transfer learning objective is : min w l ( z ( x , θ∗ ) , y , w ) ( 2 ) where z is the deep feature extractor function with pre-trained and now “ frozen ” parameters θ∗ , and w represents the trainable parameters of the last fully connected layer . To investigate how well robustness transfers , we use two source models : one that is hardened by adversarial training and another that is naturally trained . We use models trained on CIFAR-100 as source models and perform transfer learning from CIFAR100 to CIFAR-10 . The results are summarized in Table 2 . Compared to adversarial/natural training the target model , transferring from a source model seems to result in a drop in natural accuracy ( compare first row of Table 1 to the first row of Table 2 ) . This difference is wider when the source and target data distributions are dissimilar ( Yosinski et al. , 2014 ) . To evaluate our method on two datasets with more similar attributes , we randomly partition CIFAR100 into two disjoint subsets where each subset contains images corresponding to 50 classes . Table 2 shows the accuracy of transferring from one of the disjoint sets to the other ( second row ) and to the same set ( third row ) . We can compare results of transfer learning with adversarial training on CIFAR-100 by averaging the results in the second and third rows of Table 2 to get the accuracy across all 100 classes of CIFAR-100.4 By doing so , we see that the accuracy of the transferred classifier matches that of the adversarially trained one , even though no adversarial training took place in the target domain . For completeness , we have also included experiments where we use CIFAR-10 as the source and CIFAR-100 as the target domain . We make the following observations from the transfer-learning results in Table 2 . 1 ) robustness transfers : when the source model used for transfer learning is robust , the target model is also robust ( although less so than the source ) , 2 ) robustness transfers between models that are more similar : If 4The robust CIFAR-100 classifier has 59.87 % validation accuracy and 22.76 % accuracy on PGD-20 adversarial examples . The average validation accuracy of the two half-CIFAR-100 classifiers on validation examples is 64.96 % +58.48 % 2 = 61.72 % while the average robustness is 25.16 % +15.86 % 2 = 20.51 % . the source and target models are trained on datasets which have similar distributions ( and number of classes ) , robustness transfers better , and 3 ) validation accuracy is worst if we use a robust model as the source compared to using a conventionally trained source model : if the source model is naturally trained , the natural validation accuracy is better , although the target model is then vulnerable to adversarial perturbations .

Summary: The paper studies transfer learning from the point of view of adversarial robustness. The goal is, given a robust deep neural network classifier for a source domain, learn a robust classifier for a target domain as efficiently and with as few samples as possible. The authors empirically evaluate different strategies and compare with relevant baselines.