A Little Robustness Goes a Long Way: Leveraging Robust Features for Targeted Transfer Attacks

1 Introduction . Neural-network image classifiers are well-known to be susceptible to adversarial examples—images that are perturbed in a way that is largely imperceptable to humans but that cause the neural network to make misclassifications . Much research has gone into methods for constructing adverarial examples in order to understand the nature of neural-network vulnerabilities and to develop methods to make neural-network classifiers more robust to attacks [ 7 , 11 , 21 , 45 , 50 , 55 , 73 ] . Untargeted adversarial examples are designed to elicit an unspecified incorrect class , while targeted adversarial examples are designed to elicit a specific ( incorrect ) target class . A given adversarial perturbation is designed via optimization with respect to a given trained network ; here we call this the source network . While untargeted adversarial examples are often transferable—examples designed to attack a source network also successfully attack other trained networks with different parameters and architectures [ 21 , 73 ] —targeted adversarial examples tend to be less transferable [ 44 ] . Prior research on constructing transferable adversarial examples for image classifiers has focused primarily on improving optimization methods for generating successful image perturbations . In this paper , we take a different tack—focusing on the source neural network used in constructing adversarial examples . Specifically , we find that “ slightly-robust ” convolutional neural networks ( CNNs ) —ones that have been trained to be robust to small adversarial perturbations—can be leveraged to substantially improve the transferability of targeted adversarial examples to different architectures . Surprisingly , we show that targeted adversarial examples constructed with respect to a slightly robust CNN transfer successfully not only to different CNN architectures but also to transformer architectures such as ViT [ 16 ] , LeViT [ 22 ] , CCT [ 24 ] , and even CLIP [ 57 ] , which is trained on different objective than the source CNN . Such transfers are all “ black box ” attacks—while generating an adversarial example 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . requires knowledge of the source network ’ s architecture , no such knowledge is required for the black-box architectures that can also be attacked by the same example . The vulnerability of networks to both targeted and untargeted attacks has huge significance for security purposes , and thus understanding how to construct highly effective attacks can help motivate defenses . However , we believe that targeted attacks are especially important for understanding neural-network classifiers , as they provide a tool to compare the features of two models . When a targeted attack transfers from one network to another , it suggests that the two networks rely on similar information for classification , and that they use the information in the same way . In our work , we show that each individual slightly-robust neural network transfers features effectively to all tested non-robust networks , suggesting the surprising result that slightly-robust networks rely on features that overlap with every non-robust network , even though it is not the case that any particular non-robust network has features that substantially overlap with all other non-robust networks . In addition , we leverage the techniques of adversarial transferability to examine which features are learned by neural networks . In accordance with prior work [ 71 ] , we find that , on the spectrum from non-robust ( standard ) to highly robust classifiers , those that are only slightly robust exhibit the most transferable representation-targeted adversarial examples , suggesting that the features of slightly-robust networks overlap substantially with every tested desination network . This can explain why slightly robust networks give rise to more transferable adversarial attacks and have better weight initializations for downstream transfer-learning tasks [ 43 , 62 , 76 , 80 ] . The main contributions of this paper are the following : 1 . We demonstrate that adversarial examples generated with respect to slightly robust CNNs are more transferable than those generated with respect to standard ( non-robust ) networks , This transferability extends not only to other CNNs , but also to transformer architectures . 2 . We find that , as the robustness of the source network increases , there is also a substantial increase in transferability of targeted adversarial examples to adversarially-defended networks . 3 . We examine the role of the adversarial loss function in generating transferable adversarial examples . 4 . We show , surprisingly , that non-robust neural networks do not exhibit substantial feature ( representation ) transferability , while slightly-robust neural networks do . This helps explain why slightly-robust neural networks enable superior transferability of targeted adversarial examples . 2 Background . Adversarial Examples . In this paper we are primarily concerned with properties of source networks that facilitate transferability of adversarial examples . Let F : X → Y denote a “ white-box ” network ( i.e. , one whose architecture and weights are known to the adversary ) and letG : X → Y denote a “ black-box ” network ( weights and architecture are unknown to the adversary ) . Let ( x , y ) ∈ X × Y be an ( unperturbed ) input-label pair , where X is the input-space and Y is the label-space . Given a maximum perturbation size ε , we construct an adversarial example x + δ where ‖δ‖∞ ≤ ε , such that F ( x + δ ) 6= y for the untargeted case , and F ( x + δ ) = t for some target class t ∈ Y for the targeted case . We then say that x+ δ is transferable to black-box network G if G ( x+ δ ) 6= y for the untargeted case and G ( x+ δ ) = t for the targeted case . Optimizers . Prior research has identified a number of methods for optimizing adversarial examples given a white-box classifier F , many of which are based on the Iterative Fast Gradient Sign Method ( I-FGSM ) [ 21 , 40 ] , in which a perturbation δi is iteratively updated to maximize the loss of the network while obeying an ` ∞ norm constraint . We adopt the state-of-the-art method recently proposed by [ 91 ] , which combines three variants of I-FGSM and optimizes over many steps : 1 . Diverse Input Iterative Fast Gradient Sign Method ( DI2-FGSM ) , which applies a random affine transformation to the input at each step prior to computing the gradient [ 87 ] , 2 . Translation-Invariant Iterative Fast Gradient Sign Method ( TI-FGSM ) , which convolves the gradient with a Gaussian filter [ 14 ] , 3 . Momentum Iterative Fast Gradient Sign Method ( MI-FGSM ) , in which a momentum term is added to the gradient [ 13 ] . We follow the convention of Zhao et al . [ 91 ] and call the combination of these processes TMDI-FGSM . We describe the method in detail in the appendix . For targeted adversarial examples , the loss function L should be maximized when the target label is predicted with high confidence . For untargeted adversarial examples , this occurs when the predicted label differs from the true label , and when the true label is given a low confidence . A number of adversarial loss functions have been proposed , including standard cross-entropy loss [ 73 ] , CW loss [ 7 ] , and feature-disruption loss [ 34 ] . We use the highly-effective logit loss , proposed by [ 91 ] , which is maximized for targeted examples when the logit score for a target class ( i.e. , the value of the output neuron associated with the target class prior to the softmax operation ) is maximized . Similarly , the untargeted version aims to minimize the logit score associated with the true class . Constructing Robust Source Networks . We construct robust source networks by performing adversarial training with projected gradient descent [ 21 , 45 ] . Each source network is trained to be robust to adversarial examples with ` 2 norm less than a specified ε parameter , which we call the robustness parameter . For this paper , we rely on pre-trained robust ImageNet models [ 62 ] , which have been released under the MIT License . These source networks , along with many of our experiments , are implemented in PyTorch [ 56 ] . Features . In this paper , we will refer to neural network features [ 32 , 71 , 78 ] . A feature f : X → R maps input to a real number to describe how “ strongly ” the feature appears in the image . Every neuron in a neural network computes a feature . However , we are primarily concerned with the representation-layer features , i.e. , the features computed by the neurons in the penultimate layer [ 32 ] . When we are referring to the features of a specific neural network , we are referring to the features described by the neurons in the representation layer . When we say that the features of two different neural networks overlap , we mean that the patterns of the input that affect the features of one neural network also affect the features of the other neural network . When features are easily manipulated by small perturbations to the input , they are said to be non-robust ; likewise when they are not easily manipulated in this way , they are said to be robust [ 32 ] . Robust networks , i.e. , networks that are less vulnerable to adversarial perturbations , should rely primarily on robust features , though non-robust networks can rely on a mixture of non-robust and robust features [ 32 , 71 ] . 3 Transferability of Adversarial Examples . In this section , we describe the methodology and results of our experiments on the transferability of adversarial examples as a function of the robustness of the source network . We evaluate the targeted and untargeted effectiveness of each constructed adversarial example on a collection of convolutional classifiers , Xception [ 9 ] , VGG [ 68 ] , ResNet [ 25 , 26 ] , Inception [ 74 ] , MobileNet [ 29 ] , DenseNet [ 30 ] , NASNetLarge [ 93 ] , and EfficientNet [ 75 ] , as well as transformer-based classifiers , ViT [ 16 ] , LeViT [ 22 ] , CCT [ 24 ] , and CLIP [ 57 ] . Here we use the term destination network to denote the networks on which we will evaluate transferability of adversarial examples that were generated with respect to a source network . For our ImageNet experiments , we rely on pre-trained models [ 10 , 62 ] . Generating Adversarial Examples . We choose 1000 images randomly from the ImageNet validation dataset such that every image has a different class . We generate target classes randomly for each image such that each class is a target for exactly one image , and no image has a target that is the same as its true class . For each classifier ( of different robustness ) , we generate targeted adversarial examples for each of the 1000 selected images , targeting the associated target class . To optimize each adversarial example , we run the TMDI-FGSM algorithm for 300 iterations . When the exact image input dimensions differs between the source and destination network , we rescale the adversarial example to fit the dimensions required by the destination network using bilinear interpolation . To generate adversarial examples , we use the Robustness library [ 17 ] . Transferability to Convolutional Network Classifiers . We find that across every destination convolutional network , using adversarial examples optimized with respect to a source network with small robustness parameter improves transfer success rate in both the targeted and untargeted setting compared to the success rate of the non-robust ( ε = 0 ) network ( Figure 1 ) . Our results on untargeted adversarial examples can be found in the Appendix . The success peak is approximately the same across every convolutional destination network ( ε = 0.1 ) , suggesting that there is an optimal sourcenetwork robustness parameter in order to maximize transferability to convolutional architectures trained on ImageNet . Note that our attacks do not exclude images that are already misclassified ( untargeted case ) , and misclassified as the target class ( targeted case ) prior to adversarial perturbation . In Figure 1 , we plot the baseline performance of the attacks , i.e. , the performance of the attacks for unperturbed images . Interestingly , we observe a substantial drop in attack success as robustness increases past the optimal value . While not shown in Figure 1 , this trend continues as ε increases above 1 ( see Appendix ) . We hypothesize that as the robustness of a source network increases beyond a certain point , the network begins to entirely ignore many of the non-robust features relied upon by the ( non-robust ) destination networks , and thus attacks do not modify these features , reducing attack success . This is consistent with prior literature [ 78 ] . We dedicate the second half of this paper to describing a possible mechanism to explain these results . Transformer-Based Classifiers . Few studies have addressed the robustness of transformer-based classifiers to transfer attacks [ 66 ] . To our knowledge , our paper is the first to address targeted transfer attacks against transformer architectures . Shao et al . [ 66 ] finds that transformer-based image classifiers are more robust to transfer attacks than convolutional classifiers , especially when the source model is convolutional . We find an even more striking result : with only minimal attack performance against transformer networks , previously published methods that use non-robust source networks are almost entirely ineffective at constructing targeted transferable adversarial examples using convolutional source models ( Table 1 ) . This suggests that the features learned by transformer-based models and non-robust convolutional models are largely different . However , we find that using a slightly-robust ResNet50 classifier as a source network dramatically improves the transferability of targeted adversarial examples to transformer-based classifiers . The optimal robustness parameter for the source network is different for destination transformer networks and destination convolutional networks , though we find that any amount of robustness in the source model ( below a critical value ) substantially improves transferability . Thus , in a real-world black-box attack setting—in which the destination network ’ s architecture is entirely unknown—an adversary could find a balance between a source network robustness parameter that optimizes transfer performance for CNN classifiers and for transformer-based classifiers . CLIP . Radford et al . [ 57 ] describes CLIP , a transformer-based classifier based on the ViT architecture . CLIP is trained to simultaneously encode images and short textual descriptions of the images . CLIP can be used for highly effective zero-shot classification by determining which class label , encoded as text , has an encoding most similar to that of the input image . Despite the fact that CLIP has not been explicitly trained with ImageNet labels or to optimize for the classification task , we find that the transfer performance of targeted adversarial examples is improved substantially when the source network ( ResNet50 ) is slightly robust ( Table 1 , rightmost column ) , again suggesting that slightly-robust neural networks rely on features which overlap with non-robust networks , even when the non-robust networks differ in architecture and training algorithm . For our experiments , we use ViT-B/32 , the CLIP architecture based on ViT-B/32 , CCT-14t/7x2 , and LeViT-256 . Improvements Upon Existing Attacks . We directly compare the targeted transfer attack success rate to previous state-of-the-art black-box attacks and find that our method substantially outperforms the previous methods under similar constraints ( Table 1 ) . In particular , we evaluate our method ’ s attack performace with three different loss functions : standard cross-entropy loss ( Xent ) , Poincaré distance with a triplet loss term ( Po+Trip ) [ 41 ] , and logit loss . We include a comparison with the feature distribution attack ( FDA ) [ 33 ] , however , FDA requires that we train multiple supplemental models for each individual target class , which would require thousands of supplemental models to attack all thousand classes of ImageNet . Thus , we do not perform a direct comparison and instead report the targeted transfer attack success rate that is reported in the original FDA paper [ 33 ] . Attacking Adversarially-Trained Models . Adversarial training has been shown to improve robustness to transfer attacks [ 45 ] . We evaluate the transferability of adversarial examples to adversarially trained destination networks . Even though the adversarial perturbations which we use to attack each adversarially-trained network are larger than the magnitude for which the destination networks are trained to be robust , the adversarial examples generated using non-robust networks do not transfer to the adversarially trained networks . However , as shown in Figure 2 , as the robustness of the source network increases , the attack success rate increases substantially . Similar to Figure 1 , Figure 2 includes the baseline rate at which the destination network classifies unperturbed images as the ( incorrect ) target class . Extending Our Methods to CIFAR-10 . We repeat many of our experiments for the CIFAR-10 dataset [ 39 ] . The results are consistent with our ImageNet results . We present and discuss these results in detail in the Appendix . Computational Limitations . Due to the computational requirement of training multiple robust ImageNet classifiers , we restrict our experiments to those we can compute using pre-trained robust networks . Thus , we test adversarial attacks only using the ResNet50 architecture as a source network , and we evaluate attacks only on destination networks which are easily available to us . Similarly , we do not compute ensemble attacks using slightly-robust source networks , although we expect this technique to improve the success of our attacks .

Summary: This work studies how adversarial robustness of a source network can affect the transfer of adversarial examples to a different target network. The main finding is that slightly robust networks are ideal source networks; training the source networks with no adversarial perturbations, or with large perturbations, is less effective. The authors analyze why this might be the case, and propose that slightly robust networks have increased universality of representations.