Learning Transferable Adversarial Perturbations

1 Introduction . In recent years , deep neural networks ( DNNs ) have achieved great success in a wide range of applications [ 1 , 2 , 3 , 4 ] . However , DNNs have been demonstrated to be vulnerable to adversarial examples [ 5 ] crafted by adding imperceptible perturbations to clean images . In particular , two broad categories of attacks have been studied . The first one consists of iterative algorithms [ 5 , 6 , 7 ] , which optimize the perturbation for each instance , and thus tend to be computationally expensive . The second one encompasses generative methods [ 8 , 9 , 10 ] , which train a deep network to produce perturbations . As such , attacking the target network only involves a forward pass through the generator , typically resulting in much faster attacks than iterative methods . However , speed is not the only important factor to assess the strength of an attacker ; its ability to generalize to different situations is also key to its success . In this paper , we therefore study the transferability of perturbations obtained with generative methods . Specifically , we investigate the transfer of such perturbations when the conditions at inference time differ from the training ones in terms of ( i ) target architecture , e.g. , the generator was trained to attack a VGG-16 but the target network is a ResNet152 ; ( ii ) target data , e.g. , the generator was trained using the Paintings dataset but the test data comes from ImageNet ; ( iii ) target task , e.g. , the generator was trained to attack an image recognition model but faces an object detector at test time . To the best of our knowledge , for generative methods , our work constitutes the first attempt at transferability across tasks , and only [ 11 ] has studied generalization across architectures and data , by introducing a loss function acting on the relative class probabilities of the attacked and unattacked examples . Here , by contrast , we improve the transferability of a perturbation generator across architectures , data , and tasks by exploiting the mid-level features of DNNs . The key motivation behind our approach is our observation that the mid-level features extracted by DNNs with different architectures , different data , or for different tasks bear strong similarities . This is illustrated in Figure 1 , where we visualize the features extracted by different backbones and for different tasks , and thus different datasets , using 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . the method of [ 12 ] . Our analysis suggests that adversarial perturbations that significantly affect the mid-level features of one sample in one architecture also affect them in a different architecture , even for a different task and with different data . We therefore propose to train a perturbation generator by maximizing the distance between the normal features of a sample and their adversarial counterparts extracted in the intermediate layers of a pretrained classifier . The resulting perturbations are then transferable across architectures , datasets and tasks because they similarly affect the mid-level features of the corresponding filters in the target setup . The perturbed mid-level features are then propagated to the top layers of the network , and , as a result , lead to incorrect predictions . Contributions . Our contributions can be summarized as follows : 1 . We identify the intermediate features of CNNs as common ground across different architectures , different data distributions and different tasks . 2 . We introduce an approach that exploits such features to learn an effective , transferable perturbation generator . 3 . We systematically investigate the effect of target architecture and target data distribution on the transferability of adversarial attacks . Our experiments demonstrate that our approach yields higher fooling rates than the state-of-the-art universal [ 8 ] and transferable [ 11 ] attacks . Our code is available at https : //github.com/krishnakanthnakka/Transferable_ Perturbations . 2 Related Work . Below , we review the recent works on adversarial attacks , with a focus on generator-based approaches . Adversarial Attacks . Adversarial attacks were first investigated in [ 5 ] to identify the vulnerability of modern deep networks to imperceptible perturbations in the context of image classification . Since then , several attack strategies have been studied , including single-step fast gradient descent [ 6 , 13 ] , and computationally more expensive optimization-based attacks , such as CW [ 14 ] , JSMA [ 15 ] , and others [ 16 , 17 , 7 ] . While the above methods are image-dependent , the existence of Universal Adversarial Perturbation ( UAP ) was first shown in [ 18 ] , considering the task of learning a single perturbation that can fool a classifier independently of the input image . Such a UAP was iteratively updated based on its effectiveness to move the individual data samples across the decision boundary . Other UAPs based on the computation of the singular vectors of the Jacobian matrices of feature maps have been studied in [ 19 ] . In parallel to UAPs , several works have shown that iterative adversarial attack strategies could be transferred across architectures [ 20 , 21 , 22 ] . In particular , the recent work of [ 23 ] proposed to improve targeted attack transferability across architectures by training auxiliary classifiers at each intermediate layer to attack a CNN ’ s feature maps . The above-mentioned methods , however , either are computationally expensive at inference time [ 16 , 7 , 15 , 14 , 21 , 23 ] , or suffer from low transferability rates [ 18 , 20 , 22 ] . Furthermore , most of the attacks aiming for transferability [ 18 , 19 , 23 ] strongly depend on the availability of data from the target domain . By contrast , we introduce an efficient generative method to produce adversarial perturbations that generalize not only across architectures , but also across training data and across tasks . Generator-based Attacks . Generative adversarial perturbations ( GAP ) were first introduced in [ 10 ] . In particular , [ 10 ] showed that a generator network can be used to craft a UAP that transform an input image to an image-dependent perturbation . Similarly , [ 24 ] introduced the advGAN generative framework to learn to produce adversarial perturbations , and further proposed to make use of distillation to perform black-box attacks . Based on the observation that the effectiveness of these two methods strongly depends on the availability of data from the target domain during training , [ 11 ] introduced the relativistic cross-entropy loss , which was shown to better generalize across datasets . Furthermore , [ 8 ] proposed to generate UAPs by transforming the generator ’ s output using a regional norm layer that enforces perturbation homogeneity . While effective , the above-mentioned works explicitly rely on the classification boundary of the attacked model , which we will show to tend to make them overfit to the source data . By contrast , we identify the mid-level features as a more robust signal shared across not only different architectures and datasets , but even across different tasks . Our experiments will showcase the superiority of exploiting this information over the losses used in previous approaches to train a transferrable perturbation generator . 3 Methodolgy . Let xi ∈ RH×W×3 be a color image of size H ×W , and yi be the associated ground-truth label . Furthermore , let f denote the task-related convolutional neural network that takes x as input and extracts , at layer l = { lj } Lj=1 , a feature map f ` ( xi ) ∈ RN ` ×D ` , which we assume to be reshaped in matrix form , such that N ` = H ` ×W ` , with H ` and W ` the spatial dimensions of the feature map , and D ` its number of channels . Our goal is to train a generator G that produces a perturbation δi ∈ RH×W×3 , which , when added to the clean image xi , results in predicting a different label from yi . To this end , we feed the input image xi to the generator to synthesize an unbounded adversarial image G ( xi ) , which is then clipped to be within an bound of xi under the ` ∞ norm . Let x̂i be the adversarial image obtained after such a clipping of G ( xi ) . In contrast to [ 10 , 11 ] , which rely on the final classification boundary of the task-related network f to train the generator with a cross-entropy-based loss , we exploit the mid-level features of f . Specifically , we maximize the L2 distance between the normal feature map f ` ( xi ) and adversarial feature map f ` ( x̂i ) at layer l using a feature separation loss term defined as Lfeat ( xi , x̂i ) = ||f ` ( xi ) − f ` ( x̂i ) ||2F , ( 1 ) where ‖ · ‖F denotes the Frobenius norm . The overall training scheme of our perturbation generator is provided in Algorithm 1 . In practice , we observe the feature separation loss of Eq . 1 to encourage the generator to learn low-level , quasiimperceptible structured patterns that affect that activations of a few filters while either suppressing the other ones or leaving them unaffected . More importantly , these perturbations are highly transferable . Reason for transferability . Given the simplicity of our formulation , a natural question that arises is what affects the transferability of perturbations among CNNs . To understand this , we analyze the internal workings of a CNN , whose filters in each layer encode different levels of information . In particular , in Figure 2 , we visualize the filters of various CNNs , sampled at different layers , using the technique of [ 12 ] , which maximizes the mean activation of the each filter . This allows us to discover common characteristics across the different architectures . On one hand , the bottom filters , close to the input image , extract color and edge information , as shown in the left block of Figure 2 . On the other , the top-level filters , close to the output layer , shown in the right block , are more focused on the object representation , and thus more task specific . By contrast , the mid-level filters learn more nuanced features , such as textures , and therefore tend to display similar patterns across architectures , datasets tasks . As a consequence , and as indicated by our experiments , attacking the initial layers requires large perturbation strengths . Furthermore , while attacking the top-level features works well in the white box scenario , it does not transfer well to different data or architectures , because the features are already too architecture- and task-specific . Attacking mid-level features thus comes as a natural choice , which we will show to yield highly transferable perturbations . Let us now focus on understanding the reason for the transferability when the target data is unavailable to the attacker . In Figure 3 , we plot t-SNE visualizations of different datasets at 4 layers of DenseNet121 . The proximity of the features extracted from the Comics or Paintings datasets , containing images from a few classes particularly focused on humans , to those extracted from ImageNet suggests these datasets as good candidates to train a generator that will fool an ImageNet-trained network ( as shown in Tables 1 and 2 ) . By contrast , the ChestX dataset presents a larger domain gap . Nevertheless , we will show that , albeit for a drop in fooling rate , our use of mid-level features still make a ChestX-trained generator reasonably effective on ImageNet-based classifiers . Furthermore , in Figure 4 , we plot the top 80 activations of ImageNet , Paintings and ChestX at layer 8 of DenseNet121 . This shows that the peaks of the ImageNet dataset ( orange ) are more strongly correlated to those of the Paintings dataset ( left ) than to those of ChestX ( right ) . This is confirmed by our experiments , where a Paintings-trained generator is more effective than a ChestX-trained one . In short , while mid-level features lead to generator transferability , the resulting effectiveness remains affected by the similarity of the data . Algorithm 1 Training a transferable adversarial perturbation generator Input : xi : clean images ; f : pretrained classifer ; : perturbation ` ∞ bound Initialize generator G ; Load classifier f and freeze its parameters ; repeat Get a clean image xi and feed xi to f to obtain f ` ( xi ) ; Generate an unbounded adversarial image G ( xi ) and clip it within an bound of xi to get x̂i ; Pass the adversarial image to f to obtain f ` ( x̂i ) ; Compute the feature separation loss Lfeat ; Compute the gradient of Lfeat w.r.t . the weights of G and update these weights using Adam . until convergence return trained Generator G∗

Summary: The work studied the generalization of adversarial attacks that are induced by generative methods. Specifically, the transferability of the perturbations for different testing conditions are evaluated. The mid-level features of DNNs are used to learn a transferable perturbation generator. Evaluations of transferable attacks are performed on tasks with a different network architecture, and/or data distributions.