Learning Transferable Adversarial Perturbations

1 Introduction . In recent years , deep neural networks ( DNNs ) have achieved great success in a wide range of applications [ 1 , 2 , 3 , 4 ] . However , DNNs have been demonstrated to be vulnerable to adversarial examples [ 5 ] crafted by adding imperceptible perturbations to clean images . In particular , two broad categories of attacks have been studied . The first one consists of iterative algorithms [ 5 , 6 , 7 ] , which optimize the perturbation for each instance , and thus tend to be computationally expensive . The second one encompasses generative methods [ 8 , 9 , 10 ] , which train a deep network to produce perturbations . As such , attacking the target network only involves a forward pass through the generator , typically resulting in much faster attacks than iterative methods . However , speed is not the only important factor to assess the strength of an attacker ; its ability to generalize to different situations is also key to its success . In this paper , we therefore study the transferability of perturbations obtained with generative methods . Specifically , we investigate the transfer of such perturbations when the conditions at inference time differ from the training ones in terms of ( i ) target architecture , e.g. , the generator was trained to attack a VGG-16 but the target network is a ResNet152 ; ( ii ) target data , e.g. , the generator was trained using the Paintings dataset but the test data comes from ImageNet ; ( iii ) target task , e.g. , the generator was trained to attack an image recognition model but faces an object detector at test time . To the best of our knowledge , for generative methods , our work constitutes the first attempt at transferability across tasks , and only [ 11 ] has studied generalization across architectures and data , by introducing a loss function acting on the relative class probabilities of the attacked and unattacked examples . Here , by contrast , we improve the transferability of a perturbation generator across architectures , data , and tasks by exploiting the mid-level features of DNNs . The key motivation behind our approach is our observation that the mid-level features extracted by DNNs with different architectures , different data , or for different tasks bear strong similarities . This is illustrated in Figure 1 , where we visualize the features extracted by different backbones and for different tasks , and thus different datasets , using 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . the method of [ 12 ] . Our analysis suggests that adversarial perturbations that significantly affect the mid-level features of one sample in one architecture also affect them in a different architecture , even for a different task and with different data . We therefore propose to train a perturbation generator by maximizing the distance between the normal features of a sample and their adversarial counterparts extracted in the intermediate layers of a pretrained classifier . The resulting perturbations are then transferable across architectures , datasets and tasks because they similarly affect the mid-level features of the corresponding filters in the target setup . The perturbed mid-level features are then propagated to the top layers of the network , and , as a result , lead to incorrect predictions . Contributions . Our contributions can be summarized as follows : 1 . We identify the intermediate features of CNNs as common ground across different architectures , different data distributions and different tasks . 2 . We introduce an approach that exploits such features to learn an effective , transferable perturbation generator . 3 . We systematically investigate the effect of target architecture and target data distribution on the transferability of adversarial attacks . Our experiments demonstrate that our approach yields higher fooling rates than the state-of-the-art universal [ 8 ] and transferable [ 11 ] attacks . Our code is available at https : //github.com/krishnakanthnakka/Transferable_ Perturbations . 2 Related Work . Below , we review the recent works on adversarial attacks , with a focus on generator-based approaches . Adversarial Attacks . Adversarial attacks were first investigated in [ 5 ] to identify the vulnerability of modern deep networks to imperceptible perturbations in the context of image classification . Since then , several attack strategies have been studied , including single-step fast gradient descent [ 6 , 13 ] , and computationally more expensive optimization-based attacks , such as CW [ 14 ] , JSMA [ 15 ] , and others [ 16 , 17 , 7 ] . While the above methods are image-dependent , the existence of Universal Adversarial Perturbation ( UAP ) was first shown in [ 18 ] , considering the task of learning a single perturbation that can fool a classifier independently of the input image . Such a UAP was iteratively updated based on its effectiveness to move the individual data samples across the decision boundary . Other UAPs based on the computation of the singular vectors of the Jacobian matrices of feature maps have been studied in [ 19 ] . In parallel to UAPs , several works have shown that iterative adversarial attack strategies could be transferred across architectures [ 20 , 21 , 22 ] . In particular , the recent work of [ 23 ] proposed to improve targeted attack transferability across architectures by training auxiliary classifiers at each intermediate layer to attack a CNN ’ s feature maps . The above-mentioned methods , however , either are computationally expensive at inference time [ 16 , 7 , 15 , 14 , 21 , 23 ] , or suffer from low transferability rates [ 18 , 20 , 22 ] . Furthermore , most of the attacks aiming for transferability [ 18 , 19 , 23 ] strongly depend on the availability of data from the target domain . By contrast , we introduce an efficient generative method to produce adversarial perturbations that generalize not only across architectures , but also across training data and across tasks . Generator-based Attacks . Generative adversarial perturbations ( GAP ) were first introduced in [ 10 ] . In particular , [ 10 ] showed that a generator network can be used to craft a UAP that transform an input image to an image-dependent perturbation . Similarly , [ 24 ] introduced the advGAN generative framework to learn to produce adversarial perturbations , and further proposed to make use of distillation to perform black-box attacks . Based on the observation that the effectiveness of these two methods strongly depends on the availability of data from the target domain during training , [ 11 ] introduced the relativistic cross-entropy loss , which was shown to better generalize across datasets . Furthermore , [ 8 ] proposed to generate UAPs by transforming the generator ’ s output using a regional norm layer that enforces perturbation homogeneity . While effective , the above-mentioned works explicitly rely on the classification boundary of the attacked model , which we will show to tend to make them overfit to the source data . By contrast , we identify the mid-level features as a more robust signal shared across not only different architectures and datasets , but even across different tasks . Our experiments will showcase the superiority of exploiting this information over the losses used in previous approaches to train a transferrable perturbation generator . 3 Methodolgy . Let xi ∈ RH×W×3 be a color image of size H ×W , and yi be the associated ground-truth label . Furthermore , let f denote the task-related convolutional neural network that takes x as input and extracts , at layer l = { lj } Lj=1 , a feature map f ` ( xi ) ∈ RN ` ×D ` , which we assume to be reshaped in matrix form , such that N ` = H ` ×W ` , with H ` and W ` the spatial dimensions of the feature map , and D ` its number of channels . Our goal is to train a generator G that produces a perturbation δi ∈ RH×W×3 , which , when added to the clean image xi , results in predicting a different label from yi . To this end , we feed the input image xi to the generator to synthesize an unbounded adversarial image G ( xi ) , which is then clipped to be within an bound of xi under the ` ∞ norm . Let x̂i be the adversarial image obtained after such a clipping of G ( xi ) . In contrast to [ 10 , 11 ] , which rely on the final classification boundary of the task-related network f to train the generator with a cross-entropy-based loss , we exploit the mid-level features of f . Specifically , we maximize the L2 distance between the normal feature map f ` ( xi ) and adversarial feature map f ` ( x̂i ) at layer l using a feature separation loss term defined as Lfeat ( xi , x̂i ) = ||f ` ( xi ) − f ` ( x̂i ) ||2F , ( 1 ) where ‖ · ‖F denotes the Frobenius norm . The overall training scheme of our perturbation generator is provided in Algorithm 1 . In practice , we observe the feature separation loss of Eq . 1 to encourage the generator to learn low-level , quasiimperceptible structured patterns that affect that activations of a few filters while either suppressing the other ones or leaving them unaffected . More importantly , these perturbations are highly transferable . Reason for transferability . Given the simplicity of our formulation , a natural question that arises is what affects the transferability of perturbations among CNNs . To understand this , we analyze the internal workings of a CNN , whose filters in each layer encode different levels of information . In particular , in Figure 2 , we visualize the filters of various CNNs , sampled at different layers , using the technique of [ 12 ] , which maximizes the mean activation of the each filter . This allows us to discover common characteristics across the different architectures . On one hand , the bottom filters , close to the input image , extract color and edge information , as shown in the left block of Figure 2 . On the other , the top-level filters , close to the output layer , shown in the right block , are more focused on the object representation , and thus more task specific . By contrast , the mid-level filters learn more nuanced features , such as textures , and therefore tend to display similar patterns across architectures , datasets tasks . As a consequence , and as indicated by our experiments , attacking the initial layers requires large perturbation strengths . Furthermore , while attacking the top-level features works well in the white box scenario , it does not transfer well to different data or architectures , because the features are already too architecture- and task-specific . Attacking mid-level features thus comes as a natural choice , which we will show to yield highly transferable perturbations . Let us now focus on understanding the reason for the transferability when the target data is unavailable to the attacker . In Figure 3 , we plot t-SNE visualizations of different datasets at 4 layers of DenseNet121 . The proximity of the features extracted from the Comics or Paintings datasets , containing images from a few classes particularly focused on humans , to those extracted from ImageNet suggests these datasets as good candidates to train a generator that will fool an ImageNet-trained network ( as shown in Tables 1 and 2 ) . By contrast , the ChestX dataset presents a larger domain gap . Nevertheless , we will show that , albeit for a drop in fooling rate , our use of mid-level features still make a ChestX-trained generator reasonably effective on ImageNet-based classifiers . Furthermore , in Figure 4 , we plot the top 80 activations of ImageNet , Paintings and ChestX at layer 8 of DenseNet121 . This shows that the peaks of the ImageNet dataset ( orange ) are more strongly correlated to those of the Paintings dataset ( left ) than to those of ChestX ( right ) . This is confirmed by our experiments , where a Paintings-trained generator is more effective than a ChestX-trained one . In short , while mid-level features lead to generator transferability , the resulting effectiveness remains affected by the similarity of the data . Algorithm 1 Training a transferable adversarial perturbation generator Input : xi : clean images ; f : pretrained classifer ; : perturbation ` ∞ bound Initialize generator G ; Load classifier f and freeze its parameters ; repeat Get a clean image xi and feed xi to f to obtain f ` ( xi ) ; Generate an unbounded adversarial image G ( xi ) and clip it within an bound of xi to get x̂i ; Pass the adversarial image to f to obtain f ` ( x̂i ) ; Compute the feature separation loss Lfeat ; Compute the gradient of Lfeat w.r.t . the weights of G and update these weights using Adam . until convergence return trained Generator G∗

Summary: The paper proposes an adversarial attack generated by another deep neural network (DNN), that shown to have potential to transfer the attack to different network architectures, different datasets and different tasks. The authors inspire the attack from an observation that mid-level features of a DNN share similarities with different architectures, datasets and tasks. Learning the adversarial perturbation generator network to attack the mid-level features of a DNN shown to transfer to different attack settings. The effectiveness of the attack is justified on different attack settings (white box and variants of black box attacks) that emphasize the transferrable nature of the attack.