Adversarial Reprogramming of Neural Networks

1 INTRODUCTION . The study of adversarial examples is often motivated in terms of the danger posed by an attacker whose goal is to cause model prediction errors with a small change to the model ’ s input . Such an attacker could make a self-driving car react to a phantom stop sign ( Evtimov et al. , 2017 ) by means of a sticker ( a small L0 perturbation ) , or cause an insurance company ’ s damage model to overestimate the claim value from the resulting accident by subtly doctoring photos of the damage ( a small L∞ perturbation ) . With this context , various methods have been proposed both to construct ( Szegedy et al. , 2013 ; Papernot et al. , 2015 ; 2017 ; 2016 ; Brown et al. , 2017 ; Liu et al. , 2016 ) and defend against ( Goodfellow et al. , 2014 ; Kurakin et al. , 2016 ; Madry et al. , 2017 ; Tramèr et al. , 2017 ; Kolter & Wong , 2017 ; Kannan et al. , 2018 ) this style of adversarial attack . Thus far , the majority of adversarial attacks have consisted of untargeted attacks that aim to degrade the performance of a model without necessarily requiring it to produce a specific output , or targeted attacks in which the attacker designs an adversarial perturbation to produce a specific output for that input . For example , an attack against a classifier might target a specific desired output class for each input image , or an attack against a reinforcement learning agent might induce that agent to enter a specific state ( Lin et al. , 2017 ) . In practice , there is no requirement that adversarial attacks will adhere to this framework . Thus , it is crucial to proactively anticipate other unexplored adversarial goals in order to make machine learning systems more secure . In this work , we consider a novel and more challenging adversarial goal : reprogramming the model to perform a task chosen by the attacker , without the attacker needing to compute the specific desired output . Consider a model trained to perform some original task : for inputs x it produces outputs f ( x ) . Consider an adversary who wishes to perform an adversarial task : ∗Work done as a member of the Google AI Residency program ( g.co/airesidency ) . for inputs x̃ ( not necessarily in the same domain as x ) the adversary wishes to compute a function g ( x̃ ) . We show that an adversary can accomplish this by learning adversarial reprogramming functions hf ( · ; θ ) and hg ( · ; θ ) that map between the two tasks . Here , hf converts inputs from the domain of x̃ into the domain of x ( i.e. , hf ( x̃ ; θ ) is a valid input to the function f ) , while hg maps output of f ( h ( x̃ ; θ ) ) back to outputs of g ( x̃ ) . The parameters θ of the adversarial program are then adjusted to achieve hg ( f ( hf ( x̃ ) ) ) = g ( x̃ ) . In our work , for simplicity , we define x̃ to be a small image , g a function that processes small images , x a large image , and f a function that processes large images . Our function hf then just consists of drawing x in the center of the large image and θ in the borders ( though see Section 4.5 for other schemes ) , and hg is simply a hard coded mapping between output class labels . However , the idea is more general ; hf ( hg ) could be any consistent transformation that converts between the input ( output ) formats for the two tasks and causes the model to perform the adversarial task . We refer to the class of attacks where a model is repurposed to perform a new task as adversarial reprogramming . We refer to θ as an adversarial program . In contrast to most previous adversarial work , the attack does not need to be imperceptible to humans , or even subtle , in order to be considered a success . However , we note that it is still possible to construct reprogramming attacks that are imperceptible . Potential consequences of adversarial reprogramming include theft of computational resources from public facing services , repurposing of AI-driven assistants into spies or spam bots , and abusing machine learning services for tasks violating the ethical principles of system providers . Risks stemming from this type of attack are discussed in more detail in Section 5.2 . It may seem unlikely that an additive offset to a neural network ’ s input would be sufficient on its own to repurpose the network to a new task . However , this flexibility stemming only from changes to a network ’ s inputs is consistent with results on the expressive power of deep neural networks . For instance , in Raghu et al . ( 2016 ) it is shown that , depending on network hyperparameters , the number of unique output patterns achievable by moving along a one-dimensional trajectory in input space increases exponentially with network depth . Further , Li et al . ( 2018 ) shows that networks can often be trained to high accuracy even if parameter updates are restricted to occur only in a low dimensional subspace . An additive offset to a neural network ’ s input is equivalent to a modification of its first layer biases ( for a convolutional network with biases shared across space , this operation effectively introduces new parameters because the additive input is not shared across space ) , and therefore an adversarial program corresponds to an update in a low dimensional parameter subspace . In this paper , we present the first instances of adversarial reprogramming . In Section 2 , we discuss related work . In Section 3 , we present a training procedure for crafting adversarial programs , which cause a neural network to perform a new task . In Section 4 , we experimentally demonstrate adversarial programs that target several convolutional neural networks designed to classify ImageNet data . These adversarial programs alter the network function from ImageNet classification to : counting squares in an image , classifying MNIST digits , and classifying CIFAR-10 images . Next , we examine the susceptibility of trained and untrained networks to adversarial reprogramming . We then demonstrate the possibility of reprograming adversarial tasks with adversarial data that has no resemblance to original data , demonstrating that results from transfer learning do not fully explain adversarial reprogramming . Further , we demonstrate the possibility of concealing adversarial programs and data . Finally , we end in Sections 5 and 6 by discussing and summarizing our results . 2 BACKGROUND AND RELATED WORK . 2.1 ADVERSARIAL EXAMPLES . One definition of adversarial examples is that they are “ inputs to machine learning models that an attacker has intentionally designed to cause the model to make a mistake ” ( Goodfellow et al. , 2017 ) . They are often formed by starting with a naturally occuring image and using a gradient-based optimizer to search for a nearby image that causes a mistake ( Biggio et al. , 2013 ; Szegedy et al. , 2013 ; Carlini & Wagner , 2017 ) . These attacks can be either untargeted ( the adversary succeeds when causing any mistake at all ) or targeted ( the adversary succeeds when causing the model to predict a specific incorrect class ) . Adversarial attacks have been also proposed for other domains like malware detection ( Grosse et al. , 2017 ) , generative models ( Kos et al. , 2017 ) , network policies for reinforcement learning tasks ( Huang et al. , 2017 ) , and network interpretation ( Ghorbani et al. , ( b ) ( a ) + ( c ) Adversarial Program yyadv 1 square 2 squares 5 squares 4 squares 3 squares 7 squares 6 squares 9 squares 8 squares 10 squares tench goldfish hammerhead tiger shark white shark stingray electric ray hen cock ostrich ImageNetcounting = 2017 ) . In these domains , the attack remains either untargeted ( generally degrading the performance ) or targeted ( producing a specific output ) . We extend this line of work by developing reprogramming methods that aim to produce specific functionality rather than a specific hardcoded output . Several authors have observed that the same modification can be applied to many different inputs in order to form adversarial examples ( Goodfellow et al. , 2014 ; Moosavi-Dezfooli et al. , 2017 ) . For example , Brown et al . ( 2017 ) designed an “ adversarial patch ” that can switch the prediction of many models to one specific class ( e.g . toaster ) when it is placed physically in their field of view . We continue this line of work by finding a single adversarial program that can be presented with many input images to cause the model to process each image according to the adversarial program . 2.2 PARASITIC COMPUTING AND WEIRD MACHINES . Parasitic computing involves forcing a target system to solve a complex computational task it wasn ’ t originally designed to perform , by taking advantage of peculiarities in network communication protocols ( Barabasi et al. , 2001 ; Peresini & Kostic , 2013 ) . Weird machines , on the other hand , are a class of computational exploits where carefully crafted inputs can be used to run arbitrary code on a targeted computer ( Bratus et al. , 2011 ) . Adversarial reprogramming can be seen as a form of parasitic computing , though without the focus on leveraging the communication protocol itself to perform the computation . Similarly , adversarial reprogramming can be seen as an example of neural networks behaving like weird machines , though adversarial reprogramming functions only within the neural network paradigm – we do not gain access to the host computer . 2.3 TRANSFER LEARNING . Transfer learning ( Raina et al. , 2007 ; Mesnil et al. , 2011 ) and adversarial reprogramming share the goal of repurposing networks to perform a new task . Transfer learning methods use the knowledge obtained from one task as a base to learn how to perform another . Neural networks possess properties that can be useful for many tasks ( Yosinski et al. , 2014 ) . For example , neural networks when trained on images develop features that resemble Gabor filters in early layers even if they are trained with different datasets or different training objectives such as supervised image classification ( Krizhevsky et al. , 2012 ) , unsupervised density learning ( Lee et al. , 2009 ) , or unsupervised learning of sparse representations ( Le et al. , 2011 ) . Empirical work has demonstrated that it is possible to take a convolutional neural network trained to perform one task , and simply train a linear SVM classifier to make the network work for other tasks ( Razavian et al. , 2014 ; Donahue et al. , 2014 ) . However , transfer learning is very different from the adversarial reprogramming task in that it allows model parameters to be changed for the new task . In typical adversarial settings , an attacker is unable to alter the model , and instead must achieve their goals solely through manipulation of the input . Further , one may wish to adversarially reprogram across tasks with very different datasets . This makes the task of adversarial reprogramming more challenging than transfer learning . 3 METHODS . In this work , we consider an adversary with access to the parameters of a neural network that is performing a specific task . The objective of the adversary is to reprogram the model to perform a new task by crafting an adversarial program to be included within the network input . Here , the network was originally designed to perform ImageNet classification , but the methods discussed here can be directly extended to other settings . Our adversarial program is formulated as an additive contribution to network input . Note that unlike most adversarial perturbations , the adversarial program is not specific to a single image . The same adversarial program will be applied to all images . We define the adversarial program as : P = tanh ( W M ) ( 1 ) where W ∈ Rn×n×3 is the adversarial program parameters to be learned , n is the ImageNet image width , and M is a masking matrix that is 0 for image locations that corresponds to the adversarial data for the new task , otherwise 1 . Note that the mask M is not required – we mask out the central region of the adversarial program purely to improve visualization of the action of the adversarial program . Also , note that we use tanh ( · ) to bound the adversarial perturbation to be in ( −1 , 1 ) – the same range as the ( rescaled ) ImageNet images the target networks are trained to classify . Let , x̃ ∈ Rk×k×3 be a sample from the dataset to which we wish to apply the adversarial task , where k < n. X̃ ∈ Rn×n×3 is the equivalent ImageNet size image with x̃ placed in the proper area , defined by the mask M . The corresponding adversarial image is then : Xadv = hf ( x̃ ; W ) = X̃ + P ( 2 ) Let P ( y|X ) be the probability that an ImageNet classifier gives to ImageNet label y ∈ { 1 , . . . , 1000 } , given an input imageX . We define a hard-coded mapping function hg ( yadv ) that maps a label from an adversarial task yadv to a set of ImageNet labels . For example , if an adversarial task has 10 different classes ( yadv ∈ { 1 , . . . , 10 } ) , hg ( · ) may be defined to assign the first 10 classes of ImageNet , any other 10 classes , or multiple ImageNet classes to the adversarial labels . Our adversarial goal is thus to maximize the probability P ( hg ( yadv ) |Xadv ) . We set up our optimization problem as Ŵ = argmin W ( − logP ( hg ( yadv ) |Xadv ) + λ||W ||2F ) , ( 3 ) where λ is the coefficient for a weight norm penalty , to reduce overfitting . We optimize this loss with Adam while exponentially decaying the learning rate . Hyperparameters are given in Appendix A . Note that after the optimization the adversarial program has minimal computation cost for the adversary , as it only requires computing Xadv ( Equation 2 ) , and mapping the resulting ImageNet label to the correct class . In other words , during inference the adversary needs only store the program and add it to the data , thus leaving the majority of computation to the target network . One interesting property of adversarial reprogramming is that it must exploit nonlinear behavior of the target model . This is in contrast to traditional adversarial examples , where attack algorithms based on linear approximations of deep neural networks are sufficient to cause a high error rate ( Goodfellow et al. , 2014 ) . Consider a linear model that receives an input x̃ and a program θ concatenated into a single vector : x = [ x̃ , θ ] > . Suppose that the weights of the linear model are partitioned into two sets , v = [ vx̃ , vθ ] > . The output of the model is v > x = v > x̃ x̃+ v > θ θ . The adversarial program θ adapts the effective biases v > θ θ but can not adapt the weights applied to the input x̃ . The adversarial program θ can thus bias the model toward consistently outputting one class or the other but can not change the way the input is processed . For adversarial reprogramming to work , the model must include nonlinear interactions between x̃ and θ . A nonlinear deep network satisfies this requirement . Inception V3 Inception V4 Inception Resnet V2 Resnet V2 152 Resnet V2 101 Resnet V2 50

Summary: This paper proposed "adversarial reprogramming" of well-trained and fixed neural networks, which can be viewed as learning a trainable input perturbation on a fixed network for multi-tasking by using a different dataset (e.g., MNIST) from the original dataset (ImageNet) as input. Domain mapping functions (h_g and h_f) are required if the data have different dimensions. The key factor to enable adversarial reprogramming of a fixed network to perform a different task is by training the additive adversarial program as defined in (1). Experimental results show that 7 different ImageNet models (adversarially trained or not) can be reprogrammed for performing counting tasks, and MNIST and CIFAR-10 classifications. The authors also show that adversarial reprogramming is less effective on untrained networks.