Adversarial Reprogramming Revisited
Anonymous Author(s)
AffiliationAddress email
Abstract
Adversarial reprogramming, introduced by Elsayed, Goodfellow, and SohlDickstein, seeks to repurpose a neural network to perform a different task, bymanipulating its input without modifying its weights.
We prove that two-layer
ReLU neural networks with random weights can be adversarially reprogrammed toachieve arbitrarily high accuracy on Bernoulli data models over hypercube vertices,provided the network width is no greater than its input dimension.
We also substantially strengthen a recent result of Phuong and Lampert on directional convergenceof gradient flow, and obtain as a corollary that training two-layer ReLU neuralnetworks on orthogonally separable datasets can cause their adversarial reprogramming to fail.
We support these theoretical results by experiments that demonstratethat, as long as batch normalisation layers are suitably initialised, even untrainednetworks with random weights are susceptible to adversarial reprogramming.
Thisis in contrast to observations in several recent works that suggested that adversarialreprogramming is not possible for untrained networks to any degree of reliability.
Introduction
Elsayed, Goodfellow, and Sohl-Dickstein [2019] proposed adversarial reprogramming : given aneural network N which performs a task F : X → Y and given an adversarial task G : (cid:101)
X
→ (cid:101) Y ,repurpose N to perform G by finding mappings h in : (cid:101)
X
→ X and h out : Y → (cid:101) Y such that
G ≈ h out ◦ F ◦ h in .
They focused on a setting where F and G are image classification tasks, X consistsof large images,
(cid:101) X consists of small images and h in and h out are simple and computationallyinexpensive: h in ( (cid:101) x )
= p + (cid:101) x draws the input (cid:101) x at the centre of the adversarial program p and h out isa hard coded mapping from the class labels Y to the class labels (cid:101) Y .
Then the challenge of adversarialreprogramming is to find p such that h out ◦ F ◦ h in approximates well the adversarial task G .
Remarkably, Elsayed et al.
[2019] showed that a range of realistic neural networks can be adversariallyreprogrammed to perform successfully several tasks.
They considered six architectures trained on
ImageNet [Russakovsky, Deng, Su, Krause, Satheesh, Ma, Huang, Karpathy, Khosla, Bernstein, Berg, and Fei-Fei, 2015] and found adversarial programs that achieve very good accuracies on acounting task, MNIST [LeCun, Bottou, Bengio, and Haffner, 1998], and CIFAR-10 [Krizhevsky,2009]. An illustration for the counting task is shown in Figure 1.
In addition to the basic setting, theyinvestigated limiting the visibility of the adversarial program by restricting its size or scale, or evenconcealing both the input and the adversarial program by hiding them within a normal image from ImageNet, and obtained good results.
Adversarial reprogramming can be seen as taking the crafting of adversarial examples [Biggio, Corona,
Maiorca, Nelson, Šrndi´c, Laskov, Giacinto, and Roli, 2013, Szegedy, Zaremba, Sutskever, Bruna,
Erhan, Goodfellow, and Fergus, 2014] to a next level.
Like the universal adversarial perturbation of
Moosavi-Dezfooli, Fawzi, Fawzi, and Frossard [2017], a single adversarial program is combinedwith every input from the adversarial task, but in contrast to the former where the goal is to cause [Figure] ImageNet counting = natural images to be misclassified with high probability, adversarial reprogramming seeks a highaccuracy of classification for the given adversarial task.
Adversarial reprogramming is also related to transfer learning [Raina, Battle, Lee, Packer, and Ng,2007, Mesnil, Dauphin, Glorot, Rifai, Bengio, Goodfellow, Lavoie, Muller, Desjardins, Warde-Farley,
Vincent, Courville, and Bergstra, 2011], however with important differences.
Whereas transferlearning methods use the knowledge obtained from one task as a base for learning to perform another,and allow model parameters to be changed for the new task, in adversarial reprogramming the newtask may bear no similarity with the old, and the model may not be altered but has to be manipulatedthrough its input.
The latter features are suggestive of potential nefarious uses of adversarial reprogramming, and
Elsayed et al.
[2019] list several, such as repurposing computational resources to perform a task whichviolates the ethics code of system providers.
However, its further explorations have demonstratedutility for virtuous deployments to medical image classification [Tsai, Chen, and Ho, 2020], naturallanguage processing [Hambardzumyan, Khachatrian, and May, 2021, Neekhara, Hussain, Dubnov,and Koushanfar, 2019, Neekhara, Hussain, Du, Dubnov, Koushanfar, and McAuley, 2022], moleculetoxicity prediction [Vinod, Chen, and Das, 2020], and time series classification [Yang, Tsai, and
Chen, 2021].
In the last three works, adversarial reprogramming was achieved between differentdomains, e.g. repurposing neural networks trained on ImageNet to perform classification of DNAsequences, and of natural language sentiments and topics.
In spite of the variety of fruitful applications, it is still largely a mystery when adversarial reprogramming is possible and why.
In the only work in this direction, Zheng, Feng, Xia, Jiang, Demontis,
Pintor, Biggio, and Roli [2021] proposed the alignment ∥ g ∥ 1 / ( 1 / n (cid:80) i ∥ g i ∥ 1 ) of the gradients
g i ofthe inputs from the adversarial task with their average g as the main indication of whether adversarialreprogramming will succeed.
However, their experiments did not show a significant correlationbetween accuracy after reprogramming and the gradient alignment before reprogramming.
Thecorrelation was statistically significant with the gradient alignment after reprogramming, but that isunsurprising and it is unclear how to use it for predicting success.
Specifically, a central question on adversarial reprogramming is:
Can neural networks with random weights be adversarially reprogrammed, and more generally, how does training impact adversarial reprogrammability?
First, addressing this question is important for assessing scope of two claims made in the literature: “[Adversarial] reprogramming usually fails when applied to untrained networks” [Zheng et al.,2021].
In addition to networks trained on ImageNet, Elsayed et al.
[2019] experimented using thesame architectures untrained, i.e. with random weights, and obtained generally poor results.
Similarly,the experimental results of Neekhara et al.
[2019] and Zheng et al.
[2021] for random networks aresignificantly worse than their experimental results for trained networks.
However, they remarked thatthis was surprising given the known richness of random networks (see e.g. He, Wang, and Hopcroft[2016c], Lee, Bahri, Novak, Schoenholz, Pennington, and Sohl-Dickstein [2018]), and that it waspossibly due to simple reasons such as poor scaling of the random weights.
“The original task the neural networks perform is important for adversarial reprogramming”
[Elsayed et al., 2019].
Nevertheless a number experimental results including those of Tsai et al.
Neekhara et al.
[2022], Vinod et al.
[2020], Yang et al.
[2021], Zheng et al.
[2021] demonstratedsuccessful adversarial reprogramming between tasks that are seemingly unrelated (e.g. repurposingnetworks trained on ImageNet to acts as HCL2000 [Zhang, Guo, Chen, and Li, 2009] classifiers) orfrom different domains, so the interaction between the original and adversarial tasks remains unclear.
Second, it is important due to implications for the following two applied considerations:
Disentangling architecture and training as factors in adversarial reprogrammability.
Clarifyingthe respective bearings on adversarial reprogramming success of the network architecture and of thetask (if any) it was trained for would improve decision making, either to maximise the success inbeneficial scenarios or to minimise it in detrimental ones.
Managing the cost of adversarial reprogramming.
Understanding when training the network is notessential, and when training for longer does not help or even hinders adversarial reprogrammability,should make it possible to control better the economic and environmental costs.
1.1 Our contributions
We initiate a theoretical study of adversarial reprogramming.
We focus on two-layer neural networkswith ReLU activation, and on adversarial tasks given by the Bernoulli distributions of Schmidt, Santurkar, Tsipras, Talwar, and M ˛adry [2018] over hypercube vertices.
The latter are binary classificationdata models in which the two classes are represented by opposite hypercube vertices, and whensampling a datapoint for a given class, we flip each coordinate of the corresponding class vertex witha certain probability.
This probability is a parameter that controls the difficulty of the classificationtask.
These data models are inspired by the MNIST dataset because MNIST images are close tobinary (many pixels are almost fully black or white).
The remaining parameters are the radius of thehypercube, the input dimension of the neural network, and its width.
We prove that, in this setting, for networks with random weights, adversarial programs exist thatachieve arbitrarily high accuracy on the Bernoulli adversarial tasks.
This holds for a wide variety ofparameter regimes, provided the network width is no greater than its input dimension.
The adversarialprograms we construct depend on the weights of the network and on the class vertices (i.e., thedirection) of the Bernoulli data model, and their Euclidean length is likely to be close to the squareroot of the input dimension.
We also prove that, in the same setting, and where the mapping from the adversarial class labelsto the original class labels is fixed, training the network on orthogonally separable datasets cancause adversarial reprogramming to fail.
Phuong and Lampert [2021] recently showed that, underseveral assumptions, training a two-layer ReLU network on such datasets by gradient flow makes itconverge to a linear combination of two maximum-margin neurons.
We provide a simpler proof of asignificantly stronger result: we show that their assumptions of small initialisation, and of positiveand negative support examples spanning the whole space, are not needed; and we generalise to theexponential loss function as well as the logistic one.
We then observe that, for any Bernoulli datamodel whose direction is in a half-space of the difference of the maximum-margin neurons, and forany adversarial program, the accuracy is at most half, and on one of the two data classes tends to 0under a mild assumption on the growth rate of the difficulty of the data model.
In the experimental part of our work, we demonstrate that, as long as batch normalisation layersare suitably initialised, even untrained networks with random weights are susceptible to adversarialreprogramming.
Our experiments are conducted with six realistic network architectures and MNISTas the adversarial task.
We investigate two different schemes to combine input images with adversarial programs: replacing the centre of the program by the image as was done by Elsayed et al.
[2019],and scaling the image to the size of the program and then taking a convex combination of the two.
Each of the two schemes has a ratio parameter, and we explore their different values.
We find thatthe second scheme gives better results in our experiments, and that for some choices of the ratioparameter, the accuracies on the test set across all six architectures are not far below what Elsayedet al.
[2019] reported for networks trained on ImageNet.
Understanding adversarial examples. Adversarial reprogramming can be seen as a challengingtype of adversarial attack.
Instead of finding perturbations that push examples over a classificationboundary, the goal is to find an adversarial program which is an offset that, when added to any inputfrom the adversarial task, with high probability makes it classified as desired. Equivalently, adversarialreprogramming seeks a single perturbation that pushes all inputs from the adversarial task to theirrespective target classes.
Therefore understanding when and why neural networks are susceptible toadversarial reprogramming is an interesting piece of the puzzle of understanding adversarial examples.
For example, our results on random networks suggest that sensitivity to well-generalising featuresin training data, which was identified by Ilyas, Santurkar, Tsipras, Engstrom, Tran, and M ˛adry[2019] as a key cause of adversarial vulnerability, does not fully explain susceptibility to adversarialreprogramming. A different conceptual framework for understanding adversarial examples, in termsof a dimpled manifold model, was proposed by Shamir, Melamed, and BenShmuel [2021]; here ourresult that training for longer can be a defence against adversarial reprogramming suggests that itmay be interesting to explore how its success depends on shapes of the dimples in classificationboundaries as they evolve during training. More directly related to our work are the results of Bartlett, Bubeck, and Cherapanamjeri [2021]who, building on the works of Daniely and Shacham [2020] and Bubeck, Cherapanamjeri, Gidel,and Tachet des Combes [2021], proved that random ReLU networks of constant depth have smalladversarial perturbations that can be found in one step along the direction opposite to the inputgradient. Those were advanced further recently by Montanari and Wu [2022], removing a restrictionon layer widths.
In comparison, our results show existence of adversarial programs that are pointswhere the classification boundary behaves approximately as required for the adversarial task, andare sensitive to the radius and the difficulty of the latter as parameters; however we leave theoreticalinvestigations of obtaining adversarial programs by gradient methods for future work.
Implicit bias of gradient descent.
By showing that, for linear logistic regression on linearlyseparable data, gradient descent always converges to the maximum-margin solution, Soudry, Hoffer,
Nacson, Gunasekar, and Srebro [2018] initiated a fruitful research direction on implicit bias of gradientdescent, which tackles one of the greatest open questions in deep learning: why do overparameteriseddeep neural networks generalise well [Zhang, Bengio, Hardt, Recht, and Vinyals, 2017].
Some of the works closest to ours are: Lyu and Li [2020], Ji and Telgarsky [2020], who establishedthat, for positively homogeneous deep networks and either exponential or logistic loss, if at sometime training attains perfect accuracy and a small loss, then continuing the training makes the lossconverge to 0 and the weights converge in direction to a Karush-Kuhn-Tucker point of a constrainedoptimisation problem on margin maximisation; Phuong and Lampert [2021], who showed that, fromsmall and balanced initialisations, and when trained with logistic loss on orthogonally separabledata whose positive and negative support examples span the whole space, two-layer ReLU networksconverge to a linear combination of two maximum-margin neurons; Lyu, Li, Wang, and Arora [2021],who proved that, from small initialisations, and when trained with logistic loss on symmetric linearlyseparable data, two-layer networks with the leaky ReLU activation converge to a globally maximummargin linear classifier, and that the result is fragile with respect to the symmetry assumption; and
Vardi, Yehudai, and Shamir [2022], who showed that, for two-layer ReLU networks with first-layerbiases and for data whose points are neither too many nor too correlated, if training converges then itproduces non-robust solutions in spite of robust ones existing.
To supplement our discussion above ofthe motivations for considering random versus trained networks in this work, we remark that we seemerit in the point of Vardi et al.
[2022] that “trained networks are clearly not random, and propertiesthat hold in random networks may not hold in trained networks.”
Physical adversarial examples.
Currently more distant but with potential for interesting connections to our work is the vibrant research direction on adversarial examples in the physical world.
Forexample, our results may inform attempts to design adversarial accessories with prescribed effectsfor a set of participants [Sharif, Bhagavatula, Bauer, and Reiter, 2016], or adversarial patches thatcause prescribed classifications for a set of objects [Brown, Mané, Roy, Abadi, and Gilmer, 2017].
Random networks
Basic notations.
We write: [ n ] for the set { 1 , . . . , n } , ∥ v ∥ for the Euclidean length of a vector v ,∠ ( v , v ′ ) for the angle between v and v ′ , and H d for the d -dimensional unit hypercube {± 1 / √ d } d .
Two-layer ReLU networks.
We consider two-layer neural networks N with the ReLU activation.
We write d for the input dimension, k for the width, w 1 , . . .
, w k ∈ R d for the weights of the firstlayer, and a 1 , . . .
, a k ∈ R for the weights of the second layer.
For an input x ∈ R d , the output is thus N
[Equation]
where ψ ( u ) = max { u, 0 } is the ReLU function.
Random weights.
In this section, we assume that the weights in N are random as follows:
• each w j consists of d independent centred Gaussians with variance 1 /d , and• each a j is independently uniformly distributed in {± 1 / √ k } .
This distribution is as in Bubeck et al. [2021], and standard for theoretical investigations.
It is similarto He’s initialisation [He, Zhang, Ren, and Sun, 2015], with the second layer discretised for simplicity.
The variances of the weights are such that, for any input x ∈ R d of Euclidean length √ d , each w ⊤ j xis a standard Gaussian, and for large widths k the distribution of N ( x ) is close to centred Gaussianwith variance 1 / 2 .
Bernoulli data models.
Adapting from Schmidt et al.
[2018], given a hypercube vertex ϕ ∈ H d , aradius ρ > 0 , and a class bias parameter 0 < τ ≤ 1 / 2 , we define the ( ϕ , ρ, τ ) -Bernoulli distributionover ( x , y ) ∈
ρ H d × {± 1 } as follows:
• first draw the label y uniformly at random from {± 1 } ,• then sample the data point x by taking yρ ϕ and flipping the sign of each coordinateindependently with probability 1 / 2 − τ .
These binary classification data models on hypercube vertices are inspired by the MNIST dataset [LeCun et al., 1998].
The class bias parameter τ controls the difficulty of the classification task, whichincreases as τ tends to zero, i.e. as 1 /τ tends to infinity.
Adversarial program.
In this section, we assume that k ≤ d , i.e. the network width is no greaterthan the input dimension, and we define an adversarial program p which depends on the weights ofthe network N and on the direction ϕ of the Bernoulli data model.
With probability 1 , for all j ∈ [ k ], we have that a j w ⊤ j ϕ ̸ = 0 .
Let us write K + for { j ∈ [ k ] |a
j w ⊤ j ϕ > 0 } , and K − for { j ∈ [ k ] | a j w ⊤ j ϕ < 0 } .
Then, for all j ∈ [ k ], let:
[Equation]
Since k ≤ d , with probability 1 , the weights vectors w j are linearly independent, i.e. the k × dmatrix W whose rows are w j has a positive smallest singular value s min ( W ) .
Hence p ∈ R d existssuch that
[Equation]
The neurons in K + can be thought of as “helpful” for the adversarial task, and those in K − as“unhelpful”.
This definition of an adversarial program p ensures that its effect is to introduce asfirst-layer biases the entries of the vector p ′ .
They are 0 (i.e. do nothing) for every “helpful” neuron,and the negative value − (cid:112) d/ | K − | (i.e. reduce the contribution to the network output) for every“unhelpful” neuron.
From the inequalities in (1), we can expect ∥ p ∥ ≈ √ d if k = o ( d ) (see the appendix for details).
Expected reprogramming accuracy.
Our main result in this section provides a lower (respectively,upper) bound on the output of the adversarially reprogrammed network for positively (respectively,negatively) labelled inputs sampled from the Bernoulli data model. The proof, which proceeds byanalysing concentrations of the underlying distributions, can be found in the appendix. Theorem 1. There exist constants C 1 , C 2 , C 3 , C 4 and C 5 such that, with probability at least(1 − C 1 γ )(1 − γ † ) over the choice of the network N and the labelled data point ( x , y ) sampled fromthe ( ϕ , ρ, τ ) -Bernoulli distribution, provided that 2 dτ 2 ≥ ln(1 /γ † ) , we havey N ( p + x ) >
[Equation]
A corollary is that, for sufficiently large input dimensions d , and under mild restrictions on the growthrates of the network width k , and the radius ρ and the difficulty 1 /τ of the Bernoulli data model,expected reprogramming accuracy is at least (1 − C 1 γ )(1 − γ † ) , which by tuning the probabilityparameters γ and γ † can be arbitrarily close to 100% .
The growth rate restrictions indicate thatnetworks with smaller widths can be reprogrammed for tasks with larger radii, but that networks withlarger widths can be reprogrammed for tasks that are more difficult.
The proof is in the appendix.
Corollary 2.
Suppose that
[Equation]
where η ( k ) , η ( ρ ) , η ( τ ) ∈ [ 0 , 1 ] are arbitrary constants that satisfyη ( ρ ) < 1 − η ( k ) / 2 and η ( τ ) <
η ( k ) / 2 .
Then, for sufficiently large input dimensions d , the expected accuracy of the adversarially reprogrammed network N on the ( ϕ , ρ, τ ) -Bernoulli data model is arbitrarily close to 100% .
Implicit bias
Orthogonally separable dataset.
In this section, we consider training the network on a binaryclassification dataset S =
{ ( x 1 , y 1 ) , . . .
, ( x n , y n ) } ⊆
R d × {± 1 } which is orthogonally separable [Phuong and Lampert, 2021], i.e. for all i, i ′ ∈ [ n ] we have: [Equation] and
[Equation]
In other words, every data point can act as a linear separator, although some data points from theopposite class may be exactly orthogonal to it.
Gradient flow with exponential or logistic loss.
For two-layer ReLU networks with input dimension d and width k as before (but without the assumption k ≤ d , which is not needed in this section),we denote the vector of all weights by
[Equation]
[Equation]
[Equation]
[Equation]
+1) , and we write N θ for the network whose weights are the coordinates of the vector θ .
The empirical loss of N θ on S is L ( θ ) :
= (cid:80) ni
=1 ℓ ( y i N θ ( x i )) , where ℓ is either the exponentialℓ exp ( u ) =
e − u or the logistic ℓ log ( u ) =
ln(1 + e − u ) loss function.
A trajectory of gradient flow is a function θ ( t ) : [ 0 , ∞ )
→ R k ( d +1) that is an arc, i.e. it is absolutelycontinuous on every compact subinterval, and that satisfies the differential inclusion
[Equation]
almost all t ∈ [ 0 , ∞ ) , where ∂ L denotes the Clarke subdifferental [Clarke, 1975] of the locally Lipschitz function L .
Gradient flow is gradient descent with infinitesimal step size.
We work with the Clarke subdifferentialin order to handle the non-differentiability of the ReLU function at zero: ∂ψ (0) is the wholeinterval [ 0 , 1 ].
At points of continuous differentiability, the Clarke subdifferential amounts to thegradient, e.g. ∂ L ( θ )
=
{∇L ( θ ) } .
For some further background, see the appendix.
Initialisation of network weights.
In this section, we assume that the initialisation isbalanced: for all j ∈ [ k ], at time t =
we have | a j | = ∥ w j ∥ > 0 ; andlive: for both signs s ∈ {± 1 } there exist i s ∈ [ n ] and j s ∈ [ k ] such that y i s =
s and at time t = 0we have y
i s a j s ψ ( w ⊤ j s x i s ) > 0 .
The balanced assumption has featured in previous work (see e.g. Phuong and Lampert [2021], Lyuet al.
It ensures that it remains to hold throughout the training, and that the signs of thesecond-layer weights a j do not change (see the proof of Theorem 3).
The live assumption (presentprobabilistically in Phuong and Lampert [2021]) is mild: it states that at least one positively initialisedneuron is active for at least one positive input, and the same for negative ones.
Convergence of gradient flow.
Our main result in this section establishes that the early phase oftraining necessarily reaches a point where the empirical loss is less than ℓ (0) , which implies thatthen every input is classified correctly by the network.
Perhaps surprisingly, no small initialisationassumption is needed, however the proof makes extensive use of orthogonal separability of the dataset(see the appendix, which contains all proofs for this section).
Theorem 3.
There exists a time t 0 such that L ( θ ( t 0 )) <
ℓ (0) .
This enables us to apply to the late phase recent results of Lyu and Li [2020], Ji and Telgarsky [2020],
Lyu et al.
[2021] and obtain the next corollary, which is significantly stronger than the main resultof Phuong and Lampert [2021], extending it to exponential loss, and showing that assumptions ofsmall initialisation, and of positive and negative support examples spanning the whole space, are notneeded.
The corollary establishes that each neuron converges to one of three types: a scaling of themaximum-margin vector for the positive data class, a scaling of the maximum-margin vector for thenegative data class, or zero.
The two maximum-margin vectors are defined as follows: for both signss ∈ {± 1 } , let I s :
= { i ∈ [ n ] | y i = s } , and let v s be the unique minimiser of the quadratic problemminimise
[Equation]
subject to ∀
[Equation]
That a trajectory θ ( t ) converges in direction to a vector (cid:101) θ means lim t →∞ θ ( t ) /
∥ θ ( t ) ∥
= (cid:101) θ / ∥ (cid:101) θ ∥ .
Corollary 4.
As the time tends to infinity, we have that the empirical loss converges to zero, the
Euclidean norm of the weights converges to infinity, and the weights converge in direction to some θsuch that for all j ∈ [ k ] we have | a j | =
∥ w j ∥ , and if a j ̸ = 0 then
[Equation]
A consequence for adversarial reprogramming with a fixed class label mapping is that training toinfinity causes that, for any Bernoulli data model whose direction is in the negative half-space of thedifference of the maximum-margin vectors, and for any adversarial program, the accuracy is at mosthalf (i.e. not better than guessing), and on one of the two data classes tends to 0 provided that thedifficulty 1 /τ of the data model increases slower than the square root of the input dimension d .
Thelatter assumption is considerably weaker than in the results of Schmidt et al. [2018] on the Bernoullidata model, where the bound is in terms of the fourth root.
Note also that, thanks to homogeneity, thesign of the network output does not depend on the norm of the weights, only on their direction.
Proposition 5.
Suppose weights θ are is in Corollary 4, data model D is any ( ϕ , ρ, τ ) -Bernoullidistribution such that cos ∠ ( v 1 − v − 1 , ϕ ) < 0 , and adversarial program p is arbitrary. Then 288 P ( x ,y ) ∼D { y N θ ( p + x ) > 0 } ≤ [Equation]
2 , and there exists s ∈ {± 1 } such that
[Equation]
Experiments
Network architectures and initialisation.
Our experiments are conducted using the following sixnetwork architectures: ResNet-
[He, Zhang, Ren, and Sun, 2016a], ResNet-50V2, ResNet-101V2,
ResNet-152V
[He, Zhang, Ren, and Sun, 2016b], Inception-v3 [Szegedy, Vanhoucke, Ioffe, Shlens,and Wojna, 2016], and EfficientNet-B0 [Tan and Le, 2019].
We use the networks exactly as implemented in Keras in TensorFlow 2.8.0 including the method forrandomly initialising the trainable weights.
For biases, this means they are initialised with 0 .
For allother trainable weights, mostly, the Xavier uniform initialiser [Glorot and Bengio, 2010] is used inthis implementation.
EfficientNet is an exception, where many layers instead use a truncated normaldistribution that has mean 0 and standard deviation (cid:112) 2 / number of output units.
All of these networks involve batch normalisation layers [Ioffe and Szegedy, 2015].
Every such layermaintains a moving mean and a moving variance based on batches it has seen during training.
Theinputs are then normalised accordingly: they are shifted by the recorded mean and scaled by theinverse of the recorded variance.
Note that the moving mean and variance values are not trainable,i.e., they are not subject to updates by the optimiser during training.
During inference, the movingmean and variance values are no longer updated, and the normalisation is performed based on the lastvalues recorded during training. Crucially, in the default implementation of these networks, these moving mean and moving variancevalues are initialised as 0 and 1 , respectively. Therefore, an untrained network initialised in this waywill behave as if the batch normalisation layers were not present. To obtain more sensible random networks, i.e., ones that can still make use of batch normalisation,we initialise the moving mean and variance of batch normalisation layers differently.
We generatea batch of 100 random images (each pixel value is chosen independently and uniformly at randomin the allowed range). This single batch is then fed through the random network and each batchnormalisation layer records the mean and variance values it sees at its input (and normalises its outputaccordingly). The trainable weights of the network are not changed during this process. We should note that, in addition to the moving mean and variance, batch normalisation layers canhave trainable weights, by means of which the output mean and variance can be tuned.
Specifically,such a layer may have trainable weights γ and β , and will scale its otherwise normalised output by γand shift it by β .
If present, these trainable weights are initialised as 1 and 0 respectively, and hencehave no effect.
Our initialisation procedure does not modify these trainable weights and they aretherefore not used in our random networks.
Combining input images with adversarial programs.
Our adversarial programs are colour imageswhose sizes match the expected input size of the networks.
This is 224 × 224 for all networks except
Inception-v3, where it is 299 × 299 .
We use two different schemes to combine input images with the adversarial program.
The firstscheme, used by Elsayed et al.
[2019], is to take the adversarial program and overwrite a portion ofit by the input image.
We do this in such a way that the input image is, up to rounding, centred inthe adversarial program.
We can vary the construction by scaling the input image up or down beforeapplying this procedure.
In particular, we use a parameter r ∈ [ 0 , 1 ] and scale the input image, usingbilinear interpolation, in such a way that r times the width of the adversarial program is equal to thewidth of the scaled image.
That we focus on the width is not important because all our inputs andprograms are square.
Our second scheme involves scaling the image to the same size as the adversarial program and thentaking a convex combination of the two.
We use a parameter v ∈ [ 0 , 1 ] to specify how much weightthe input image should get in this convex combination.
Specifically, the combined image is obtainedby calculating v · I + (1 − v ) · P , where I is the input image and P is the adversarial program.
Adversarial task dataset.
Elsayed et al.
[2019] evaluate adversarial reprogramming on randomnetworks using the MNIST dataset.
In other words, they were asking whether it is possible torepurpose a random network for the task of classifying the handwritten digits from the MNISTdataset. We use the same dataset, which consists of 60 , 000 training images and 10 , 000 test images,for our experiments. It is available under the Creative Commons Attribution-Share Alike 3.0 licence.
[Figure]
{+Scheme0 .0 .
+}
v
Scheme
[Figure]
Finding adversarial programs.
Internally, we represent adversarial programs using unconstrainedweights.
We then apply a softsign function to the weights to map them into the range ( − 1 , 1) , andfurther shift and scale the program such that the pixel values lie in the same range that is used forthe input images.
The program is initialised in such a way that after the application of the softsignfunction, each value lies uniformly at random in ( − 1 , 1) .
We use the 60 , 000 training images torun an Adam optimiser [Kingma and Ba, 2015] with learning rate 0 .
01 and a batch size of 100 tooptimise the unconstrained weights of the adversarial program.
We report the accuracy on the 10 , 000test images after 20 epochs, please see Figure 2.
The experiments were run on a standard desktopcomputer with two NVIDIA GeForce RTX 3080 Ti GPUs.
We did not explore other optimisers, batch sizes, and learning rates, since our first choices alreadyresulted in suitable adversarial programs for these random networks.
However, we did explore thetwo different schemes of combing input images with adversarial programs, and different values forthe respective parameters r and v .
For each network, and each value of r and v , we ran a singleexperiment. While averaging over multiple experiments could potentially reduce the noise in theresults, due to the tests for different values of r and v , an overall picture emerges even when runningonly one experiment per data point.
Discussion.
Overall, the second scheme of combining input images with adversarial programsappears to give better and more reliable results in our experiments.
For both schemes, the choice ofparameters is important.
Clearly, when r or v is 0 , the input image is not visible to the network at all.
On the other hand, when r or v is 1 , there no longer is an adversarial program.
In most cases, bestresults are achieved when the adversarial program is significantly larger (either by actual size in thefirst scheme, or in terms of pixel value ranges in the second scheme) than the input image.
In the second scheme in particular, we see accuracies on the test set which are lower, but not muchlower than what Elsayed et al.
[2019] reported for networks trained on ImageNet.
For v ≈ 0 .
046for example, we see accuracies of 93 .
4% for EfficientNet-B0,
ResNet-50, Inception-v3, ResNet-101V2, ResNet-50V2, ResNet-152V2, respectively.
This suggeststhat, while training, say, on ImageNet may impact the possibility of finding suitable adversarialprograms, such a training may be less important than previously thought and other factors are ofsignificant importance.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] They are determined by the
assumptions of the theoretical results and by the restrictions of the experiments ran,
and we stated both.
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] They stem
from potential nefarious uses of adversarial reprogramming, which we indicated on
page 2 with a reference to a fuller discussion in the literature.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes]
(b) Did you include complete proofs of all theoretical results?
[Yes]
In the paper together
with its appendix.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] In thesupplemental material.
(b) Did you specify all the training details (e g., data splits, hyperparameters, how theywere chosen)?
[Yes] See Section 4.
(c) Did you report error bars (e g., with respect to the random seed after running experiments multiple times)?
[N/A] We only ran each experiment once.
See the discussion in Section 4.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] The experiments are not particularly
demanding and were run on a standard desktop computer with two NVIDIA GeForce RTX 3080 Ti GPUs.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[Yes]
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation?
[N/A]