Universal Adversarial Attack Using Very Few Test Examples

1 INTRODUCTION . Neural network models achieve state of the art results on several image classification tasks . However , these models are known to be vulnerable to adversarial attacks . Szegedy et al . ( 2013 ) show that small , pixel-wise changes that are almost imperceptible to the human eye can make neural network models grossly misclassify images . Regarding an image as a vector x in Rd , an adversarial , pixelwise perturbation gives a small perturbation vector v ∈ Rd to perturb x so that the network classifies x+v differently from x. Szegedy et al . ( 2013 ) find such a perturbation vector using box-constrained L-BFGS . Goodfellow et al . ( 2015 ) propose the Fast Gradient Sign Method ( FGSM ) as a faster approach to find such an adversarial perturbation . In subsequent work on FGSM , its iterative variant was introduced by Kurakin et al . ( 2017 ) and another version called Projected Gradient Descent ( PGD ) by Madry et al . ( 2018 ) . In the works of Goodfellow et al . ( 2015 ) and Madry et al . ( 2018 ) , the adversarial perturbations considered are from an ` ∞-ball around the input x - namely , each pixel value is perturbed by a quantity within [ − , + ] . DeepFool by Moosavi-Dezfooli et al . ( 2016 ) gives a method to compute a minimal ` 2-norm perturbation to fool the input . To understand how adversarial attacks transfer across different models Tramer et al . ( 2017 ) also study model-agnostic perturbations using the direction of the difference between the intra-class means . Universal adversarial perturbations are input-agnostic so that the same perturbation fools the trained model on a large fraction of test inputs . Recent work by Moosavi-Dezfooli et al . ( 2017a ; b ) on universal adversarial perturbations looks at the curvature of the decision boundary . Their universal attacks are more sophisticated than the simple and fast , gradient-based adversarial perturbations , and require significantly more computation . In Moosavi-Dezfooli et al . ( 2017b ) the authors give a theoretical analysis of existence of universal adversarial perturbations using directions in which the decision boundary has positive curvature . Khrulkov & Oseledets ( 2018 ) propose a different method to construct a universal adversarial perturbation . Their perturbation is based on computing the ( p , q ) -singular vectors of the Jacobian matrices of hidden layers of the network . The above discussion raises an important question : are there any interesting properties shared by the input dependent adversarial attacks which can be exploited to come up with an easy to compute universal perturbation ? Can one find such a universal perturbation using only a small fraction of test inputs and also ensure that a perturbation along this direction fools most test inputs ? We answer both of these questions affirmatively in this paper . Furthermore , we show that our universal attack compares favourably with the universal attack in Moosavi-Dezfooli et al . ( 2017a ) . 2 SUMMARY OF OUR MAIN RESULTS . • Our first observation is that even though the adversarial attack directions are inputdependent , overall they have only a small number of dominant principal components . We show this for attacks based on the Gradient of the loss function , the FGSM attack and DeepFool , on a variety of architectures and datasets . • Construct a matrix whose i-th row is an input dependent attack vector on the i-th test point . Our second observation is that a small pixel-wise perturbation in the direction of the top principal component of this matrix alone can be used as a universal attack . This simple algorithm gives us SVD-Gradient , SVD-FGSM and SVD-DeepFool perturbations . • Our third observation is that the top principal component can be well-approximated from a very small sample of the test data , and small samples are good enough to obtain SVDUniversal perturbations comparable to that in Moosavi-Dezfooli et al . ( 2017a ) . • We give a theoretical justification of this phenomenon using matrix concentration inequalities and spectral perturbation bounds . This observation holds across multiple inputdependent adversarial attack directions given by Gradient , FGSM and DeepFool . 3 DEFINITIONS AND THE PROBLEM FORMULATION . Assume that we have a distribution D on pairs of images and labels , with images coming from a set X . In the case of CIFAR-10 for example we can think of X to be a set of all 32 × 32 images with pixel values from [ 0 , 1 ] . Let ( X , Y ) be a sample from D and let f : X → [ k ] , be a k-class classifier . The natural accuracy δ of the classifier f is defined to be Pr ( X , Y ) ∈D [ f ( X ) = Y ] . We regard a deterministic adversary to be a function A : X → X . The error rate of A on the classifier f is defined to be Pr ( X , Y ) ∈D [ f ( X +A ( X ) ) 6= Y ] . WhenA is a distribution over functions , we get a randomized adversary . The norm of the perturbation applied to X is the norm of A ( X ) ( we only consider ` 2 norm in this paper ) . In Moosavi-Dezfooli et al . ( 2017a ; b ) and in Khrulkov & Oseledets ( 2018 ) the authors consider the fooling rate of an adversary and not the error rate . Recall that f is said to be fooled on input x by the perturbation A ( x ) if f ( x ) 6= f ( x+A ( x ) ) . And the fooling rate is defined to be the probability that f is fooled by choosing x at random . Note that Pr ( X , Y ) ∈D [ f ( X +A ( X ) ) 6= Y ] = Pr ( X , Y ) [ f ( X +A ( X ) ) 6= Y |f ( X ) = Y ] Pr ( X , Y ) [ f ( X ) = Y ] + Pr ( X , Y ) [ f ( X +A ( X ) ) 6= Y |f ( X ) 6= Y ] Pr ( X , Y ) [ f ( X ) 6= Y ] ≤ Pr ( X , Y ) [ f ( X +A ( X ) ) 6= Y |f ( X ) = Y ] + ( 1− δ ) . So , if the natural accuracy of the classifier is high , the fooling rate is close to the error rate . Pr ( X , Y ) [ f ( X +A ( X ) ) 6= f ( X ) ] ≥ Pr ( X , Y ) [ f ( X +A ( X ) ) 6= Y ] − ( 1− δ ) . The error rate of the adversary with zero perturbation is the error rate of the trained network , whereas the fooling rate of the adversary with zero perturbation is necessarily zero . However , small fooling rate does not necessarily imply small error rate , especially when the natural accuracy is not close to 100 % . Note that existing models such as VGG16 , VGG19 , ResNet50 do not achieve natural accuracy greater than 0.8 . 3.1 INPUT DEPENDENT ATTACKS . There are a number of input dependent attacks known and our generic algorithm can be used to produce a universal adversary from each of them . We demonstrate the effectiveness of our method by applying it to a few popular input dependent attacks , which we discuss below . Let L ( θ , x , y ) denote the loss function used to train the model . Here θ are the model parameters , x is the input and y is the class label . In the Gradient attack the adversarial example produced on input x is x′ = x+ ∇xL ( θ , x , y ) . In the FGSM attack proposed by Goodfellow et al . ( 2015 ) the adversarial example produced on input x is x′ = x+ sign ( ∇xL ( θ , x , y ) ) . So the perturbed input is within an ` ∞ ball of radius about x. DeepFool proposed by Moosavi-Dezfooli et al . ( 2016 ) is an iterative algorithm . At each stage of the iteration the region of space around the current perturbed input xt where the classifier outputs label f ( xt ) is approximated by a polyhedron Pt . xt is projected to the closest face of Pt . We stop at the smallest t for which f ( x ) 6= f ( xt ) . 3.2 EXISTING UNIVERSAL ATTACKS . In Moosavi-Dezfooli et al . ( 2017a ) , the authors fix a budget , a bound on the allowed ` 2 norm of the attack1 and also the desired fooling rate . They pick a sample S of the training data . Starting with the zero perturbation v , the algorithm proceeds in stages as long as the fooling rate of the current perturbation vector v is less than the desired fooling rate on S. They run over x ∈ S , for which f ( x ) = f ( x+ v ) . For each such x they find the smallest perturbation rx sending x to the boundary ( using DeepFool ) and v is updated to Π ( v + rx ) where Π is the projection onto the ball of radius . In Khrulkov & Oseledets ( 2018 ) the authors consider the outputs fi ( x ) of the i-th hidden layer of a network on an input x. Denote by Ji ( x ) the matrix of partial derivatives ∂fi ( x ) ∂x evaluated at x . They pick a sample S of X of size m and compute for each x ∈ S , Ji ( x ) . Stacking these in different rows of a matrix J , they use as their universal attack the vector argmax||Jv||q subject to ||v||p = 1 . In Mopuri et al . ( 2017 ) the authors propose a universal attack which is completely data independent . It only depends upon the model architecture . Given a trained neural network with K hidden ( convolution ) layers , the authors start with a random image v and minimize Πi=Ki=1 ¯̀i ( v ) , ||v||∞ ≤ . Here ¯̀ i ( v ) is the mean activation of layer i when the input is v. The authors show that the perturbation v achieving the minimum is a data agnostic attack and achieves a reasonable fooling rate . 4 UNIVERSAL ADVERSARIAL ATTACK USING A SMALL SAMPLE . 4.1 SVD-UNIVERSAL ALGORITHM . Algorithm 1 gives the pseudocode to construct a universal attack for a given model . The output of the algorithm is a universal attack vector , and this depends upon the input dependent attack used . It is the simplicity of the algorithm which we would like to emphasize . The algorithm samples a set of n images from the test set . We use the terms batch size and sample size interchangeably . For each of the sample points it computes an input-dependent attack direction . We stack these attack 1the authors consider other ` p norms as well . We restrict ourselves to the ` 2-norm . directions in a matrix , one for each of the sample points and return the top singular vector of the matrix of attack directions as our SVD-Universal . Algorithm 1 : SVD-Universal Algorithm Data : Network N , any input-dependent adversarial attack A , and n test samples . Result : A universal attack direction for network N 1 Obtain input-dependent perturbation vectors a1 , ... , an based on A for the n test samples . 2 Normalize ai ’ s to get only the attack directions or unit vectors ui = ai/||ai||2 , for i = 1 to n. 3 Form a matrix M whose rows are u1 , u2 , . . . , un . 4 Compute Singular Value Decomposition ( SVD ) of M as M = USV T , with V = [ v1|v2| . . . |vn ] . 5 Return the top right singular vector v1 as the universal attack vector .

Summary: This paper presents a universal adversarial attack, which firstly conducts existing gradient-based attacks on the sample images and then applies SVD on the perturbations from those attacks. The universal attacks are the right singular vectors of the SVD.  The experiments are conducted on attacking VGG and ResNet. In addition, theoretical analysis is also provided in the paper.