Universal Adversarial Attack Using Very Few Test Examples
Authors: Amit Deshpande
Sandesh Kamath
K V Subrahmanyam
Venue:  (2020)
Keywords: universal
adversarial
SVD

Abstract: Adversarial attacks such as Gradient-based attacks, Fast Gradient Sign Method (FGSM) by Goodfellow et al.(2015) and DeepFool by Moosavi-Dezfooli et al. (2016) are input-dependent, small pixel-wise perturbations of images which fool state of the art neural networks into misclassifying images but are unlikely to fool any human. On the other hand a universal adversarial attack is an input-agnostic perturbation. The same perturbation is applied to all inputs and yet the neural network is fooled on a large fraction of the inputs. In this paper, we show that multiple known input-dependent pixel-wise perturbations share a common spectral property. Using this spectral property, we show that the top singular vector of input-dependent adversarial attack directions can be used as a very simple universal adversarial attack on neural networks. We evaluate the error rates and fooling rates of three universal attacks, SVD-Gradient, SVD-DeepFool and SVD-FGSM, on state of the art neural networks. We show that these universal attack vectors can be computed using a small sample of test inputs. We establish our results both theoretically and empirically. On VGG19 and VGG16, the fooling rate of SVD-DeepFool and SVD-Gradient perturbations constructed from observing less than 0.2% of the validation set of ImageNet is as good as the universal attack of Moosavi-Dezfooli et al. (2017a). To prove our theoretical results, we use matrix concentration inequalities and spectral perturbation bounds. For completeness, we also discuss another recent approach to universal adversarial perturbations based on (p, q)-singular vectors, proposed independently by Khrulkov & Oseledets (2018), and point out the simplicity and efficiency of our universal attack as the key difference.

that the network classifies Under review as a conference paper at ICLR 2020
UNIVERSAL ADVERSARIAL ATTACK USING VERY FEW TEST EXAMPLES
Anonymous authors Paper under double-blind review
ABSTRACT
Adversarial attacks such Gradient- attacks, Fast Gradient Sign Method (FGSM) by Goodffelow et al. (2015) and DeepFool by Moosavi-Deztooli et al. (2016) are input-dependent, small pixel-wise perturbations of images which fool state of the art neural networks into misclassifying images but are unlikely to fool any human. On the other hand universal adversarial attack an inpur-agnostic perturbation. The same perturbation is applied to all inputs and yet the neural network is fooled on a large fraction of the inputs. In this paper, we show that multiple known input-dependent pixel-wise perturbations share common spec- tral property. Using this spectral property, we show that the top singular vec- tor of input-dependent adversarial attack directions be used as very simple universal adversarial attack on neural networks. We evaluate the error rates and fooling rates of three universal attacks, SVD-Gradient, SVD-DeepFool and SVD- FGSM, on state of the art neural networks. We show that these universal attack vectors can be computed using a small sample of test inputs. We establish our results both theoretically and empirically. On VGG19 and VGG16, the fooling rate of SVD--DeepFol and SVD-Gradient perturbations constructed from observ- ing less than 0.2% of the validation set of ImageNet is as good as the universal attack of Mloosavi-Dezfoor et al. (2017a). To prove our thcoretical results, we use matrix concentration inequalities and spectral perturbation bounds. For com- pleteness, we also discuss another recent approach to universal adversarial per- turbations based on (p, -i-iiulla vectors, proposed independently by Khrulkov & Oseledets (2018), and point out the simplicity and efficiency of our universal attack as the key difference.
INTRODUCTION
Neural network models achieve state of the art results several image classification tasks. How- ever, these models are known to be vulnerable to adversarial attacks. |Szegedy et al. (2013) show that small, pixel-wise changes that are almost imperceptible to the human eye make neural network models grossly misclassify images. Regarding an image as a vector x in Rd. an adversarial, pixel- wise perturbation gives small perturbation vector RRd perturb x+ differently from x. [Szegedy et al.] (2013) find such perturbation vector using box-constrained L-BFGS. Goodfellow al. (2015) propose the Fast Gradient Sign Method (FGSM) faster ap- proach to find such an adversarial perturbation. In subsequent work on FGSM, its iterative variant was introduced by Kurakin et al. (2017) and another version called Projected Gradient Descent (PGD) by Madry et al. (2018). In the works of Goodfellow et al. (0113) and Madry et al. (2018), the adversarial perturbations considered are from an -all around the input x namely, each pixel value is perturbed by quantity within -e,+te, DeepFool by Moosavi-Dezfoolf et al. (2016) gives method to compute minimal 2--nrrm perturbation fool the input. To understand how adversarial attacks transfer across different models Tramer et al. (2017) also study model-agnoste perturbations using the direction the difference between the intra-class means.
Universal adversarial perturbations are input-agnostic that the same perturbation fools the trained model on a large fraction of test inputs. Recent work by Moosavi- Dezzooll et al. (2017a b) on universal adversarial perturbations looks at the eurvature of the decision boundary. Their universal attacks are more sophisticated than the simple and fast, gradient- adversarial perturbations, and require significantly more computation. In Mossavi-Deztoolii al. (2017b) the authors give

---

Under review as a conference paper at ICLR 2020
theoretical analysis of existence of universal adversarial perturbations using directions in which the decision boundary has positive curvature. Khrulkov & Oseledets (2018) propose different method to construct universal adversarial perturbation. Their perturbation is based on computing the (p, 4) -singulrr vectors of the Jacobian matrices of hidden layers the network.
The above discussion raises important question: the input dependent adversarial attacks which can be exploited to come up with an easy to compute universal perturbation? Can one find such a universal perturbation using only small fraction of test inputs and also ensure that a perturbation along this direction fools most test inputs? We answer both of these questions affirmatively in this paper. Furthermore, we show that our universal attack compares favourably with the universal attack in Giboosavi-Dezfoor et al. (2017a).
there any interesting properties shared by
2 SUMMARY OF OUR MAIN RESULTS
Our first observation is that even though the adversarial attack directions are input- dependent, overall they have only small number of dominant principal components. We show this for attacks based on the Gradient of the loss function, the FGSM attack and DeepFool, variety of architectures and datasets.
Construct a matrix whose i-th row is an input dependent attack vector on the i-th test point. Our second observation is that a small pixel-wise perturbation in the direction of the top principal component of this matrix alone can be used as a universal attack. This simple algorithm gives us SVD-Gradient, SVD-FGSM and SVD-DeepFool perturbations.
Our third observation is that the top principal component can be -ell-approximated from a very small sample of the test data, and small samples are good enough to obtain SVD- Universal perturbations comparable to that in Moosavi-Dezfooli al. (2017)).
We give theoretical justification of this phenomenon using matrix concentration in- equalities and spectral perturbation bounds. This observation holds across multiple input- dependent adversarial attack directions given by Gradient, FGSM and DeepFool.
DEFINITIONS AND THE PROBLEM FORMULATION
Assume that we have a distribution on pairs of images and labels, with images coming from set X. In the case of CIFAR-IO for example we can think of X' to be set of all 32 x 32 images with pixel values from [o, 1). Let (X sample from D and let f X - [kc] be a k-class classifier. The natural accuracy 8 of the classifier f is defined to be
(f = Y].
We regard a deterministic adversary a function A X + X. The error rate of on the classifier f is defined to be
Pr(xy))Dlf(X + A(X)) # Y).
When A is a distribution over functions, we get randomized adversary. The norm of the perturba- tion applied to X is the norm of A(X) (we only consider E2 norm in this paper).
In Moosavi-Dezfooli al. 2017a b) and in Khrulkov & Oseledets (2018) the authors consider the fooling rate of an adversary and not the error rate. Recall that / is said to be fooled on input æ by the perturbation A(x) if f(x) # f(z + A(z)). And the fooling rate is defined be the probability that f is fooled by choosing x at random. Note that
Pr(x.y)ed[((X + A(X)) # YY] = =Pr(x.)) +0/((x+AAXX A(X)) # Ylf(X) = Y If = Y]
+ Pf +AX))) * Ylf(X) # Y] Pr(x,))((X) # Y]
PF +A(X)) + (|f(X) +
So, if the natural accuracy of the classifier is high, the fooling rate is close to the enror rate.
Pfxx + A(X)) + f(x) 2 lf A(X) # YJ (1 - 5).
2

---

Under review as a conference paper at ICLR 2020
The error rate of the adversary with perturbation is the error rate of the trained network, whereas the fooling rate the adversary with zero perturbation is necessarily zero. However, small fooling rate does not necessarily imply small error rate, especially when the natural accuracy is not close to 100%. Note that existing models such as VGG16, VGG19, ResNer50 do not achieve natural accuracy greater than 0.8.
3.1 INPUT DEPENDENT ATTACKS
There are number of input dependent attacks known and generic algorithm be used to produce universal adversary from each of them. We demonstrate the effectiveness of our method by applying it to few popular input dependent attacks, which discuss below. Let L(0, xr y) denote the loss function used to train the model. Here 0 are the model parameters, æ is the input and y is the class label.
In the Gradient attack the adversarial example produced on input xr is
= x - cV L
In the FGSM attack proposed by Goodfellow al. (2015) the adversarial example produced on input is
x' x + sign L,, x. y))
So the perturbed input is within an foc ball of radius c about
DeepFool proposed by Moosavi-Dezzooli et al.) (2016) is iterative algorithm. At each stage of the iteration the region of space around the current perturbed input x' where the classifier outputs label f(x') is approximated by a polyhedron P.. x' is projected the closest face of Pt. We stop the smallest for which (xx & f(z").
3.2 EXISTING UNIVERSAL ATTACKS
In Mloosavi-Dezfoor et al. 70)7a),, the authors fix a budget c, a bound on the allowed l2 norm of the attack"" and also the desired fooling rate. They pick sample S of the training data. Starting with the zero perturbation v, the algorithm proceeds in stages as long as the fooling rate of the current perturbation vector v is less than the desired fooling rate on S. They run over x E S, for which f(x) = fxx + v). For each such æ they find the smallest perturbation Tz sending to the boundary (using DeepFool) and is updated to II(e + rz) where II is the projection onto the ball of radius c.
In Khrulkov & Oseledets 2018) the authors consider the outputs fi (z) of the i-th hidden layer of They pick sample S of X' of size m and compute for each x E S, Jx(x). Stacking these in different rows of matrix J, they use as their universal attack the vector [rg subject ||b||p 1.
network input x. Denote by (1))) the matrix of partial derivatives 0f((z) evaluated
In Mopuri et al. (017) the authors propose a universal attack which is completely data independent. only depends upon the model architecture. Given trained neural network with K hidden (convo- lution) layers, the authors start with a random image v and minimize I1 (v),, ||e]loo < e. Here (v) is the mean activation of layer when the input v. The authors show that the perturbation achieving the minimum is data agnostic attack and achieves a reasonable fooling rate.
4 UNIVERSAL ADVERSARIAL ATTACK USING SMALL SAMPLE
4.1 SVD-UNIVERSAL ALGORITHM
Algorithm I gives the pseudocode to construct a universal attack for a given model. The output of the algorithm universal attack vector, and this depends upon the input dependent attack used.
the simplicity of the algorithm which we would like to emphasize. The algorithm samples a of images from the test set. We use the terms batch size and sample size interchangeably. For each of the sample points it computes an input-dependent attack direction. We stack these attack
the authors consider other 4 norms as well. We restrict ourselves the fn-nomm
3

---

Under review as a conference paper at ICLR 2020
directions in matrix, one for each of the sample points and return the top singular vector of the matrix of attack directions as our SVD-Universal.
Algorithm 1: SVD-Universal Algorithm
Data: Network N, any input-dependent adversarial attack A, and n. test samples.
Result: A universal attack direction for network N
Obtain input-dependent perturbation vectors @1 ... an based on A for the n test samples.
Normalize a;'s 's to get only the attack directions or unit vectors u, a./ //://alll,, for i 1 to n.
Form a matrix M whose rows are 1/1, U2....., My.
Compute Singular Value Decomposition (SVD) of M as M = USVT, with
V= [va |2|| ..- [vn].
Return the top right singular vector V1 the universal attack vector.
4.2 ERROR RATES AND FOOLING RATES OF
We evaluate the fooling rate of SVD-Gradient SVD-FGSM and SVD-DeepFio on the validation set of the ImageNet dataset on VGG16, VGG19 and ResNet50. We used off the shelf code available for these architectures. In these architectures pixel intensities of images are scaled down and images normalized before they are deployed for use in classifiers. Our (unit) attack vectors are constructed using batch sizes of 64(0..13 64(1%.)%) 13%). 128(0.26%) and 1024(2%)
In Figure we plot the fooling rate of cv as a function of c, in all the architectures, where v obtained using batch size of 64. The top left plot is the fooling rate of SVD-gradient, the top right that of SVD-FGSM and the bottom left that of SVD-DeepFool. We explain the bottom right plot in the next section. SVD-Gradient attack scaled to have norm 50 fools about 32% of the validation set of ImageNet on VGG19. Note that the average & norm of this validation set is 45 and using the language of MMoosvi-Dezfoolli et al. (017a), a perturbation with norm 50 is also quasi-impercepiibl.. We visualize the perturbed images in Figure 4 in Appendix A.
We plot the error rate of SVD-Universal for ImageNet on VGG19 in Figure Appendix A.I on different batch sizes. It is clear from the plots that SVD- Universal attacks obtained using batch size of 64 perform as well (and sometimes better) as attack vectors obtained using larger batch sizes.
In Appendix A..2 we plot the eтror rate of SVD- Universersal attacks on CIFAR- 10 trained on ResNet18 88 On ResNet18, 8, the error rate of SVD-DeepFool obtained from 100 samples of the CIFAR-10 test data is 0.55 when it is scaled to norm 15 (0.23 of the average norm), see Figure 6 We notice that the SVD-Universal perturbations constructed with small batch sizes perform better.
The visualizations of our perturbations images given in Appendix B
4.3 COMPARISON WITH MoosAVI-DEEZOOL ET AL. (017A) KHRULKOV & OSELEDETS (2018)
In this section we compare our results with that reported in Moosavi-Deziooli al. 201 and Khrulkov & Oseledets (008)) on the validation set of Imagenet. The pixel intensities of the dataset used in the experiments of Moosavi-Dezfooli al. (2017a) and Khrulkov & Oseledets (2018) in the range [0, 255 The average E2 norm of the validation set is 50,000 and the average foo norm is 250, Moosovi-Deztoolli al. [2017a] Footnote, Page 4). The &a norm of the universal perturbation used in Moosavi-Dezzoooli et al. (2017a) is 1/25 of 50,000 = 2000. The for norm of the universal perturbation used in Khrulkov & Oseledets (2018) is 1/25 of 250 10. The pixel intensities of the validation set of Imagenet used in our experiments is normalized and the average lg norm of this normalized dataset 450.
In Khrulkov & Oseledets 2018 Figure 9) the authors report that the universal perturbation of Moosavi-Deziooli et al. (2017a) constructed from a batch size 64 and having foc norm 10 has fooling rate of 0.14 on VGG19, A comparable perturbation in our model has & norm 1/25 of 450- = 18. Figure I shows that SVD-Gradient and SVD-DeepFool perturbation with norm 18 give a fooling
rate of 0.13 on VGG19.
4

---

Under review as a conference paper at ICLR 2020
VGG16 GGadient-
VGG16-FOSM F-SM-
VGG19 Gradient- -4
VGG19. FGSM -44
ResNet50 Gradient-
ResNet50 FGSM-
Fooling Rate
0.4
0.2
02
0.0
0.0
50
300
150
200
250
0
300
150
200
250
Norm of Perturbaltton
Norm Perturbatiio
VGG16 Deepfool 64
VGGI6 M.DFFF 64
VGG19 DespFasl- 64
VGG19 M.DFFF 64
0.0
ResNet50 DeepFool-64
ResNet50 M..FFF-
Rooiing Rate
0.6
0.4
0.4
0.2
02
0.0
50
200
300
150
250
Norm Perturbatton
Perturbation
Figure 1: On ImageNet validation, VGG16 VGG19 vs ResNet50: fooling rate norm of perturbation. Attacks constructed using 64 samples. (top left) WVD-Gradient (top right) SVD-FGSM (bottom left) SVD-DeepFool and (bottom right) M-DFFF universal.
To further compare our universal perturbations with that of Moosavi-Dezfool et al. (2), the same 64 samples used construct our SVD-Universal vectors were used to obtain the universal perturbation vector, M-DFFF, of Moosavi-Dezfoolli et al. (2017a). This vector was scaled down to get unit vector in &2 norm. Figure (right bottom) we plot the fooling rate M-DFFF function of e, in all the architectures. We observe trends similar what reported in Moosavi-Dezionn et al. 2201aa Table 1) - the error rate achieved by M-DFFF on VGG16 is higher VGG19. Observe from this figure that VGG19, the M-DFFF universal
than that of M-DFFF perturbation of & norm 18 gives a fooling rate 0.13.
The plots in Figure of the fooling rates of the four universal attacks on VGG16 and ResNet50 are collated together in Figure 2 We also plot the error rate of these universal attacks on VGG16 and ResNet50. SVD-DeepFool and M-DFFF have comparable fooling and error rates for ImageNet on the two networks.
Khrulkov & Oseledets (2018) get fooling rate of more than 0.4 using batch size of 64 and Loo norm 10. At this norm their universal attack is stronger than both our attack and that of MMosavi- Dezfooli al. 72)17a). However Khrulkov Oseledets (21))) do an extensive experimentation and determine which intermediate layer to attack and (p. q) are also optimized to maximize the fooling rate of their (p. sinnulr vector. However (p. q) SVD computation is expensive and is known to be hard problem, (Bhaskara & Vijayaraghavan 2011 Bhattiprolu et al. 2019). We do no such optimization and use p = q 2, our emphasis being on the simplicity of our SVD-Universal algorithm. We also observe trend similar to what is reported by Khrulkov & Oseledets (2018) the error rate of SVD-Universal attacks is higher on VGG16 and VGG19 than on ResNet50.
Figure in Appendix A.2 compares the error rates of SVD-Universal attacks and M-DFFF con- structed using samples of size 100 from the test data CIFAR-10 on ResNet18. Error rates of SVD- DrD-DeeFFool and M-DFFF are comparable for this dataset, SVD-Grade is not good. When the norm of the perturbation is about 15 (0.23 of the average norm), the error rate of SVD-DeepFool fool about 0.55.
5

---

Under review as a conference paper at ICLR 2020
VGG16
ResNet50
Gradient Deepfool
Gradient-
FGSM
Deeptool 64
0.
M MDFFF 64
M-OFFF 64
0.7
Etror
0.6
0.6
0.
0.5
04
0.3
50
100
150
200
150 Norm Perturbation
100
250
Norm of Perturbation
Gradient
Gradient
0.8
Deepfool-
0.
Deepfool 64
M.DFFF-66 64
M.OFFF 64
Fooling Rate
0.6
Fooling 0.6
0.4
02
Norm Perturbation
Perturbation
Figure 2: On ImageNet validation, (top left): VGGI6: error rate (bottom left) VGGI6: fooling rate (top right) ResNet50: error rate, (bottom right) ResNet50: fooling rate, vs. norm of perturbation along top singular vector of attack directions on 64 samples.
THEORETICAL ANALYSIS OF OUR UNIVERSAL PERTURBATION
In this section, we attempt to provide a theoretical justification for the existence of universal adver- sarial perturbations. Let our data be given by joint distribution 'D on feature vectors in IRd and corresponding true labels in [k] = {1, ...... k.. Let (X, YY) denote a random sample from D. Let : [K] be given classifier, and for any Rd, let A(x) be its adversarial perturbation given by a fixed adversarial attack A. e.g., FGSM, DeepFool. As in MMosavi-Dezfooll et al. (2017b), the fooling rate of this attack is given by Pr (f(X)= # f(X + A(X))) where the probability is over the data distribution D.
Define A fx f(x) * f(x + A(x)). For any x A, assume that x + A(xx) lies the decision boundary, and the hyperplane H₂ = fx + z E Rd (z, = ||A(()) be a local, linear approximation to the decision boundary at + A(x). This holds for adversarial attacks such as DeepFool by Mdoosavi-Dezforee
al. (2016) which try find an adversarial perturbation A(x) such that x + A(x) is the nearest point to on the decision boundary. Now consider the halfspace Sx = (x+2 TRd (z. A(x)) > [l) Note that Sx and x A(x) Sx. For simplicity of analysis, we assume that f(x) * f(x + 2). for all x E A and x z E Sx. This is a reasonable assumption in small neighborhood of x, and is implied by the positive curvature of the decision boundary assumed in the analysis of /MMosavi-Dezfooll et al. (20175). In other words, we assume that if an adversarial perturbation such as A(x) fools the model input x, then any perturbation z, where z has a sufficient projection along A(x), also fools the model at and this is a reasonable assumption in a small neighborhood of x.
Theorem 1. Given any joint data distribution D on features or inputs in IRd and true labels in [k], let (X,Y Y) denote a random sample from D. For any x E IR,, let A(x) be its adversarial perturbation by a fixed input- inputddependent adversarial attack A. Let
A(X)
A(X)T
M =
E gdddd
||XXI|,
[A(XI

---

Under review as a conference paper at ICLR 2020
and 0 S X S 1 be the top eigenvalue of M and Ra be the corresponding eigenvector (normal- ized to have unit lg norm). Then under the assumption that f(x) + f(x + z), for all x E A and + z E Sx, we have
Pr (f +u) f(X)) 2 Pr(f(X) + f(X + A(X))) - 1 1
1 82
where u = +(//)/v, where c = max ||(())]22
Proof. Let ju(x) denote the induced probability density on features or inputs by the distribution 'D. Define A fxx f(x) + f(x A(x)] and - {x [A(x),, v) 2 6 [|A(x)]l2)- Since À the top eigenvalue of M with v as its corresponding (unit) eigenvector,
x = E K
A(X)
}
(LA(X))22
A(x)
2
JKHG ||A(x))ll v)) m(x)dx
A(x)
|1A(x)]l2 .V) u(x)ddx XEG
[ p(x)dx because ||v||2 =1
1(x)dx + 002
IxeG
XEG
= Pr(G)) + --00( (1--Pr(G)
= (1 11-08 82) Pr PG)
Thus, Pr (G) 2 (A /2) (1- 62), and equivalently, Pr ( =1 Pr Pr(
(1- (11-2)((11-1 82).
Now for any E G. we have ((A(x), v)l 2 5 ||A(x)]l2 Letting = maxx ||(x))] l2 we get (A(x), (c/5)v)
||A(x)) Thus, x= (c/5)v Sx, where Sx fx
Rd (z, A(x)) 2
[lA()))2) and therefore, by our assumption stated before Theorem | we have f(x) * f(x + u), where =(e/8lv. Putting all of this together, Pr (f(X + u) f(X)) > Pr (G 2 Pr (A) - Pr (r(An))) 2 Pr (A) Pr (G"), and therefore,
1-
Pr( + u) # f(X)) 2 Pr( # ( - 1- 82
Theorem shows that any norm norm-bounded bounded, input-dependent, adversarial attack A can be converted into a universal attack u of comparable norm, without losing much in the fooling, if the top eigen- value À of M is close to 1. This universal attack direction lies in the one-dimensional span of the top eigenvector v of M. The proof of Theorem [ be easily generalized to the top SVD subspace of M and where the top few eigenvalues of M dominate its spectrum (note that tr (M) 1).
Singular value drop We empirically verify our hypothesis about top eigenvalue (or the top few eigenvalues) dominating the spectrum of M in Theorem Let X1. X2...., Xm be m i.i.d. samples of X drawn from the distribution D and consider the unnormalized, empirical analog of M as follows,
A(X)) A(X,)
Σ
|(A(X)) 11A(X)I12
Figure 3 shows how the singular values drop for the three input dependent attacks, Gradient, FGSM, and DeepFool on CIFAR- trained on ResNet18 on batch sizes 500 and 10,000. These plots indicate that the drop in singular values is shared phenomenon across different input dependent attacks, and the trend is similar even when we look at a small number of input samples.
Our second contribution is finding good approximation to the universal adversarial perturbation given by the top eigenvector v of M, using only a small sample X1. X....., Xmm from the distribu- tion D. Theorem (sswoss shows that we can efficiently pick such small sample whose size is independent of D, depends linearly on the intrinsic dimension of M, and logarithmically on the feature dimension d.
7

---

Under review as a conference paper at ICLR 2020
2.00
Gradient-500
Gradient-10k
FCSM-500 oel 500
FCSM-30K DeepPooo-200 30k
1.50
Singuiar Vaku
125
030
100
200
300
400
500
0
500
1000
1500
2000
2500
3000
Index
Index
Figure 3: On CIFAR-IO, ResNet18, Singular values of attack directions over sample of (left) 500 and (right) 10,000 test points.
Theorem 2. Given any joint data distribution D on features in I& and true labels in [k)], let (X, denote random sample from D. For any x E Rd, let A(x) denote the adversarial perturbation of according a fixed deput-dependent adversarial attack A. Let
A(X) A(X)*
=
E cRa
[IA(X)I2 [A(X)]
Let = ||MI| S 1 denote the top eigenvalue of M and let denote its corresponding eigenvector (normalized have unit (2 norm). Letr tr (M) / ||Ml|2 be the intrinsic dimension of M. Let X2.., be m i.i.d. samples of X drawn from the distribution D, and let Ā = AM be the top eigenvalue of the matrix M,
A(X)) A(X.) m |A(X)))l [A(X)))2
M
and v be the top eigenvector of M.
Also suppose that there is gap of at least yA between the top eigenvalue X and the second eigen- value of M. Then for any 0 S < y and m = 2 2og d), we get vvv I c/7, with a constant probability. This probability can be boosted the ((-).
ô by having an additional log (1/5) in
Proof. Take m = O(E log d). By the covariance estimation bound Theorem and Markov's inequality, we get that M Mi ed, with constant probability. Applying Theorem due
to Weyl on eigenvalue perturbation, we get A -- X| < <A. Moreover, if there gap of at least 7 between the first and the second eigenvalue of M with 7 c, of Davis-Kahes Kahan to bound the difference between the corresponding eigenvectors as ||v - v S c/7, with constant probability. Please Appendix D and the book by Vershynin (2018) cited therein, for more details about the covariance estimation bound, Weyl's theorem, and Davis-Kahal theorem.
the well-known Theorem
The theoretical bounds are weaker than our empirical observations about the number of test samples needed for the universal attack. What we want to highlight that the bound in Theorem 2 is independent of the support of underlying data distribution D and depends only on its dimensionality There more room to tighten our analysis using additional properties of the data distribution as well as the spectral properties of M.
6 UNIVERSAL INVARIANT PERTURBATIONS AND FUTURE DIRECTIONS
Let x be an image for which the predicted label of a neural network classifier f is f(x). For highly accurate classifiers trained data that already contains small rotation augmentations, one would
8

---

Under review as a conference paper at ICLR 2020
expect f(x + A(x)) f(x). where x A(x) is the image resulting from small rotation applied to x. For an image x, we call Z(xx) an invariant perturbation for x, if f
+ I(x)) = f(x). Using this terminology expect A(x) to be invariant perturbation for for classifiers trained on data that is inherently (or additionally) augmented with small rotations. A natural question is whether there any universal invariant perturbations
that are invariant for most We show empirically that there are such universal invariant perturbations. By their very definition we would also expect that universal invariant perturbations will not be universal adversarial perturbations. We show this empirically by plotting the error rates of the top singular invariant perturbation of CIFAR- 10 on ResNet18 8 in Figure [0 Appendix C The error rate below 0.2 even when the norm of the invariant perturbation is scaled up to 50. We give more empirical evidence of such invariant perturbations we show that the principal angles between the subspace of the top universal adversarial perturbations and the subspace of the top 5 universal invariant perturbations are all close to 90%
The empirical and theoretical results this paper hold greater generality than what we have presented. Our experiments indicate the existence of universal adversarial perturbations and univer- sal invariant perturbations for datasets trained on equivariant networks. GCNN's s Cohen & Welling (016)) are among the more popular equivariant networks, and achieve state of the In Appendix C we give a peek into this work and show plots of the error rates of SVD-Universal on CIFAR-10 on GCNN's. We also plot the singular values of CIFAR- 10 on GCNN's.
CIFAR-IO.
is common to augment data with rotations and train neural networks, so they become equivariant to rotations and image and its rotation are classified the same. We believe that study of how the subspaces of universal adversarial perturbations and universal invariant perturbations change as model trained handle more and more rotations will be important. This is ongoing work.
CONCLUSION
We show how to use a small sample of input-dependent adversarial attack directions on test inputs to find single universal adversarial perturbation that results in large drop in the accuracy of state of the art neural network models. Our main observation is a spectral property shared by different attacks directions such as Gradients, FGSM, DeepFool. We give a theoretical justification for how this spectral property helps in using the top singular vector as a universal attack direction. We justify theoretically and empirically that such a perturbation can be computed using only small sample of test inputs. Our empirical results also indicate that these universal perturbations are almost orthogonal perturbations of inputs that these models invariant to.
REFERENCES
Aditya Bhaskara and Aravindan Vijayaraghavan. Approximating matrix p-norms. In Proceedings of the Twenty-second Annual ACM-SIAM Symposium on Discrete Algorithms, SODA '11, pp- 497-511, Philadelphia, PA, USA, 2011. Society for Industrial and Applied Mathematics.
Vijay Bhattiprolu, Mrinalkanti Ghosh, Venkatesan Guruswami, Euiwoong Lee, and Madhur Tul- siani. Approximability of + q matrix norms: Generalized krivine rounding and hypercon- tractive hardness. In Proceedings of the Thirtieth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA '19, pp. 1358-1368, Philadelphia, PA, USA, 2019. Society for Industrial and Applied Mathematics.
Taco S. Cohen and Max Welling. Group equivariant convolutional networks. In Proceedings of the International Conference Machine Learning (ICML), 2016.
Ian Goodfellow. Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations, 2015.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and patlern recognition, pp- 770-778, 2016.
Valentin Khrulkov and Ivan Oseledets. Art of singular vectors and universal adversarial perturba- tions. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018.

---

Under review as a conference paper at ICLR 2020
Alexey Kurakin, lan Goodfellow, and Samy Bengio. Adversarial examples in the physical world. arXiv preprint 10X7:01607.02533 2017.
Aleksander Madry, Aleksandar A Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Reprexentations, 2018.
Seyed-Mohsen Moosavi-Dezfooli: Alhussein Fawzi, and Pascal Frossard. Deepfool: simple and accurate method to fool deep neural networks. Proceedings of the IEEE Conference on Com- puter Vision and Pattern Recognition, 2016.
Seyed-Mohsen
Moosavi-Dezfooli, Alhussein Fawzi, Omar Fawzi, and Pascal Frossard. Universal adversarial perturbations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017a.
Seyed-Mohsen Moosavi-Deziee
Alhussein Fawzi, Omar Fawzi, Pascal Frossard, and Stefano Soatto. Analysis of universal adversarial perturbations, CoRR, abs/1705.000 70550,954, 2017b.
Konda Reddy Mopuri, Utsav Garg, and R Venkatesh Babu. Fast feature fool: A data independent approach to universal adversarial perturbations. In Proceedings of the British Machine Vision Conference (BMVC), 2017.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Fei- Fei. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 115(3):21 Dec 2015. ISSN 1573-1405, doi: 10.1007/s 10.1007/:1122--15-5088 1263-015-081666;
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfel- low, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv: 2013.
Florian Tramer, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick McDaniel. The space of transferable adversarial examples. arXiv preprint 12.3vv1704.03444 2017.
Roman Vershynin. Iiih--Diensional Probability: An Introduction with Applications in Data Sci- ence. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, 2018. doi: 10.101779781108000 108233159..
APPENDIX A DATA SETS AND MODEL ARCHITECTURES USED
Data sets CIFAR-10 dataset consists of 60, 000 images of 32 x size, divided into 10 classes. 40. 000 used for training, 10, 000 for validation and 10, 000 for testing. ImageNet dataset refers ILSRVV 2012 dataset (Russakovsky et al.) 2015) which consists of images of 224 224 size, divided into 1000 classes. We use the validation set of 50, 000 images for our testing purposes.
All experiments performed on neural network-bard models were done using the validation set of ImageNet and test set of CIFAR-10 datasets.
Model Architectures For the ImageNet based experiments we use pre-trained networks of VGG16, VGG19 and ResNet50 architectures For the CIFAR-10 based experiments we use the ResNet18 architecture as in He et al. (2016).
10

---

Under review as a conference paper at ICLR 2020
A.1
ON IMAGENET
Figure 4 shows that the images with SVD-DeepFool perturbations upto norm 100 are quasi- imperceptible. Figure 5 shows the error of SVD-Universal VGG19 calculated with varying sample size.
Norm
Norm 16
Norm 50
Norm 100
Norm 450
By the
to 30
# 00n
Figure 4: Sample images misclassified at norm 16. Images perturbed with SVD SVD-DeepFool.

---

Under review as a conference paper at ICLR 2020
09
0.
0.7
Errer Rate
0.6
05
04
03
100
150
200
250
50
100
150
200
250
Norm Perturbation
Norm of Perturtation
0.9
129
64
1024
0.8
0.6
0.5
0.4
0.3
50
100
150
250
Norm Pertorbation
Figure 5: On ImageNet validation, error rate, VGG19: rate vs. norm of perturbation along top singular vector of attack directions on 64/128/1024 sample, (top left) Gradient (top right) FGSM (bottom) DeepFool
A.2
SVD-UNIVERSAL ON CIFAR-10
In this section we plot the error rates of SVD-Universal on CIFAR-10 trained on ResNet 88
0.8
500
0.7
10000
10000
0.6
0.6
0.5
Ereer
0.4
0.3
0.2
0.1
Norm Perturbation
Perturbation
500
0.7
10000
Entor
0.4
0.3
10
30
Norm of Perturbation
Figure 6: On CIFAR-10, ResNet18: error rate vs. norm of perturbation along top singular vector of attack directions on 100/500/10000 sample, (top left) Gradient (top right) FGSM (bottom) DeepFool
12

---

Under review as a conference paper at ICLR 2020
0.8
0.7
Error Bate
0.5
Gradient 100
0.2
FSSM 100
DeepPool 100
0.1
M.DFFF
10
30
40
Norm of Perturbation
Figure 7: On CIFAR-10, ResNet18: error rate vs. norm of perturbation along top singular vector of attack directions on 100 samples.
13

---

Under review as a conference paper at ICLR 2020
APPENDIX B VISUALIZING SVD-UNIVERSAL PERTURBATIONS
We visualize the top singular vectors for the Gradient, FGSM, DeepFool directions from ImageNet and CIFAR-I0 this section.
Figure 8: For ImageNet, Top SVD vector from (left) Gradient, (center) FGSM, (right) DeepFool on ResNet50.
Figure 9: For CIFAR-I0 (top) Top 5 SVD vectors from Gradient, (middle) Top 5 SVD vectors from
FGSM, (bottom) Top 5 SVD vectors from DeepFool on ResNet18.
14

---

Under review as a conference paper at ICLR 2020
APPENDIX C INVARIANT PERTURBATIONS
C.I PRINCIPAL COMPONENTS OF ATTACK AND INVARIANT DIRECTIONS
To construct the matrix, pick a batch of 100 images, and for each image our sample we com- pute the difference between x and its degree rotation. We stack these (input dependent invariant directions) as rows of the matrix and compute the top singular (unit) vector w. We call w the uni- versal invariant perturbation. Figure [0 shows the error rate of ew as function of CIFAR-IO. The curves are flat indicating that these directions are invariant perturbations.
0.16
CIFARIO
ResNet18
0.15
CIFARI0
GCNN/ResNet18
0.14
0.13
0.12
Error
0.11
0.10
0.09
0.08
10
20
50
Norm of Perturbation
Figure 10: Error rate norm of perturbation using top singular invariant vector for StdCNN and GCNN on CIFAR-10 test data.
Recall that for two subspaces V. V.W, the first principal angle defined the minimum angle between two unit vectors E V, W1 W. The second principal angle is the minimum angle between unit vectors 02 V,w, W2 E W, with U2 V1 and w2 w1. The other principal angles are defined similarly.
Table 1: Principal angles between 5op-5 SVD-subspace of gradient directions of test points, and the Top-5 SVD-subspace of invariant directions on test points (small 2 rotations) respectively.
Dataset
Model
CIFAR- 10
ResNet18 89.99958413
89.82902968
89,52802116
98.91712082
88. 64889062
15

---

Under review as a conference paper at ICLR 2020
APPENDIX D PROOF DETAILS
Theorem 5.6.1 (General covariance estimation) from Vershynin (2018) bounds the spectral norm of covariance matrix estimated from a small number of samples as follows.
Theorem 3. Let X be random vector in Rd d 2 2. Assume that for some K 2 1, ||x|| S
[ ) 1/2
almost surely. Let Σ = E [xX*] be the covariance matrix of X and Em 1 DE (1 X be the estimated covariance from i.i.d. samples X2.X2,.... m positive integer m, we have
K
E
Xm Thenffor every
1 211 SC crsccvV
(VK'dlogd K2dlogd
|
m
m
for some positive constant C and ||ا||| being the spectral norm (or the top eigenvalue) of E.
Note that using m = ( Po d) we get E [[] E122 S E ||
A tighter version of Theorem 5.6.1 appears Remark 5.6.3, when the intrinsic dimension tr (E) |||| < d.
Theorem 4. Let X be random vector in RRd and d 2 2. Assume that for some K 2 1, ||X||2 S 1/2
almost surely. Let Σ = E [xx*] be the covariance matrix of X and Em -i=1 X be the estimated covariance from m ii.d samples X1, X2,... Xm Then for every positive integer m, we have
K
E
[
1 EE
(V d
logl
E II E1
E1
m
m
for some positive constant C and ||ا|| being the operator norm (or the top eigenvalue) of
Note that using m = O ( log d) we get [||2m. 211 S 2|
Theorem 4.5.3 (Weyl's Inequality) from Vershynin 2018) upper bounds the difference between i-th cigenvalues two symmetric matrices and using the spectral norm of A B.
Theorem 5. For any two symmetric matrices A and B in Rdxd (A, - X,BB) s ||A-B] BlI, where (A) and (B) are the i-th eigenvalues of A and B, respectively.
In other words, the spectral norm of matrix perturbation bounds the stability of its spectrum.
Here is special tioned in Vershynin| (2018).
of Theorem 4.5.5 (Davis-Kahie Theorem) and its immediate corollary men-
Theorem 6. Let A and be symmetric matrices Raxd Fix E [d] and assume that the largest eigenvalue of A is well-separated from the rest of the spectrum, that is, >1 )2(A) 2 > 0. Then the angle between the top eigenvectors (A) and vn (B) of and B, respectively, satisfies sin 0 || B Bl2/ /5.
As an easy corollary, it implies that the top eigenvectors U1 (A) and vn (B) are close to each other up sign, namely, there exists [ such that
23/2 | Bl1
|| (A) - s (B) <
APPENDIX E ERROR RATES OF CIFAR- ON GCNN'S
For fair comparison, the GCNN configuration used for CIFAR-10 is the same that of ResNet18 8, except that the operations going from one layer to the next are replaced by equivalent GCNN oper- ations, as given in Cohen & Welling (2016).
16

---

Under review as a conference paper at ICLR 2020
0.9
100
100
0.0
500
0.
0.6
0.6
Ertrr Rate
0.4
0.2
0.1
10
0.8
100
0.7
Entor Rade
0.5
02
0.1
Perturbation
Figure 11: On CIFAR-10, GCNN: error rate norm of perturbation along top singular vector of attack directions on 100/500/10000 samples, (top left) Gradient (top right) FGSM (bottom) Deep- Fool
Gradient-500
Gradient:10k
1.
FGSM-500 DeepFool-500
DeepFool-10k
Singuarr Value
1.4
Singular Volue
1.2
0.6
0.4
Index
Figure 12: On CIFAR-10, GCNN/ResNe118, Singular values of attack directions over a sample of (left) 500 and (right) 10,000 test points.
Table Principal angles between Top-5 SVD-subspace of gradient directions of test points, and the Top-5 SVD-subspace of invariant directions on test points (small 2 rotations) respectively.
Dataset
Model
CIFAR-10
GUNN/ResNet18
$9.90921558
89.82466625
$9.59947099
89.31534574
88.96739966
17

---

Under review as a conference paper at ICLR 2020
APPENDIX F TABLES REQUESTED BY REVIEWERS
Table 3: On ImageNet validation, VGGI6 vs VGG19 ResNet50 vs M-DFFF: fooling rate norm of perturbation in tabular form. Attacks constructed using 64 samples.
Network
Vector (using 64 samples)
Norm
Norm 16
Norm 32
Norm 64
VGGI6
SVD-Gradient
0,05720
0.10870
0.19470
0.34100
VGG16
SVD-FGSM
0.04450
0.09182
0.17364
0.31124
VGG16
SVD-DeepFool
0.05894
0.11334
0.20374
0.36744
VGGI6
M-DFFF
0.05720
0.11192
0.20426
0.36154
VGGI9
SVD-Gradient
0.05728
0.11490
0.20892
0.37672
VGG19
SVD-FGSM
0.04366
0.09052
0. 0.16956
0.30300
VGG19
SVD-DeepFool
0.05376
0.10492
0.18602
0.32786
VGG19
M-DFFF
0.05256
0.10542
0.18528
0.31764
ResNet50
SVD-Gradient
0.04604
0.09334
0.16008
0.27532
ResNet50
SVD-FGSM
0,03600
0.07740
0. 0.14062
0.23984
ResNet50
SVD-DeepFool
0,04742
0.09400
0.16580
0.28880
ResNet50
M-DFFF
0.04602
0.09398
0.16318
0.28242
18


Reviews:
[{"review_id": "SyliF_gxcH", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents an observation that one can use the top singular vector of a matrix consisting of adversarial perturbation (Gradient attack, FGSM attack, or DeepFool) vectors of a subset of data as a universal attack (applying the same perturbation to all inputs and fools a large fraction of inputs). The paper gives a theoretical justification of their method using matrix concentration inequalities and spectral perturbation bounds.\n\nStrengths:\n- A simple and effective technique to fool a large fraction of examples leveraging the observation that only a small number of dominant principal components exist for input-dependent attack directions.\n\n- Clean theoretical justification of the performance of the proposed methodology.\n\n- I also like the observation and the generality, simplicity, and theoretical proof of the proposed universal attack algorithm SVD-Universal.\n\nWeaknesses:\n- Performance seems to be inferior to previous methods e.g. Khrulkov & Oseledets 2018. The paper does not give a comparison between SVD-Universal and (p,q)-SVD.\n\n- Although the author gives a justification of why they do not compare with (p,q)-SVD, I still like to see a comparison between the two methods such that we can have a better idea about what is the potential performance loss by using the SVD-Universal when compared with (p,q)-SVD.\n\n- It is not clear to me how the authors build the matrix corresponding to the universal invariant perturbations in sec 6. \n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I made a quick assessment of this paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "HJxMVJT3KS", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper studied the problem of universal adversarial attack which is an input-agnostic perturbation. The authors proposed to use the top singular vector of input-dependent adversarial attack directions to perform universal adversarial attacks. The authors evaluated the error rates and fooling rates for three attacks on standard benchmark datasets.  \n\n- The paper is generally well-written and easy to follow. My main concern towards this paper is about the experiments part from several aspects. First, the proposed method needs quite large L2 norm (50 on ImageNet) to work, while common adversarial attack experiments on ImageNet are usually conducted with L2 perturbation strength of 5 or less. I totally understand that performing universal attack would be much more difficult, yet having such loose L2 norm constraint still seems impractical. Second, the authors did not compare with any other baselines such as  (Moosavi-Dezfooli et al. 2017a) arguing that their universal attack is different for different perturbation strength and pixels are normalized. I do not think normalized pixel will be a problem as you can simply scale the perturbation strength accordingly. And because (Moosavi-Dezfooli et al. 2017a) uses different attack vectors for different perturbation strength, some comparison between these two types of universal attacks should be presented in order to mark the difference and demonstrate your advantages. I would suggest the authors to compare with several mentioned baselines in the paper to show the superiority of the proposed method.\n\n- Theorem 1 seems interesting, yet it needs a special assumption. The authors argue that this is a reasonable assumption in a small neighborhood of x. I wonder if the authors could conduct some demonstrative experiments to verify this? Because the definition of S_x depends on the attack function, does it mean that the assumption need to be held for any attack function? Also regarding the choice of \\delta, it seems that \\delta is different for different x? If so, since u is also depend on \\delta, this attack vector seems not universal?\n\n\nDetailed comments:\n- In proof of Theorem 1, all S should be G?\n- In proof of Theorem 2, how to get \\|v - \\hat v\\|_2 \\leq \\epsilon/(\\gamma - \\epsilon)? Directly applying the Theorems seems to get \\epsilon / (\\gamma) only?\n\nDepending on whether the authors can address my concerns, I may change the final rating.\n\n\n======================\nafter the rebuttal\n\nI thank the authors for their response but I still feel that the assumption is not well-justified and there is still a lot to improve in terms of experiments. Therefore I decided to keep my score unchanged.  ", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}, {"review_id": "ByxmcCp6tB", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents a universal adversarial attack, which firstly conducts existing gradient-based attacks on the sample images and then applies SVD on the perturbations from those attacks. The universal attacks are the right singular vectors of the SVD.  The experiments are conducted on attacking VGG and ResNet. In addition, theoretical analysis is also provided in the paper.\n\nCompared with instance-wise attacks, universal attacks are relatively rare. The idea of this paper is intuitive but I feel that it is highly related to the one in Khrulkov & Oseledets (2018). The latter finds singular vectors with the gradients of the hidden layers of the targeted classifier. In general, the instance-wise attacks such as FGSM and Gradient are essentially based on gradients of the classifiers, as well. Therefore, given Khrulkov & Oseledets (2018), I would consider the novelty of this paper is not large enough, although I can see that the proposed may be more efficient.\n\nIn addition to attacking raw classifiers, I would also expect the comparisons with defence methods against universal attacks, such as the one in [1].\n\nMinors:\n\nIt is a bit hard to compare the performance across different methods in Figure 1. I would suggest using tables to give a clearer comparison.\n\nOverall, I think the paper stands on the borderline. \n\n[1] Akhtar, Naveed, Jian Liu, and Ajmal Mian. \"Defense against universal adversarial perturbations.\" Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018.", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes to get universal adversarial examples using few test samples. The approach is very close to the Khrulkov & Oseledets, and the abstract for some reason claims that it was proposed independently, which looks like a very strange claim. Overall, all reviewers recommend rejection, and I agree with them.
Decision: Reject