Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attack
Authors: Francesco Croce
Matthias Hein
Venue:  (2020)
Keywords: adversarial attacks
adversarial robustness

Abstract: The evaluation of robustness against adversarial manipulations of neural networks-based classifiers is mainly tested with empirical attacks as the methods for the exact computation, even when available, do not scale to large networks. We propose in this paper a new white-box adversarial attack wrt the $l_p$-norms for $p \in \{1,2,\infty\}$ aiming at finding the minimal perturbation necessary to change the class of a given input. It has an intuitive geometric meaning, yields quickly high quality results, minimizes the size of the perturbation (so that it returns the robust accuracy at every threshold with a single run). It performs better or similarly to state-of-the-art attacks which are partially specialized to one $l_p$-norm.

Under review as a conference paper at ICLR 2020
MINIMALLY DISTORTED ADVERSARIAL EXAMPLES WITH A FAST ADAPTIVE BOUNDARY ATTACK
Anonymous authors Paper under double-blind review
A ABSTRACT
The ovaluation of robustness against adversarial manipulations of neural nctworks-basee classifiers is mainly tested with empirical attacks as the methods for the exact computation, even when available, do not scale to large networks. We propose in this paper new white-box adversarial allack wrt the 1p-norms for p E (1, 2, 20) aiming at finding the minimal perturbation necessary to change the class of a given input. It has an intuitive geometric meaning, yields quickly high quality results, minimizes the size of the perturbation (so that it returns the robust accuracy at every threshold with a single run). It performs better or similarly to statc-of-tho-art attacks which are partially specialized to one (n-norm.
INTRODUCTION
The finding of the vulnerability of neural networks-basse classifiers to adversarial examples, that small perturbations of the imput able to modify the decision of the models, started fast development of a variety of attack algorithms. The high effectiveness of adversarial attacks reveals the fragility of these networks which questions their safe and reliable use in
the real world, espocially in safety critical applications. Many defenses have been proposed
to fix this issue (Gu & Rigazio 2015 Zheng et al. 2016 Papernot et al.] 2016
Huang
el al.] 2016 Bastani et. al. 2016: Madry et al.] 2018). but with limited success, 8s new
more powerful allacks showed [Carlini & Wagner 2017b; Athalye el. al.
2018 Mosbach
al. 2018 In order trust the decision of a model,
necessary to evaluate the exact
adversarial robustness. Although this is possible for ReLU U networks (Katz et al. 2017 Ljeng ltt al. 2019) these techniques do not scale to commonly used large networks. Thus, the
robustness is evaluated approximating the solution of the minimal adversarial perturbation
problem through adversarial attacks.
One can distinguish attacks into black-box (Narodytska & Kasiviswanathan 2016 Brendel
et al. 2018 Su al. 2019), where one is only allowed to query the classifier, and white-box attacks, where one has full control over the network, according to the attack model used to create adversarial examples (typically some I, no-or but others have become popular as well, e-g. Brown et al. (0177) Engstrom et al. (2017); Wong et al.), whether they aim at the minimal adversarial verturbation (Carlini & Wagner 201 7a. CChen et al. 2018 Croce
let al. 2019) or rather any perturbation below a threshold (Kurakin al. 2017 Madry al.
2018 Zheng et al.] 2211)) if they have lower (Moosavi-Dezzel
et al. 2016 Modas et al.
2019) or higher (Carlini Wagner 2017a Croce al. 2019) computational cost. Moreover, it is clear that due to the non-convexity of the problem there exists no universally best attack (apart from the exact methods), since this depends on runtime constraints, networks architecture, datasel, etc. However, our goal is to have an altack which performs well under a broad spectrum of conditions with minimal amount of hyperparameter tuning.
In this paper we propose new white-box attacking scheme which performs comparably or better than established attacks and has the following features: first, tries to produce adversarial samples with minimal distortion compared to the original point, measured wrt the no-nssms with E 21, po). Respect to the quite popular PGD-attack of Madry et al. (2018) this has the clear advantage that our method does not need to be restarted for every threshold e one wants to evaluate the success rate of the attack with perturbations constrained to be in (6 E Rdd ||ō1, & <c). Thus it is particularly suitable to get a complete

---

Under review as a conference paper at ICLR 2020
picture on the robustness of classifier with low computational cost. Second, it achieves fast good quality in terms of average distortion or robust accuracy. At the same time we show that increasing the number of restarts keeps improving the results and makes it competitive with the strongest available attacks. Third, although it comes with a few parameters, these mostly generalize across datasets, architectures and norms considered, so that have almost off- shethe-sheeff method. Most importantly, unlike PCD and other methods, there is no step size parameter which potentially has be carefully adapted to every new network.
FAB: A FAST ADAPTIVE BOUNDARY ATTACK
We first introduce minimal adversarial perturbations, then recall the definition and propertics of the projection wrt the 1p-norms of a point on the intersection of a hyperplanc and box constraints, they are an essential part of our attack. Finally, we present our FAB-atlack algorithm to generate minimally distorted adversarial examples.
2.1 MINIMAL ADVERSARIAL EXAMPLES
Let f: Rd + RK be a classifier which assigns every input Rd (with d the dimension of the input space) to one of the K classes according to arg max fz().. In many scenarios r=1...,K
the input of has to satisfy specific set of constraints C, e.g. images are represented as clements of [0, 1]d. Then, given a point E I&d with true class c, we define the minimal adversarial perturbation for wrt the 1y-norm
umin = arg min ||ö|pp s.th. dER"
max fi( + folx 8), Z+8EC. lyf.
(1)
The optimization problem ① is non-convex and NP-hard for non-trivial classifiers (Ktt al. (007)) and, although for some classes of networks it can be formulated as mixed-integer program (see Tieng al. (2019)), the computational cost of solving is prohibitive for large, normally trained networks. Thus, ōmin.p is usually approximated by an attack algorithm, which can be soen as a hcuristic to solve (1) We will see the experiments that current attacks sometimes drastically overestimate [minnpllp and thus the robustness of the networks.
2.2 PROJECTION ON A HYPERPLANE WITH BOX CONSTRAINTS
Let w E Rd and b E R be the normal vector and the offset defining the hyperplane T: (w, x) +b=0. 0. Let x E Rd, we denote by the bor-construined projection wrt the lp- fp-norm of x on π (projection onto the intersection of the box C = {a E Rd 4 S 24 S [u} and the hyperplane π) the following minimization problem:
arg min 2 - =II, #E
s.th. (w, 2) + b 0, 4 2, S Ui == 1. d.
(2)
where 4.th u4 E R are lower and upper bounds each component of For 2 the optimization problem (2) is convex. Hein & Andriushchenko) (2017) proved that for p E (1,2, oo) the solution can be obtained in Old log d) time, that. is the complexity of sorting vector of d elements, as well as determining that it has no solution.
Since this projection is part of our iterative scheme, we need to handle specifically the case of (2 being infeasible. In this case, defining sign( x) + +b, we instead compute
= arg min p((w, 2) + b) s.th. 4 S 2, S ui 1,...,d,
(3)
*ER
(4 if pw; > 0, U2 if pw:; < 0,,
whose solution is given componentwise, for every i = 1,... d, by 24
Iz if w. 0
Assuming that the point x satisfies the box constraints (as it will be in algorithm), this is equivalent to identifying the corner of the d-dimensional box defined by the componentwise constraints on closest to the hyperplane Notice that if (2) is infeasible then the objective
2

---

Under review as a conference paper at ICLR 2020
function of 33 stays positive and the points and are strictly contained in the same of the two halfspaces divided by æ. Finally, we define the operator
projp (z, n, C) -
x²
if Problem ( is feasible
(4)
else
yiclding the point which gets as close as possible to x without violating the box constraints.
2.3 FAB ATTACK
We introduce now our algorithm to produce minimally distorted adversarial examples, wrt any f--norm for 21, oo,, for given point Zorig initially correctly classified by f as class c. The high-level idea is that we use the linearization of the classifier at the current iterate 200), compute the box-constrained projections of xt)) respectively Zorig onto the approximated decision hyperplane and take a convex combinations of these projections depending on the distance of zd and Forig to the decision hyperplane, followed by some extrapolation step. We explain below the geometric motivation behind these steps. The attack closest in spirit is DeepFool (Moosavi-Dezzoolli et al. (2016)) which is known to be very fast but suffers from low quality. DeepFool just tries to find the decision boundary quickly but has no incentive to provide a solution close to Zorig. Our scheme resolves this main problem and, together with the exact projection we use, lcads to principled way to track the decision boundary (the surface where the decision of f changes) close to Torig.
f was a linear classifier then the closest point to x:()) on the decision hyperplane could be found in closed form. Although neural networks are highly non-linear, ReLU networks (neural networks which use ReLU as activation function) are piecewise affine functions and thus locally linearization of the network is exact description of the classifier. Let I o c, then the decision boundary between classes I and c can be locally approximated using a first order Taylor expansion at (x()) by the hyperplane
m= : f0( - (c( (Vii(a()) VSc(x=")), ===(("))===()))))) -x(==)))))))) O.
(5)
Morcover the di-ttanec dø (1) of 2(() to mi is given by
lf((((0) - fa(x()))
dp(m,,(00) = [ Vfc(>())1,''
with
p + = 1.
(6)
Note that if d, .)) 0 then a(() belongs to the true decision boundary, Moreover, if the local linear approximation of the network is correct then the class a with the decision hyperplane closest to the point (1)) can be computed as
1 - fc
arg min
Vi((2())
Lefe
Thus, given that the approximation holds in some large enough neighborhood, the projection proj (x(0), T,, C) of z(0) onto n. lies on the decision boundary (unless (2) is infeasible).
Biased gradient step: The itcrative algorithm (i(i-) (x(),, T.. C) would be similar to DeepFool except that our projection operator exact whereas they project onto the hyperplane and then clip to 1] This scheme is not biased towards the original larget point Torig, thus it goes typically further than necessary to find a point on the decision boundary basically the algorithm does not aim at the minimal adversarial perturbation. Thus we consider additionally proip(Forig, Tay C) and use instead the iterative step, with
(0))) Torig, defined as
((+1) = (1-a) (r)), T., C') + a proj,( n., R.,C). C)2
which biases the step towards Forig (sce Figure 1. Note that this is convex combination of two points a. and in C and thus also ((++1) lies n. and is contained in C. As wish
3

---

Under review as a conference paper at ICLR 2020
30 2
21
Figure 1: Visualization of FAB-attack scheme, with on the left the case r) = 1, on the right > 1. In blue we represent the next iterate x(++1) one would get without any bias toward the original point Torig, in green the effect of the bias we introduce and red the 2(++1) obtained with our scheme in (1O). We see that our algorithm tends to stay closer to the original point compared to the one with an unbiased gradient step.
a scheme with minimal amount of parameters, we want to have an automatic selection of based on the available geometric quantities. Let
g = proj,(r' Tes x0) and oong = projp (Iorig Ts. () Zorig.
Note that 1| and | are the distances of x(0) and Torig to n5 (inside C). We propose to use for the parameter a the relative magnitude of these two distances, that
||8(0)
min
Omax
[0,
(9)
||500] 1|
+
g(() (orii
The motivation for doing so is that if (+) is close to the decision boundary, then we should stay close to this point (note that T, is the approximation of f computed at x(1) and thus it is valid in small neighborhood of 2(1), whereas Zorig is farther away). On the other hand we want to have the bias towards Torig in order not to go too far away from Forig- This is why a depends on the distances of !9 and Torig to but we limit it from above with Omaxx Finally, we use a small extrapolation step we noted empirically, similarly to Gloosavi-Dezfood et al. (2016). that this helps to cross faster the decision boundary and get an adversarial sample. This leads to the final scheme:
(1+1)) proje (1 ((l-a)(z"))
770(0) + ax T6
(10)
where chosen as in 7 2 1 and proje just the projection onto the box which can be done by clipping. In Figure we visualize the scheme: in black one can see the hyperplane
and the vectors ((0) and g(0) in blue the step we would make going to the decision boundary with the DeepFool variant, while in red the actual step we have in our method. The green vector represents instead the bias towards the original point we introduce. On the left of Figure we use 1, while on the right we use overshooting > 1.
Interpretation of proj, (Forig: Ta, C): The projection of the target point onto the inter- section of T. and C is defined as
arg min ||z - Zoriellp s.th. (w, + b 6=0, 0, 4, 4.2 S u. ZERR
Note that replacing z by (+)) + 5 we can rewrite this as (w, b) 4, S x, + 6 us.
arg min dERd
(x((9) - soris] s.th.
This be interpreted the target point Forig so that (0)) + 8 lics on the intersection of the (approximate) decision hyperplane and the box C. This point of view on the projection p Way C)) again
the minimization of the distance of the next iterate (0( + 8 to
justifies using convex combination of the two projections in our iterative scheme in 10
4

---

Under review as a conference paper at ICLR 2020
Backward step: The described scheme finds in few iterations adversarial perturbations. However, we are interested in minimizing their norms. Thus, once we have a new point 1(tt1, we check whether it is assigned by to class different from c. In this case, we apply
(++1)
F) Balit1, B (0,
(11)
that is we go back towards Zorig on the segment [x Torigls effectively starting again the algorithm at a point which is quite close to the decision boundary. In this way, due to the bias of the method towards Torig we successively find adversarial perturbations of smaller norm, meaning that the algorithm tracks the decision boundary while getting closer to Forig.
Final search: Our scheme finds points close to the decision boundary but often they are slightly off as the linear approximation is not exact and apply the extrapolation step with > 1. Thus, after finishing Niter iterations of our algorithmic scheme, we perform a last, fast step to further improve the quality of the adversarial examples. Let Fout be the closest point to Torig classified differently from say & c, found with the iterative scheme. It holds that (oot) fe(oo > O and (Zorig) ff((oorg)) O. This means that, assuming continuous, there exists point xx on the segment [Fout; Forig] such that fa(x*) - fe(x") = 0 and || Forigll, < [|xontt Torig llp Tf linear
fo foout) fc((o x Zout (xout) fe(vont) + fa (Foria) fe (oori))))
(Fout Torlg).
(12)
Since f is typically non-linear, but close to linear, we compute iteratively for few steps fa(wout) - fo(Fout)
Tremp
Tout
[a(Honut) - fe( + [((ori)) - So (a(Foria) (Fout - Zorig),
(13)
each time replacing in 13 Fout with Tremp if fe (Ftemp) - So (Ftem))) > 0 or Forig with Fremp if instead fa (Ztemp) ftemp) < O. With this kind of modified binary search one can find a better adversarial sample with the cost of a few forward passes of the network.
So far all the steps are deterministic. To improve the results, introduce the option of random restarts, that is x(()) is randomly sampled in proximity of
Random restarts:
Torig instead of being Forig itself. Most attacks benefit from random restarts, e.g. Madry
let al. 2018): Zheng et al. (2019). especially dealing with gradient-masking defenses (Mosbach al. 2018)) as allows wider exploration of the input space. We choose to sample from the b -pheee centered in the original point with radius half the 4--norm of the current best adversarial perturbation (or a given threshold if no adversarial example has been found yet).
Computational cost: Our attack, in Algorithm [ consists of two main operations: the computation of and its gradients and solving the projection (2). We perform, for each iteration, a forward and a backward pass of the network in the gradient step and a forward pass in the backward step. The projection can be efficiently implemented to run in batches on the GPU and its complexity depends only the input dimension. Thus, except for shallow models, its cost is much smaller than the passes through the network. We can approximate the computational cost of our algorithm by the total number of calls of the classifier
Niter X Nrestarts X (2 x forward passes + 1 x backward pass).
(14)
One has to add the forward passes for the final scarch, fixed to 3, that happens just once.
2.4 COMPARISON TO DEEPFOOL
The idea of exploiting the first order local approximation of the decision boundary is not novel but the basis of one of the first white-bol box adversarial attacks, DeepFool (DF) from Moosavi-Dezfooli et al. 2206)). While DF and our FAB-attack share the strategy of using linear approximation of the classifier and projecting on the decision hyperplancs, we want to point out many key differences: first, DF does not solve the projection (2) but its simpler version without box constraints, clipping afterwards. Second, their gradient step does not have any bias towards the original point, that is equivalent to in (10). Third, DF does
5

---

Under review as a conference paper at ICLR 2020
Algorithm 1: FAB-attack
Input :Zorig original point, original class, Nrestarts: Niter mmax 3, 17, c, Output: :oott adversarial example
< +00
for j I...., Nestarts do
if then 20() < Torigi
else (0) randomly sampled 8.th. ||x(0) Forigll, min(ure)/2
5
for == 0 - do
8 - arg min f((aa")) [ thf
8t proip((!) T3. 8(0)
orig projo (Forigg To: C)
compute in Equation
9
x(++1) projc (1 a) (x(1) + n) + (Forig ro
(b+1) is not classified in then
11
if [2(it]) Eoriy lp then
12
x(+11
Tout || Enrigll
end
((++) &- (1 - B)Forgg + Ball+1)
end
18
end
19 end
perform steps of final search Fout in 13
MNIST -T
accuracy
0.6
DeepFool
mmxx -0.1. backward
0.4
0 no restarts
max
max =0 0.1. no restarts
0.2
=0. 100 restarts
0.1, 100 restarts (FAB)
82
Figure 2: Ablation study to DeepFool for attacks. The introduction of the convex combination (Qmax = 0.1, no backward step) already improves over DeepFool. Moreover, one does our full approach, the case amax 0 (can seen as an improved iterative DeepFool) is worse than Omax = 0.1 with the same number of restarts. In the plots we show the robust accuracy as a function of the threshold under the different attacks on the AT model on MNIST.
not have any backward step, final search or restart, as it stops soon a misclassified point is found (its goal is to provide quickly an adversarial perturbation of average quality). ablation study of the differences to DF in Figure 2 where show the curves of the robust accuracy as a function the threshold e (lower better). We present the results of DeepFool (blue) and FAB-attack with the following variations: Qmax 0.1 and no backward step (magenta), amax = (that is no bias in the gradient step) and no restarts (light green), Omax 0.1 and restarts (orange), Omax and 100 restarts (dark
We perform

---

Under review as a conference paper at ICLR 2020
green) and dmax = 0.1 and 100 restarts, that is FAB-attack, (red). We can see how every addition we make to the original scheme of DeepFool contributes to the significantly improved when compared to the original DeepFool.
performance of
EXPERIMENTS
We run experiments on MNIST, CIFAR-10 (Krizhessky et al.) and Restricted ImageNet (Tsipras et al. (2019)). For each dataset we consider a naturally trained model (plain) and two adversarially trained ones in Madry et al. (2018), , one to achieve robustness
Models:
wrt the -nor (TT and the other wrt the 12-norm (12-AT) (sec A.1
Attacks: We compare the performances of FAB-attack to those of attacks representing
the state-of-the-ar in each norm: DeepFool (DF) Moosavi-Dezzooli et al. (2016)). Carlini-
Wagner lg-attack (CW) (Carlini & Wagner (017a ), Linear Region lt-Attack (LRA) (Croce
et al.
(019)),, Projected Gradient Descent on the cross-cntropy function (PGD) (Krakin
ct al.
2017 Madry et al., 2018 Tramèr & Bonch 2019). Distributionally Adversarial Attack
(DAA)
Zheng et al. (2019)) Sparselool (SF) (Modas et al. (2019)), ), Elastic-net Attack
[EA)
Chen et al. (2018)). We use DF from Rauber et al. (2017), CW and EAD as in
Papernol et al. (2017 DAA and LRA with the code Irom the original papers, while we
reimplemented SF and PGD. For MNIST and CIFAR-10 we used DAA with 50 restarts, PGD and FAB with 100 restarts. For Restricted ImageNet,
used DAA, PGD and FAB with 10 restarts (for l we used 5 restarts, since both methods benefit from more iterations). Moreover, we could not use LRA since it hardly scales to such models and CW and EAD for compatibility issues between the implementations of attacks and models. See A.2 for more details e.g. regarding number of iterations and other hyperparameters.
Evaluation metrics: The robust accuracy of a model at threshold is defined as the classification accuracy (in percentage) the model achieves when an attack is allowed to change every input of the test set with perturbations of 1p-norm smaller than in order to change the decision. Thus stronger attacks produce lower robust accuracies. For each model and dataset we fix five thresholds at which we compute the robust accuracy for each attack (we choose the thresholds to have values of the robust accuracy that cover the range between clean accuracy and 0). We evaluate the attacks through the following statistics: i) avg. rob. accuracy: the mean of all the values of robusi. accuracy given by the allack over all models and thresholds, ii) # best: how many times the attack achieves the lowest robust accuracy (it is the most effective), iii) avg. difference to best: for each model/threshol threshold we compute the difference between the robust accuracy of the attack and the best one across all the attacks, then we average over all models/threshhng thresholds, iv) max difference to best: as 'avg. difference to best', but with the maximum difference instcad of the average onc. In A.4 we report the average -o-norm of the adversarial perturbations given by the attacks.
We report the complete results in Tables to 13 of the Appendix, while we summarize them in Tables I (MNIST and CIFAR- 10 aggregated, as we used the same attacks) and 2 (Restricted ImageNet). Our FAB-attack achieves the best results in all statistics for every norm (with the only exception of 'max diff. to best" in (zo) on MNIST -IFAR-I,,, meaning that it is the most effective attack. In particular, while on Ix the 'avg. robust accuracy" of PGD is not far from that of FAB, the gap is large when considering la and 4, Interest ingly, the second best al tack, at least in terms of average robust accuracy, is different for every norm (PGD for Ls LRA for lN. EAD for 41), which implies that FAB outperforms algorithms specialized in the individual norms.
Results:
We also report the results of FAB-10, that is our attack with only 10 restarts, to show that FAB yields high quality results already with a low budget in terms of sine/computations cost. In fact, FAB-10 has 'avg. robust accuracy' better than or very close to that of the strongest versions of the other methods (see below for a runtime analysis, where one observes that FAB-10 is the fastest attack excluding DF and SF which however give much worse results). On Restricted ImageNet, FAB-attack gets the best results in all statistics for 4, while for lxx and 12, although PGD performs often better, the difference in 'avg. robust accuracy" is small, meaning that FAB performs mostly similarly PGD.
7

---

Under review as a conference paper at ICLR 2020
Table Performance summary of all attacks on MNIST and CIFAR-10 (aggregated). We report, for cach norm, 'avg. rob. acc.', the mcan of the values of robust accuracy across all the models and datascts, ## best', number of times the attack is the best onc, 'avg. diff. to best" and "max diff. to best', the mean and maximum differences between the robust accuracy of the attack and that of the best attack for each model/Ubreshold (on the first 1000 points for loe and (1, 500 for l2, of the lest sets). The numbers after the name of the attacks indicates the number of restarts used. In total we consider thresholds x 6 models 30 cases for each of the norms. *Note that for FAB-10 (i.e. with 10 restarts) the '# best" is computed excluding the results of FAB-100.
statistics on MNIST + CIFAR-10
1xc norm
DF
DAA-50
PGD-100
FAB-10
FAB-100
avg. rob, acc.
58.81
60.67
46.07
46.18
45.47
# best
0
8
12
13*
17
avg. diff. to best
14.58
16.45
1.85
1.96
1.25
max diff. to best
78.10
49.00
10.70
20.30
17.10
22-norm
CW
DF
LRA
PGD-100
FAB-10
FAB-100
avg. rob. acc.
45.09
56.10
36.97
44.94
36.41
35.57
# best
4
11
19*
23
avg. diff. to best
9.65
20.67
1.54
9.51
0.98
0.13
max diff. to best
65.40
91.40
13.60
64.80
8.40
1.60
no-n wo/ Madry's model avg. rob. acc. # best avg. diff. to best
CW
DF
LRA
PGD-100
FAB-10
FAB-100
40.85
49.18
40.25
42.95
39.98
39.57
14
18
1.44
9.78
0.84
3.54
0.57
0.16
max diff. to best
8.80
44.00
4.00
22.00
4.20
1.60
4-norm
EAD
PGD-100
FAB-10
FAB-100
avg. rob. acc.
64.47
35.79
49.51
33.26
29.46
# best avg. diff. to best
10*
17
35.31
6.63
20.35
4.10
0.30
max diff. best
95.90
58.40
74.00
21.80
1.60
h-norm wo/ Madry's model avg. rob. acc. # best avg. diff. best max diff. to best
SF
EAD
PGD-100
FAB-10
FAB-100
58.06
32.56
43.82
33.79
32.06
13
0
5*
12
26.36
0.87
12.12
2.10
0.36
53.10
4.80
31.90
3.90
1.60
In general, both average and maximum difference to best of FAB-attack are small for all the datasets and norms, implying that it does not suffer severe failures, which makes it efficient, high quality technique to evaluate the robustness of classifiers for all fo-nomms. Finally, we show in Table 4 that FAB-attack outperforms or matches the competitors in 16 out of 18 cases when comparing the average -n-norms of the generated adversarial perturbations.
Runtime comparison: DF and SF are definitely much faster than the others as their primary goal is to find as soon as possible adversarial examples, without emphasis on minimizing their norms, while LRA is rather expensive noted in the original paper. Below we report the runtimes (for 1000 points on MNIST and CIFAR-10, 50 on R-ImageNet) for the attacks as used in the experiments (if not specified otherwise, it includes all the restarts). For PGD and DAA this the time for evaluating the robust accuracy at thresholds, while for the other methods single run is sufficient to compute all the statistics.
MNIST: DAA-50 11736s, PGD-100 3825s for loe/ /12 and PGD-100 14106s for 4 CW 944s, FAD 606s, FAB-10 161s, FAB-100 1613s. C-FAR-10: DAA-50 11625s, PGD-100 31900s
8

---

Under review as a conference paper at ICLR 2020
Table 2: As in Table D statistics of the performance of different attacks on Restricted ImageNet (on the first 500 points of the validation set). In total we consider 5 thresholds x 3 models 15 cases for each of the 3 norms.
statistics on Restricted ImageNet.
Lxc lx-norm
12-norm
41-norm
DF
DAA-
PGD-
FAB-
DF
PGD-
FAB-
EAD
PGD-
FAB-
10
10
10
10
10
5
5
avg. rob. acc.
35.61
38.14
26.91
27.83
45.69
31.75
33.21
71.31
40.64
38.12
best
0
0
12
avg. diff. best
8.75
11.57
0.04
0.96
13.99
0.04
1.53
33.52
2.85
0.33
max diff'. best
14.60
37.20
0.40
2.00
25.40
0.60
3.40
59.00
6,20
2.40
for 122 and 70110s for 1, CW 3691s, EAD 3398s, FAB-10 1209s, FAB-100 12093s. R- ImageNet: DAA-10 6890s, PGD-10 4738s for / and PGD-5 24158s for 4. FAB-10 2268s for l2//22 and FAC-5 3146s for (note that different numbers of restarts/itcration for la are used on R-ImageNet). PGD needs a forward and a backward pass of the network for each iteration. Thus it is given 1.5 times more iterations than FAB, so that overall they have same budget of passes (we assume here that forward and backward passes take the same amount of time). In Appendix B.2 compare PGD-1 versus FAB-1 one iteration of PGD, 3 passes are one itcration of FAB the plots show 300 passes mcaning 150 iterations of PGD and 100 iterations of FAB) that the comparison fair concerning runtime in order to compare the performance of PGD-1 and FAB-1 as function of runtime. If one considers just the performance up to 20 passes (10 iterations PGD, 7 iterations FAB) then FAB outperforms PGD in 18 out of 27 cases. However, one also observes that there is no general superiority of one method. For both PGD-1 and FAB-1 there are cases where the method requires the full amount of 300 passes to get to good performance whereas the other method achieves it with significantly less iterations/pann
function of the number of passes (2 passes
CONCLUSION
In summary, our goometrically motivated FAB-attack outperforms in terms of runtime and on average in terms of quality all other high quality state-of-the-art attacks and can be used for all p-norms p E {1, 2, 3 which not the case for most other methods.
REFERENCES
A. Athalye, N. Carlini, and D. A. Wagner. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In JCML, 2018.
O. Bastani, Y. Ioannou, L. Lampropoulos, D. Vytiniotis, A. Nori, and A. Criminisi. Mcasuring neural net robustness with constraints. In NeurIPS, 2016.
W. Brendel, J. Rauber, and M. Bethge. Decision-based adversarial attacks: Reliable attacks against black-box machine learning models. In ICLR, 2018.
T. B. Brown, D. Mané, A. Roy, M. Abadi, and J. Gilmer. Adversarial patch. In NewrIPS 2017 Workshop on Machine Learning and Computer Security, 2017.
N. Carlini and D. Wagner. Towards evaluating the robustness of neural networks. In JEEE Symposium on Security and Primacy, 2017a.
N. Carlini and D. Wagner. Adversarial examples are not easily detected: Bypassing ten detection methods. In ACM Workshop on Artificial Intelligence and Security, 2017b.
P. Chen, Y. Sharma, H. Zhang, J. Yi, and C. Hsich. Ead: Elastic-net attacks to deep neural networks via adversarial examples. In AAAI, 2018.

---

Under review as a conference paper at ICLR 2020
F. Croce, J. Rauber, and M. Hein. Scaling up the randomized gradient-fres adversarial attack reveals overestimation of robustness using established attacks. preprint, arXiv:1903.11355 2019.
L. Engstrom, B. Tran, D. Tsipras, L. Schmidt, and A. Madry. A rotation and translation suffice: Fooling CNNs with simple transformations. In NeurIPS 2017 Workshop on Machine Learning and Compuler Securily, 2017.
S. Gu and L. Rigazio. Towards deep neural network architectures robust to adversarial examples. In ICLR Workshop, 2015.
K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In CVPR, pp. 770 778, 2016.
M. Hein and M. Andriushchenko. Formal guarantees on the robustness of a classifier against adversarial manipulation. In NeurIPS, 2017.
R. Huang, B. Xu, D. Schuurmans, and C. Szepesvari. Learning with strong adversary. In ICLR, 2016.
G. Katz, C. Barrett, D. Dill, K. Julian, and M. Kochenderfer. Reluplex: An efficient smt solver for verifying deep neural networks. In CAV, 2017.
A. Krizhevsky, V. Nair, and G. Hinton. Cifar-10 (canadian institute for advanced research). URL httt:://www.. cs. toronto. tor.ott.eeaa/krrii// fatfa...mlll
A. Kurakin, I. J. Goodfellow, and Bengio. Adversarial examples the physical world. In ICLR Workshop, 2017.
A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Valdu. Towards deep learning models resistant to adversarial attacks. In ICLR, 2018.
A. Modas, Moosavi-Dezfooli,, and P. Frossard. Sparsefool: a few pixels make a big difference. In CVPR, 2019.
S.-M. Moosavi-Dezfoolis A. Fawzi, and P. Frossard. Deepfool: simple and accurate method to fool deep neural networks. In CVPR, pp. 2574 2574-2582, 2016.
M. Mosbach, M. Andrinshchenko, T. Trost, M. Hein, and D. Klakow. Logit pairing methods can fool maddient-based attacks. In NewrIPS 2018 Workshop on Security in Machine Learning, 2018.
N. Narodytska and S. P. Kasiviswanathan. Simple black-box adversarial perturbations for deep networks. In CVPR 2017 Workshops, 2016.
N. Papernot, P. McDonald, X. Wu, S. Jha, and A. Swami. Distillation as a defense to adversarial perturbations against deep networks. In IEEE Symposium on Security & Privacy, 2016.
N. Papernot, N. Carlini, I. Goodfellow, R. Feinman, Faghri, A. Matyasko, K. Ham- bardzumyan, Y.-L.. Juang, A. Kurakin, R. Sheatsley, A. Garg, and Y.-C. Lin. cleverhans v2.0.0: an adversarial machine learning library. preprint, 111.006610.00768, 2017.
Rauber, W. Brendel, and M. Bethge. Foolbox: python toolbox to benchmark the robustness of machine learning models. In ICML Reliable Machine Learning in the Wild Workshop, 2017.
J. Su, D. V. Vargas, and S. Kouichi. One pixel attack for fooling deep neural networks. preprint, 1rXiv:1710.088666
2019.
V. Tjeng, K. Xiao, and R. Tedrake. Evaluating robustness of neural networks with mixed integer programming. In ICLR, 2019.
F. Tramèr and D. Boneh. Adversarial training and robustness for multiple perturbations. preprint, arxiv:1904 13000, 2019.
10

---

Under review as a conference paper at ICLR 2020
D. Tsipras, S. Santurkar, L. Engstrom, A. Turner, and A. Madry. Robustness may be at odds with accuracy. In ICLR, 2019.
E. Wong, F. R. Schmidt, and J. Z. Kolter. Wasserstein adversarial examples via projected sinkhorn iterations. preprint, arXiv:1902.07966
Zheng, Y. Song, T. Leung, and I. Goodfellow. Improving the robustness of deep neural networks via stability training. In CVPR, 2016.
T. Zheng, C. Chen, and K. Ren. Distributionally adversarial attack. In AAAI, 2019.
11

---

Under review as a conference paper at ICLR 2020
A EXPERIMENTS
A.1 MODELS
The plain and AT models on MNIST are those available at https://githut com/ MadryLab/mnist chaallnggeleeeeeee and consist of two convolutional and two fully-connected layers. The architecture of the CIFAR-10 models has 8 convolutional layers (with number of filters increasing from 96 to 384) and dense layers, while on Restricted ImageNet use the models (ResNet-50 He et al. (2016) from Tsipras et al. (2019) and available al https://gi thtbb/ttthuuu..m The models on MNIST achieve the following clean accuracy: plain 98.7%, (x-AT 98.5%, 12-AT 98.6%. The models on CIFAR-10 achieve the following clean accuracy: plain 89.2%, 1x-AT 79.4%, 12-AT 81.2%.
A.2 ATTACKS
We use CW with 10000 itcrations and confidence 0, EAD with 1000 iterations, 4 de- cision rule and 8 = 0.05. In both cases we set the parameters to achieve minimally (wrt l2 for CW and 4 for EAD) distorted adversarial examples. We could not use these methods Restricted ImageNet since, to be compatible with the attack from Papernot et al. (017)) it would be necessary to reimplement from scratch the models of [Tipras al. (019)) done in https: / thub. wtt.://o.//eeeeeee htwpm//tssoofffofooooooo www./llowwweeee wwweaorroloo/cleeehhaannn. httpc/eeeeeeeseeeeeeeeeeeeeeeeee cleverhans/ model zoo zoo/madry lab challenges for similar situation.
For DAA
200 itcrations for MNIST, 50 for the other datascts and, given threshold step size of 9/30 for MNIST, 5/10 otherwise.
We perform PGD with 150 iterations, except for the case of i Restricted ImageNet where we use 450 iterations. For PGD wrt Too we use, given a threshold e, a step size of 6/10 in the direction of the sign of the gradient of the cross entropy loss, for PGD wrt l2 we perform at each iteration a step in the direction of the gradient of size s/1, for PGD wrt 4 we use the gradient step suggested in Tramer & Boneh (2019) (with sparsity levels of 1% for MNIST and 10% for CIFAR-10 and Restricted ImageNet), with size f/2. The above stepsize parameters s/4 for 12 and s/10 for 1x for PGD were obtained, by doing grid search for cach norm separately and using the values working best on average on MNIST and CIFAR-10. 10. In Appendix we discuss the influence the step-size and show that our chosen values perform best on average, Figure] 3
For FAB-attack we set 100 iterations, except for the case of l on Restricted ImageNet where we use 300 iterations. Moreover, use the following parameters for all the cases MNIST and CIFAR-10: Omax = 0.1, 17 1.05, 3 = 0.9. On Restricted ImageNet we set Qnux = 0.05, 7 1.3, 0.9. When using random restarts, FAB-attack needs value for the parameters c. It represents the radius of the ball around the original point inside which we sample the starting point of the algorithm, found (see Algorithm D. We use the values of reported in Table 3 Note however that the attack usually finds at the first run an adversarial perturbation small enough so that e in practice rarely comes into play.
least, until a sufficiently small adversarial perturbation is
Table 3: We report the values of used for sampling in case FAB-attack uses random restarts.
values of e used for random restarts
MNIST
CIFAR-10
Restricted ImageNet
plain
AT
l--AT
plain
be-AT
l--AT
plain
bT-AT
1--AT
Toc
0.15
0.3
0.3
0.0
0.02
0.02
0.02
0.08
0.08
l
2.0
2.0
2.0
0.5
4.0
4.0
5.0
5.0
5.0
4
40.0
40.0
40.0
10.0
10.0
10.0
100.0
250.0
250.0
12

---

Under review as a conference paper at ICLR 2020
A.3 COMPLETE RESULTS
In Tables to 13 we report the complete values of the robust accuracy, wrt either lee la or h,, computed by every allack, for 3 datasets, models for each dataset, thresholds for each model (135 evaluations overall).
A.4 FURTHER RESULTS
In Table we report the average 1y-norm of the adversarial perturbations found by the different attacks, computed on the originally correctly classified points on which the attack successful. Note that we cannot show this statistic for the attacks which do not minimize the distance of the adversarial example to the clean input (PGD and DAA). FAB-attack produces also in this metric the best results most of the cases, being the best for every model when considering Lo and l2, and the best in 4 out of 6 cases in I (lower values mean a stronger attack).
We report mean 1p-norm of the adversarial perturbations found by the attacks (when successful, excluding the already misclassified points) for every model.
Table 4:
average norm of adversarial perturbations
noom
DF
FAB
plain
0.078
0.066
MNIST
loo-at
0.508
0.326
12-al
0,249
0.170
plain
0.008
0.006
CIFAR-10
0.032
0.024
12-at
0.026
0.019
12-norm
DF
CW
LRA
plain
1.13
1.01
1.00
MNIST
loo-at
4.95
1.76
1.25
1.12
12-at
3.10
2.35
2.25
2.24
plain
0.28
0.21
0.22
0.21
CIFAR-10
loc-at
0.96
0.74
0.74
0.73
12-at
0.91
0.71
0.72
0.70
4--ormm
EAD
FAB
plain
6.38
6.04
MNIST
lac-at
8.26
3.36
Ig-at
12.18
12.16
plain
3.01
2.87
CIFAR-10
5.79
6.03
12-at
7.94
8.05
13

---

Under review as a conference paper at ICLR 2020
Table 5: Comparison of loo, 12- and /iattackk on a naturally trained model on MNIST. We report the accuracy in percentage of the classifier on the test set if the atlack is allowed to perturb the test points of lp-distance. The statistics are computed on the first 1000 points the test set for Lo and /1, 500 points for lz.
Robust accuracy MNIST plain model
metric
DF
DAA-
DAA-
PGD-
PGD-
PGD-
FAB-
FAB-
FAB-
50
10
100
10
100
0.03
93.2
91.9
91.9
92.0
91.9
91.9
92.0
92.0
92.0
0.05
76.7
76.0
74.9
74.6
76.8
76.1
loc
0.07
61.5
59.8
56.3
43.8
41.8
40.4
44.3
43.1
42.6
0.09
33.2
46.7
41.0
16.5
14.2
12.8
16.2
14.8
14.4
0.11
13.1
34.4
26.2
4.0
2.8
2.4
3.3
3.1
2.4
CW
DF
LRA
PGD- 1
PGD-
PGD-
FAB-
FAB-
FAB-
10
100
10
100
0.5
92.6
93.6
92.6
92.6
92.6
92.6
92.6
92.6
92.6
1
47.4
47.4
48.4
47.4
46.2
46.8
46.2
l2
1.5
8.8
19.8
7.8
9.8
8.8
8.2
7.8
7.2
7.0
2
0.6
1.8
0.2
1.2
0.6
0.6
0.2
0.2
0.2
2.5
0.0
0.0
0.0
0.6
0.2
0.2
0.0
0.0
0.0
SparseFool
EAD
PGD-
PGD- 10
PGD-
FAB-
FAB-
FAB-
100
10
100
2
95.5
93.6
94.4
93.9
93.7
94.2
93.7
93.5
4
76.7
79.8
77.5
76.6
4
6
75.8
48.1
57.4
52.2
49.3
54.5
47.2
43.3
8
60.3
26.6
46.7
36.3
31.6
31.3
25.3
22.4
10
43.8
11.2
40.0
27.4
22.1
13.2
9.8
8.4
Table 6: Comparison of loo;; l2- and 4-attacks on an Iso-robust model on MNIST. We report the accuracy the test set if the attack is allowed to perturb the test points of in disittance The statistics are computed on the first 1000 points on the test set for loo and 4,, 500 points for l2.
Robust accuracy of MNIST robust model
metric
DF
DAA- DAA-
PGD-
PGD-
PGD-
FAB-
FAB-
FAB-
50
10
100
10
100
0.2
95.2
94.6
93.7
95.0
94.2
93.7
94.6
94.4
93.9
0.25
91.1
93.1
91.8
91.7
loc
0.3
93.9
89.5
87.2
91.3
88.3
87.6
91.2
89.2
88.5
0.325
92.5
72.1
64.2
74.9
68.4
64.7
86.2
83.1
81.3
0.35
89.8
19.7
11.7
32.1
19.3
13.8
48.7
32.0
23.8
CW
DF
LRA
PGD-
PGD-
PGD-
FAB-
FAB-
10
100
10
100
1
88.8
94.6
73.6
92.2
90.8
89.8
84.2
70.6
65.4
1.5
77.6
93.0
25.8
86.0
81.2
77.0
47.0
20.6
12.2
1
2
64.4
91.6
3.2
77.8
67.0
57.8
15.6
1.8
0.2
2.5
53.8
89.6
0.4
68.2
49.6
36.4
3.8
0.0
0.0
3
46.8
84.6
0.0
59.8
29.6
13.4
1.4
0.0
0.0
SparseFool
EAD
PGD-
PGD- 10
PGD-
FAB-
FAB-
FAB-
100
10
100
2.5
96.8
92.2
94.1
93.7
93.6
90.1
74.3
56.9
5
76.0
90.9
88.9
85.8
39.4
17.6
4
7.5
96.4
49.5
85.2
81.4
79.0
82.6
19.8
5.0
10
96.4
27.4
80.2
73.5
70.3
78.4
11.9
2.4
12.5
96.4
14.6
74.9
65.6
58.7
74.5
7.7
0.5
14

---

Under review as a conference paper at ICLR 2020
Table 7: Comparison of loz, 1z- and 4-attacks on an l2-robust model on MNIST. We report the accuracy in percentage of the classifier on the test set if the attack is allowed to perturb the test points in l--distance. The statistics are computed on the first 1000 points on the test set for lo and 1, on 500 points for lz.
Robust accuracy MNIST /2-robust model
metric
DF
DAA-
DAA-
PGD-
PGD-
PGD-
FAB-
FAB-
FAB-
50
10
100
10
100
0.05
96.7
96.4
96.3
96.4
96.3
96.3
96.4
96.3
96.3
0.1
91.0
90.7
90.4
90.2
90.4
90.4
loc
0.15
86.4
74.3
72.3
74.6
73.2
72.4
74.0
72.3
72.0
0.2
73.8
34.5
27.2
36.2
29.8
26.5
34.1
28.2
24.4
0.25
55.1
1.5
0.9
2.6
1.5
1.0
1.9
0.9
0.8
CW
DF
LRA
PGD- 1
PGD-
PGD-
FAB-
FAB-
FAB-
10
100
10
100
1
92.6
93.8
92.6
93.0
93.0
93.0
92.6
92.6
92.6
1.5
84.8
87.2
83.8
83.4
83.4
83.6
83.6
l2
2
70.6
79.0
68.0
68.8
68.0
67.6
69.8
69.0
67.8
2,5
46.4
67.4
41.6
45.6
40.4
37.6
45.6
41.8
39.2
3
17.2
54.2
11.2
17.4
12.4
10.2
18.6
13.4
11.0
SparseFool
EAD
PGD-
PGD- 10
PGD-
FAB-
FAB-
FAB-
100
10
100
5
94.9
89.8
90.3
90.2
90.2
90.5
90.2
90.0
8.75
71.2
75.5
74.0
72.7
73.7
72.2
4
12.5
81.0
45.9
61.1
57.5
54.9
49.2
45.7
16.25
72.8
20.6
49.2
42.3
38.4
32.2
24.1
20.8
20
60.8
8.3
41.4
29.6
23.2
13.2
9.4
7.7
Table Comparison of loo-, lz- and 41attacks on naturally trained model on CIFAR-10. We report the accuracy in percentage of the classifier on the test set if the attack is allowed to perturb the test points of e in ln-distance. The statistics are computed on the first 1000 points the test set for loo and 41, on 500 points for lz.
Robust accuracy CIFAR-10 plain model
metric
DF
DAA-
DAA-
PGD-
PGD-
PGD-
FAB-
FAB-
FAB-
50
10
100
10
100
1/255
62.6
65.7
64.1
56.1
55.8
55.6
56.5
55.9
55.7
1.5/255
49.3
60.8
38.9
37.9
37.4
37.7
37.4
loc
4/255
37.3
58.5
24.3
23.3
22.9
21.9
21.2
2.5/555
26.4
61.2
56.3
16.2
14.8
14.0
13.2
12.0
11.8
3/255
19.0
60.2
54.4
10.7
9.2
8.6
7.4
5.8
5.4
CW
DF
LRA
PGD- 1
PGD-
PGD-
FAB-
FAB-
FAB-
10
100
10
100
0.1
69.4
72.2
69.0
68.4
67.6
67.6
68.4
68.4
68.4
0.15
55.4
62.6
55.0
54.6
53.8
53.8
54.6
54.0
53.8
12
0.2
43.4
51.2
43.4
43.8
42.8
42.0
42.4
42.0
41.8
0.3
21.6
33.8
22.0
24.8
24.2
23.6
21.6
20.8
20.6
0.4
9.4
20.8
9.8
18.2
16.2
15.4
9.6
8.2
8.0
SparseFool
EAD
PGD-
PGD- 10
PGD-
FAB-
FAB- 10
FAB-
100
100
2
72.1
54.7
54.9
54.4
53.9
55.5
52.2
50.8
4
58.6
24.1
30.0
29.1
28.9
25.1
22.4
4
6
45.6
8.9
18.8
18.6
18.4
17.0
10.5
8.1
8
34.3
3.0
14.2
14.1
14.0
7.8
3.8
2.5
10
27.2
0.7
12.9
12.5
12.3
4.7
1.5
1.0
15

---

Under review as a conference paper at ICLR 2020
Table Comparison of lo-, l2- and lı-attacks on an -o-bbutst model on CIFAR-10. We report the accuracy in percentage of the classifier on the test set if the attack is allowed to perturb the test points of lp-distance. The statistics are computed on the first 1000 points the test set for Lo and /1, 500 points for 12.
Robust accuracy of CIFAR-10 1x -oouusst model
metric
DF
DAA-
DAA-
PGD-
PGD-
PGD-
FAB-
FAB-
FAB-
50
10
100
10
100
2 2/255
66.8
66.9
66.3
65.5
65.5
65.5
65.8
65.8
65.7
2655
61.4
49.8
49.3
49.1
48.9
loc
2/555
42.9
63.1
58.4
38.0
36.9
36.6
35.4
34.7
34.6
#/255
32.9
61.2
56.3
30.5
30.0
29.6
23.8
23.5
23.3
10/255
24.5
59.8
54.1
25.8
23.7
22.4
15.4
14.7
14.4
CW
DF
LRA
PGD- 1
PGD-
PGD-
FAB-
FAB-
FAB-
10
100
10
100
0.25
64.6
67.0
64.4
64.4
64.4
64.4
64.8
64.6
64.4
0.5
48.4
53.0
48.8
49.0
48.4
48.0
48.4
48.2
l2
0.75
33.4
41.4
33.4
39.0
38.2
37.4
33.6
33.2
33.0
1
22.8
32.6
22.8
35.0
34.4
33.8
22.2
21.6
21.4
1.25
12.0
24.2
13.0
34.6
34.2
33.2
12.2
11.2
11.2
SparseFool
EAD
PGD-
PGD- 10
PGD-
FAB-
FAB-
FAB-
100
10
100
5
57.8
36.8
47.3
46.6
46.2
43.1
39.9
37.9
8.75
44.7
37.4
37.0
36.8
22.5
20.2
4
12.5
34.9
7.1
34.0
33.9
33.9
13.7
10.9
8.7
16.25
27.6
3.0
33.3
33.2
33.1
7.1
4.3
3.5
20
20.2
0.9
32.9
32.8
32.8
3.8
1.7
1.3
Table Comparison of 16o-, 12- and h-attacks on an 12-robust model on CIFAR-10. We report the accuracy in percentage of the classifier on the test set if the attack is allowed to perturb the test points of lp-distance. The statistics are computed on the first 1000 points the test set for loo and (1, 500 points for lz.
Robust accuracy CIFAR-10 gg-obustt model
metric
DF
DAA-
DAA-
PGD-
PGD-
PGD-
FAB-
FAB-
FAB-
50
10
100
10
100
2/255
64.1
67.2
66.3
62.6
62.5
62.4
62.7
62.6
62.6
4/255
65.0
62.8
45.3
45.0
44.2
loc
6/255
36.9
64.2
60.8
32.9
31.6
31.1
27.2
26.8
8/255
25.8
62.3
58.0
25.7
24.9
23.9
14.8
14.1
13.8
10/255
17.6
61.9
54.8
21.9
19.8
18.6
8.6
8.0
7.9
CW
DF
LRA
PGD-
PGD-
PGD-
FAB-
FAB-
FAB-
10
100
10
100
0.25
66.0
67.0
65.6
65.8
65.6
65.6
65.6
65.6
65.6
0.5
48.2
53.8
47.8
49.6
48.8
48.8
48.4
48.2
48.0
0.75
32.6
42.2
32.4
38.4
37.2
36.4
32.8
32.4
32.2
1
21.6
30.0
21.6
35.4
33.6
33.0
21.8
21.4
21.0
1.25
11.4
22.4
12.4
34.2
31.6
31.2
12.2
12.2
11.4
SparseFool
EAD
PGD-
PGD- 10
PGD-
FAB-
FAB-
FAB-
100
10
100
3
69.5
62.2
64.5
64.4
64.4
63.4
63.2
63.0
61.6
45.5
51.9
51.9
51.8
47.2
45.6
41
9
53.1
27.7
42.8
42.5
42.5
33.9
30.6
28.8
12
44.4
17.9
38.5
38.3
38.0
23.8
19.8
17.3
15
37.0
10.4
35.8
35.8
35.4
16.0
12.4
11.2
16

---

Under review as a conference paper at ICLR 2020
Table 11: Comparison of 6o7, 1z- and /-attackk on a naturally trained model on Restricted IetgeNet. We report the accuracy in percentage of the classifier on the test set if the attack allowed to perturb the test points first 500 points of the test set.
lp-distance. The statistics are computed on the
Robust accuracy Restricted ImageNet plain model
metric
DF
DAA-
DAA-
PGD-
PGD-
FAB-
FAB-
10
1
10
10
0,25
76.6
74.8
74.8
74.8
74.6
75.2
75.2
0.5 0.8/208
51.8
48.2
38.2
37.8
39.6
100
0.75/255
26.8
46.0
41.0
12.2
12.2
14.2
14.2
1/255
11.2
43.2
39.4
3.8
3.8
3.6
3.6
1.25/255
5.0
41.2
38.2
1.0
1.0
1.2
1.0
DF
PGD- 1
PGD-
FAB-
FAB-
10
10
0.2
80.2
76.0
76.0
77.0
76.8
0.4
58.4
40.8
40.6
42.2
l2
0.6
33.8
15.4
14.8
19.0
18.2
0.8
18.8
4.0
4.0
4.6
4.4
1
8.6
1.6
1.6
1.2
1.0
SparseFool
PGD-
PGD-
FAB-
FAB-
5
5
88.6
81.8
81.8
79.6
78.0
80.0
45.2
45.2
4 27
70.6
17.8
17.4
25.6
19.8
38
65.0
6.2
6.0
13.6
7.0
49
55.4
2.2
2.2
6.8
3.8
Table 12: Comparison of loc,, lz- and tacks on an Isc-robstt model on Restricted
Net. We report the accuracy in percentage of the classifier on the test set if the attack allowed to perturb the test points first 500 points of the test set.
disttaanc The statistics are computed on the
Robust accuracy of Restricted ImageNet -obustt model
metric
DF
DAA-
DAA-
PGD-
PGD-
FAB-
FAB-
10
10
10
2/255
75.8
75.0
75.0
74.6
74.6
75.2
75.2
4/255
53.0
46.2
46.2
45.4
45.4
47.4
Too
6/255
32.4
24.6
23.8
19.4
19.4
21.2
21.0
8/255
19.4
17.0
14.6
6.2
6.2
6.8
6.8
10/255
10.8
12.8
11.6
1.0
0.8
1.2
1.2
PGD-
PGD-
FAB-
FAB-
10
10
79.4
76.6
76.6
77.0
76.8
65.0
46.8
46.2
49.8
49.2
3
46.8
22.4
21.4
24.4
23.8
4
32.8
9.0
8.6
10.8
10.6
5
20.4
3.0
2.8
3.2
3.2
SparseFool
PGD- 1
PGD- 5
FAB-
FAB-
15
81.8
69.8
69.8
69.6
68.2
76.4
58.6
58.2
53.6
4
40
71.4
41.0
41.0
41.2
37.6
60
63.2
28.8
28.8
28.2
23.8
100
49.2
12.0
11.6
14.6
11.2
17

---

Under review as a conference paper at ICLR 2020
Table 13: Comparison of lx-, 1-- and laattacks on an 12-robust model on Restricted ImageNet. We report the accuracy in percentage of the classifier on the test set if the attack is allowed to perturb the test points of in lp-distance. The statistics are computed on the first 500 points of the test set.
Robust accuracy of Restricted ImageNet l2-robust model
metric
DF
DAA-
DAA-
PGD-
PGD-
FAB-
FAB-
10
1
10
10
3/255
74.4
73.0
73.0
73.0
73.0
73.8
73.8
4/253
49.0
39.6
39.2
37.6
37.6
39.6
39.4
1oo
6/255
27.4
22.6
21.0
13.2
13.2
15.0
15.0
8/255
13.8
18.6
16.8
3.6
3.6
3.6
3.2
10/255
6.6
15.6
13.8
0.4
0.4
0.8
0.8
DF
PGD-
PGD-
FAB-
FAB-
50
10
74.2
71.8
71.8
72.8
72.8
61.6
51.4
51.0
52.4
12
45.6
31.0
30.8
34.4
33.8
5
34.6
20.4
20.4
22.6
21.8
6
25.2
9.6
9.6
11.8
11.6
SparseFool
PGD 1
PGD-
FAB-
FAB-
5
50
85.4
81.0
81.0
78.6
78.4
100
79.6
63.8
63.6
60.8
59.0
4
150
74.4
48.4
48.4
45.2
42.2
200
68.6
32.2
32.2
31.0
29.0
250
60.0
23.0
22.4
22.8
20.2
18

---

Under review as a conference paper at ICLR 2020
MNIST
plain
linf-at c=2
12-at c=2
step size = 1/1
step size = 2/1
step size = 2/1
step
%: faramoae zenozu
step size
% Caremaae zenuzu
step
step size 1/10
step size 2/10
slep size 2/10
step size 1/25
slep size 2/25
step size 2/25
Aeration
Aoration
Aoration
plain c=1.5
linf-at c=3
12-at <=3
step size 1.5/1
step size = 3/1
step size = 3/1
step size 1.5/2
step size
% Kiremooe pamoru
step size
1,544
Saeenooe Tenoou
stop size 1.5/10
size 3/10
size 3/10
step size 1 1.5/25
step size 3/25
step size = 3/25
step
step
step
Atration
itoraioo
itortiio
CIFAR-10
plain c=0.2
linf-at - c=0.75
12-at c=0.75
step size = 0.2/1
step size = 0.75/1
step size = 0.75/1
rabust accuracy %
step size 0.2/2
step size 0.75/2
step size 0.75/2
step
step
step
size 0.2/10
size 0,75/10
size 0,75/10
step size 0
0/25
step size 0.75/25
step size 0.75/25
step size 0 0.2775
step size 0.75/75
step size 0.75/7
plain c=0.4
linf-at 1=25
12-at c=1.25
step size = 0.40/1
step size = 1.25/1
step size = 1.25/1
step size 0.40/2
step size 1.25/2
step size 1.25/2
accuracy
step size
Koiemmoe ponuou
step size
step size
size 0.40/10
size 1.25/10
size 1.25/10
step size 0.40/25
step size 1.25/25
step size 1, 1.25/25
step size 0.40/75
step size 1,25/75
step size 1,25/75
Figure We plot for different step sizes of PGD robust accuracy over iterations. In red the step size we used in the experiments of Section 3 We clearly see that our chosen step-size on average the best onc. The models used are those trained on MNIST (top row) and CIFAR-10 (bottom row).
19

---

Under review as a conference paper at ICLR 2020
B ANALYSIS OF THE ATTACKS
B.1 CHOICE OF THE STEP SIZE OF PGD
We here show the performance of PGD wrt l2 MNIST and CTFAR-10 under different choices of the step size. In particular we focus here on the largest and the middle e values chosen in the evaluation where the different stepsize choices have the largest impact. We report the robust accuracy for at each of the 150 iterations. We test step sizes s/x for E 21 10, 25, 75}. For cach step size we run the attack 10 times with random initialization and show the run which achieves the lowest robust accuracy after 150 iterations. Note however that the behaviour results for the three models for MNIST and CIFAR-10 for two different choices of used in Section 3 with step size decreasing the blue becoming darker, while our chosen step size, that is 9/1, is highlighted in red. We see that it achieves in all the models best or close to best robust accuracy and is clearly the best on average.
different. runs varies minimally. In Figure we show the
B.2 EVOLUTION ACROSS ITERATION
We here want to compare the evolution of the robust accuracy across the iterations of single run of PGD and FAB, that is PGD-1 and FAB-1 from Tables 5 to 13 Since PGD performs forward and 1 backward pass for each iteration and FAB forward passes and 1 backward pass, we rescale the robust accuracy so to compare the two methods when they have exploited the same number of passes of the network. Then 300 passes correspond 150 iterations of PGD and to 100 of FAB. In Figures accuracy for the different dataset, models and threat models (l l2 and I1), computed at the threshold median among the five used in Tables to 13
and 0 we show the evolution of robust
20

---

Under review as a conference paper at ICLR 2020
MNIST, 400
plain c=0.1
linf-at c=0.3
12-at c=0.1
PGD-1
-PGD-1
-PGD-1
roOUSE acouraty 70
FAB-1
%
-FAB-1
TOBLUS sucurracy %
-FAB-1
# ppssss
#
ppsses
# passes
MNIST, 12
plain =1.55
linf-at c=2
12-at c=2
PGD-1
PGD-1
-GGD1
raDum accuraty 7N
FAB-1
rabUS Mocurary %
-FAB-1
rabuS aoxurary %%
-FBB-1
M pesses
8
pesses
81 passes
MNIST,
plain c=6
linf-at c=7.5
12-at c=12.5
PGD-1
-PGD-1
-PGD-1
TOQUEE acouracy 70
FAB-1
-FAB-1
FFAB-1
TOBUSE sccuraiyy
robust accurarcy
Figure 4: Evolution of accuracy across iterations MNIST. We compare the robust accuracy of PGD-1 (magenta) and FAB-1 (black) as a function of the employed forward woward/backww bback ward passes in the algorithm (one iteration of PGD corresponds to 2 passes, one iteration of FAB corresponds to 3 passes). Models: plain in the first column, lat in the second and la-at in the third. Threat models: loc in the first row, l2 in the second and in the third. The thresholds used can be read above the plots. Note that the result of FAB-1 on los wrt 4 does not match that in Table due to typo in the statistics in the table.
21

---

Under review as a conference paper at ICLR 2020
CIFAR-10, 600
plain c=2/255
linf-at c=6/255
12-at c
PGD-1
-PGD-1
PGD-1
FAB-1
-FAB-1
FAB-1
robusa 2
passes
ppsses
passes
CIFAR-10, 12
plain ==0.2
linf-at ==0.8
12-at =00.8
PGD-1
PGGD1
-PGD-1
FAB-1
%
-FAB-1
%
FFBB-1
# pesses
# passes
# passes
CIFAR-10, 4
plain c=6
linf-at c=12.5
12-at c=9
PGDD1
-PGD-1
PGD-1
-AB-1
%
-FAB-1
%
FAB-1
#1 passss
# passes
# p2pses
Figure 5: Evolution of accuracy across iterations on CIFAR-10. We compare the robust accu- racy PGD-1 (magenta) and FAB-1 (black) as a function of the employed forward/backwarr waakward passes in the algorithm (one iteration of PGD corresponds to 2 passes, one iteration of FAB corresponds to passes). Models: plain in the first column, lt in the second and az- in the third. Threat models: loo in the first row, l2 in the second and 4 in the third. The thresholds used can be read above the plots.
22

---

Under review as a conference paper at ICLR 2020
Restricted ImageNet, 600
plain c=0.75/255
linf-at c=6/255
12-at c
PGD-1
PGDD1
PPD-1
FOOISS acouracy 70
FAB-1
rOblUSt accursin %6
-AB-1
rObJSS socuraan %
-AB-1
passes
spssss
# passes
Restricted ImageNet, l2
plain =00.6
linf-at c=3
12-at ==4
PGD-1
PGD-1
PGD-1 FFBB11
rocuss accuracy %%
FAB-1
rabbuS socuray %
FFBB-1
rabuss socurac %
# pesses
# passes
8 passes
Restricted ImageNet, 4
plain c=27
linf-at e=40
12-at ==150
PGD-1
PGGD1
PPGD-1
100u accourany 20
-FB-1
TD0LUST scourasy %
-FBB1
%
-FAB1
A9 p2ssss
# pppses
2 p2pses
Figure 6: Evolution of robust accuracy across iterations on Restricted ImageNet. We compare the robust accuracy of PGD-1 (magenta) and FAB-1 (black) as function the employed forward/backward barkkward passes in the algorithm (one iteration of PGD corresponds to 2 passes, one iteration of FAB corresponds to passes). Models: plain in the first column, la in the second and la-at in the third. Threat models: loo in the first row, in the second and I in the third. The thresholds e used can be read above the plots.
23


Reviews:
[{"review_id": "Hkgp5zksFS", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors propose a new gradient-based method (FAB) for constructing adversarial perturbations for deep neural networks. At a high level, the method repeatedly estimates the decision boundary based on the linearization of the classifier at a given point and projects to the closest \"misclassified\" example based on that estimation (similar to DeepFool). The authors build on this idea, proposing several improvements and evaluate their attack empirically against a variety of models.\n\nI found the proposed method quite interesting and intuitive. All the improvements made to the core method are well-motivated and clearly explained, while the ablation experiments are relatively thorough.\n\nHowever, I did find the presentation of experimental evidence quite misleading. \n\nSpecifically, reporting mean accuracy over models, datasets, and epsilon constraints in Table 2 does not give the full picture. Going through the appendix tables, we can see the following:\n-- The step size used for PGD is quite large---eps/4 for the L2 case---which is quite uncommon when using 150 iterations. Based on prior work and my own personal experience, a step size of 2 * eps / #steps (i.e., eps / 75) would seem more suitable. I wonder if this is the reason for PGD performing worse than FAB for large epsilon values on CIFAR10. The authors mention that they chose this parameter using grid search but do not provide concrete details.\n-- The adversarially trained MNIST model of Madry et al. 2018 learns to use thresholding filters as the first layer (observed in the original paper). This causes issues for most gradient-based methods (e.g., PGD performs worse than the decision-based attack of Brendel et al. 2018, also observed in other prior work). While it is encouraging that FAB is robust to such gradient obfuscation, this is arguably not the ideal setting to compare gradient based methods (especially when averaging performance over models). \n-- For MNIST and Restricted IN, PGD performs comparably or even better than FAB (modulo larger epsilon values for which the large step size used could be an issue for PGD and the Linf-trained model with the thresholding filters).\n-- For the L1-norm setting, EAD performs similarly or better compared to FAB (again modulo the Linf-trained model).\nBased on these observations, I am not fully convinced that FAB outperforms PGD (for L2 and Linf) and EAD (for L1) by as much as Table 2 suggests.\n\nMoreover, the runtime comparison performed in not exactly fair:\n-- It is not clear how many restarts where included in the runtime of PGD. Its runtime should be in the same ballpark as FAB but the time reported is ~20x higher. \n-- PGD is known to produce quite accurate estimates when run with much fewer (say 15) steps. Thus in order to make a fair comparison one would also need to look at the entire #steps vs robust accuracy curve to get a better picture of the efficiency of these two methods. Choosing an arbitrary number of steps for each method is not very enlightening.\n-- It is not necessary to run PGD 5 times to evaluate the robust accuracy at 5 thresholds. One can perform binary search for each input in order to find the smallest epsilon for which a misclassification can be found. This will result in at most 3 (sometimes 2) evaluations per point (instead of 5).\n\nDespite these shortcomings of the experimental evaluation, I still believe that the paper has merit. After all, the method is clean and well-motivated,  performs comparably to the best of PGD and EAD in a variety of settings, and is robust to a certain degree of gradient masking. In that sense, it could potentially be a valuable contribution and could be of interest to a subset of the adversarial ML community.\n\nIn the sense, while my initial stance is to recommend (weak) rejection, I would be open to increasing my score and recommending (weak) acceptance should my concerns be addressed.\n\nUPDATE: I appreciate the response and the additional experiments performed by the authors. The authors have addressed my concerns in their response. I am increasing my score to a weak accept.\n\nOne thing that would be nice to add in the next version of the manuscript is a note inviting the reader to consider the appendix tables since average robust accuracy can be inconclusive.", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}, {"review_id": "HJgiUscAFH", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Authors extend deepFool by adding extra steps and constraints to find closer points to the source image as the adversarial image. They both project onto the decision boundary. Deepfool does and adhoc clipping to keep the pixel values in (0,1) but the new proposed method respects the constraints during the steps. Also during the steps they combine projection of last step result and original image to keep it closer to the original image. Moreover, at the end of the optimization they perform extra search steps to get closer to the original image. Also they add random restarts. Rather than considering the original image, they randomly choose an image in the half ballpark of the total delta.\n\nAccording to the results in fig.2 the backward steps has the highest impact in comparison to deepfool. But mixing with original projection always helps a little and random restarts help a little too. Without the backward steps there is almost no gain from mixing the projections.\n\nConsidering the full results in the appendix, the results are mixed with no obvious advantage in comparison to PGD specially.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I made a quick assessment of this paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "BJeCBkS5Kr", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper studies the problem of the white-box attack of neural network-based classifiers, with an emphasis on the \"minimal distortion solution\": The new input that changes the labeling output of the network with the minimal distance (l1, l2, l_inf) with respect to a given input. \n\nThe main intuition of the algorithm is to do a local linear approximation of the network at the current point (which is the Taylor expansion up to the gradient term). After that, the algorithm identifies a class (output coordinate) with the minimal \"margin to gradient norm ratio\", i.e. the total movement in gradient direction to change the labeling function in that coordinate, within this linear approximation. The algorithm solves the subproblem of minimizing a linear function inside lp ball as the critical routine.\n\nOverall, the notion of finding the minimal distortion attacker as opposed to finding the best attacker inside a fixed distortion ball is quite interesting to me. The main concern for me about this paper is the comparison to other methods such as PGD. As far as I know, these attackers DO NOT explicitly minimize the distortion, thus it is quite believable that these models do not identify the minimal distortion solution (rather it will more likely to find a solution that lies in the boundary since it would be the easiest way to attack). However, for the proposed algorithm in this paper, the algorithm is explicitly minimizing the distance to the given input (x_orig in their language). \n\n\nI would like to see more implementation details of the other algorithms, for example, what is the performance if we add an additional regularizer as the distance of the current attacker to the given input to PGD. So far, the paper lacks solid proof of the usefulness of this particular algorithm. (In particular the justification for solving the local linear system instead of doing a gradient descent step).\n\nAfter Rebuttal: I have read the authors' responses and acknowledge the sensibility of the statement. I apologize for the earlier misunderstanding and higher the score accordingly.\n", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This work presents a method for generating an (approximately) minimal adversarial perturbation for neural networks. During the discussion period, the AC raised additional concerns that were not originally addressed by the reviewers. The method is an iterative first order method for solving constrained optimization problems, however when considered as a new first order optimization method the contribution seems minimal. Most of the additions are rather straightforward---e.g. using a line search at each step to determine the optimal step size---and the reported gains over PGD are unconvincing. PGD can be considered as a "universal" first order optimizer [1], as such we should be careful that the reported gains are substantial and not just a question of tuning. Given that using a line search at each step increases the computational cost by a multiplicative factor, the comparison with PGD should take this into account.

The AC notes several plots in the Appendix show PGD having better performance (particularly on restricted Imagenet), and for others there remain questions on how PGD is tuned (for example the CIFAR-10 plots in Figure 5). One of two things explains the discrepancies in Figure 5: either PGD is finding a worse local optimum than FAB, or PGD has not converged to a local optimum. There needs to be provided experiments to rule out the second possibility, as this is evidence that PGD is not being tuned properly. Some standard things to check are the step size and number of steps. Additionally, enforcing a constant step size after projection is an easy way to improve the performance of PGD. For example, if the gradient of the loss is approximately equal to the normal vector of the constraint, then proj(x_i+ lambda * g) ~ x_i will result in an effective step size that is too low to make progress.

Finally, it is unclear what practical use there is for a method that finds an approximately minimum norm perturbation. There are no provable guarantees so this cannot be used for certification. Additionally, in order to properly assess the security and reliability of ML systems, it is necessary to consider larger visual distortions, occlusions, and corruptions (such as the ones in [2]) as these will actually be encountered in practice. 

1. https://arxiv.org/pdf/1706.06083.pdf
2. https://arxiv.org/abs/1807.01697
Decision: Reject