Fast and Reliable Evaluation of Adversarial Robustness with Minimum-Margin Attack
Authors: Ruize Gao
Jiongxiao Wang
Kaiwen Zhou
Feng Liu
Binghui Xie
Gang Niu
Bo Han
James Cheng
Venue: ICLR 2022 Submitted (2022)
Keywords: adversarial machine learning
evaluation of adversarial robustness

Abstract: The AutoAttack (AA) has been the most reliable method to evaluate adversarial robustness when considerable computational resources are available. However, the high computational cost (e.g., 100 times more than that of the project gradient descent attack) makes AA infeasible for practitioners with limited computational resources, and also hinders applications of AA in the adversarial training (AT). In this paper, we propose a novel method, minimum-margin (MM) attack, to fast and reliably evaluate adversarial robustness. Compared with AA, our method achieves comparable performance but only costs 3% of the computational time in extensive experiments. The reliability of our method lies in that we evaluate the quality of adversarial examples using the margin between two targets that can precisely identify the most adversarial example. The computational efficiency of our method lies in an effective Sequential TArget Ranking Selection (STARS) method, ensuring that the cost of the MM attack is independent of the number of classes. The MM attack opens a new way for evaluating adversarial robustness and contributes a feasible and reliable method to generate high-quality adversarial examples in AT.

Under review as a conference paper at ICLR 2022
FAST AND RELIABLE EVALUATION OF ADVERSARIAL ROBUSTNESS WITH MINIMUM-MARGIN ATTACK
Anonymous authors Paper under double-blind review
ABSTRACT
The AutoAttack (AA) has been the most reliable method to evaluate adversarial robustness when considerable computational resources are available. However, the high computational cost (e.g.. 100 times more than that of the project gradient descent attack) makes AA infeasible for practitioners with limited computational resources, and also hinders applications of AA in the adversarial training (AT). In this paper, we propose a novel method, minimumm-mee maninu (MM) attack, to fast and reliably evaluate adversarial robustness. Compared with AA, our method achieves comparable performance but only costs 3% of the computational time in extensive experiments. The reliability of our method lies in that we evaluate the quality of adversarial examples using the margin between two targets that can precisely identify the most adversarial example. The computational efficiency of our method lies in an effective Sequential TArget Ranking Selection (STARS) method, ensuring that the cost of the MM attack is independent of the number of classes. The MM attack opens a new way for evaluating adversarial robustness and contributes a feasible and reliable method generate high-quality adversarial examples in AT.
INTRODUCTION
The deep neural network (DNN) has attracted large number of researchers from different disciplines such as computer science (Goodfellow et al., 2016; Castelvecchi, 2016; Vaswani al., 2017), physics (De Vries et al., 2018; Huang et al., 2019; Levine et al., 2019), biology (Maxmen, 201 8a;b; Webb, 2018) and medicine (Hao et al., 2015; Esteva et al., 2017). The success DNN mainly lies in its ability to learn useful high-level features from abundant data (Deng & Yu, 2014; LeCun et al., 2015). These learned features have been successfully used to address many difficult tasks. For example, DNNs can recognize images with high accuracy comparable to human beings (LeCun et al., 1998; Krizhevsky et al., 2012). In addition, DNNs are also widely used for speech recognition (Hinton al., 2012), natural language processing (Andor et al., 2016), and playing games (Mnih et al., 2013; Silver et al., 2016).
As the impacts of DNN increase fast, its reliability has been key to deploy it in real-world applications (Huang et al., 2011; Kurakin et al., 2017). Recently, a growing body of research shows that DNNs are vulnerable to adversarial examples, i.e., test inputs that are modified slightly yet strategically to cause misclassification (Szegedy et al., 2014; Nguyen et al., 2015; Kurakin et al., 2017; Carlini & Wagner, 2017a; Finlayson al., 2019; Wang et al., 2019; Zhang al., 2020b; Gao et al., 2021). The existence of such adversarial examples undoubtedly lowers the reliability of DNNs. Meanwhile, researchers have also been considering finding a reliable way to evaluate adversarial robustness of a DNN before deploying it in the real world.
The high-level idea of evaluating adversarial robustness of a DNN is quite straightforward, i.e., generating adversarial examples and calculating the accuracy of the DNN on these examples (this kind of accuracy is also known as adversarial robust accuracy). Szegedy et al. (2014) first pointed out the existence of adversarial examples and used less powerful box-constrained L-BFGS method to generate them. Based on the studies in (Szegedy et al., 2014), Goodfellow et al. (2015) put forward the fast gradient sign method (FGSM). One common loss function they used is cross-entropy (CE) loss, and to maximize the loss function, FGSM uses its gradient to determine which direction the pixel's intensity should be increased or decreased, Madry al. (2018) 8) introduced simple refinement

---

Under review as a conference paper at ICLR 2022
56
6300
.44
93.52
53.88
51.0
5300
Attack success Tele
50
31442
30 4300
576
3003
13
2181
1000
110
PGD A-ULR A-8
PAB MMS MMS
T-AA
MM
RGD A-DLR A-CE
BAB
T-AA MM
Attacks
Attacks
(a) Reliability ranking
(b) Computational time
Figure 1: Comparison of reliability computational cost among baselines and different versions of MM attack. MM3, MM5 and MM+ are three versions of our MM attack. The subfigure (a) shows the (sorted) attack success rates of diffcrent attacks. The higher the success rate, the stronger the attack. The subfigure (b) shows the computational cost of these attacks. The less time, the better the attack. AA is an ensemble of APGD-CE (A-CE for short), APGD-DLR (A-DLR for short), FAB and Square. T-AA considers each target for APGD-DLR and FAB AA, and is thus more time-consuniig. Compared with AA (or T-AA), our MM3 achieves comparable performance but only costs 3% (or 2%) of the computational time. The model structure we used is ResNet-18, which follows the adversarial training of (Madry et al., 2018) on CIFAR-10.
of the FGSM: projected gradient descent attack (PGD), where instead of taking single step of size in the direction of the gradient sign, multiple smaller steps are taken.
effective method to evaluate adversarial robustness of standard-trained DNN (Madry et al., 2018) since adversarial robust accuracy of a standard-trained DNN is always very low after using PGD. Nevertheless, the existence of adversarial examples has already inspired research on training a robust DNN to defend against them, which means that -andard-trained DNN is not the only DNN we might meet and we need to evaluate adversarial robustness of a robust DNN as well. Unfortunately, PGD fails to reliably evaluate adversarial robustness of a robust DNN (Carlini & Wagner, 2017b; Croce & Hein, 2020).
Existing Evaluation Methods. PGD
Carlini & Wagner (b)17b) observed the phenomenon of gradient vanishing in the widely used CE loss for the potential failure L-BFGS, FGSM and PGD, and replaced the CE loss with many possible choices. Croce & Hein (2020) claimed that the fixed step size and the single attack used are the causes of poor evaluations, and they put forward an ensemble of diverse attacks (consisting of APGD-CE, APGD-DLE DLR, FAB and Square) called AutoAnack (AA) to test adversarial robustness. Until now, AA has been the most reliable method to evaluate adversarial robustness (Croce & Hein, 2020).
However, though AA performs well in reliability, it needs a large amount of computational time. As shown in Figure ((a), for attacking a ResNet-18 model on CIFAR- 10 (following the adversarial training in (Madry et al., 2018)), the computational cost of AA (or T-AA) is 65 times (or 100 times) more than PGD used in (Madry et al., 2018), where T-AA is more time-consuming since it considers each target for APGD-DIR and FAB in AA. Worse still, in the worst the computational cost of AA (or T-AA) is even 109 times (or 440 times) more than PGD.
analyzed in Appendix A,
Dilemma Between Reliability and Computational Efficiency. The high computational cost makes AA infeasible when considerable computational resources are unavailable. Unfortunately, such scenarios are common in the real world, e.g., for practitioners who need real-time evaluation at each epoch of the training process of robust model, such high computational cost is unacceptable. Similarly, since large number of adversarial examples need to generated each epoch during adversarial training (AT), such high computational cost hinders applications of AA in AT. In consid- eration of the high reliability but low computational efficiency of AA, and the high computational efficiency but low reliability of PGD, up one factor (reliability or computational efficiency) when evaluating the adversarial robustness.
to encounter a dilenma: have consider giving
Our Reliable and Fast Solution. In this paper, are dedicated to achieving reliability and computational efficiency simultaneously. For reliability, we evaluate the quality adversarial examples using the margin between two targets for precisely identifying the most adversarial example. For computational efficiency, we put forward an effective Sequential TArget Ranking Selection (STARS) method to ensure that the cost of the MM attack is independent of the number of classes.
2

---

Under review as a conference paper at ICLR 2022
Bounded perturbation By[k]
Misclassified
Classified arca
Misclassified area
x' of minimum- mungin-of--roobbblitt
minimum- minimum-probsaiilly
Classified
Margin of probability Py -P
Py
Probubility Py
Figure 2: Minimum margin of probability. p denotes the predicted probability, Py and pr are the predicted probability on the true label and targeted fälse label l. The gray shape is the image of the adversarial variants a' within the bounded perturhation hall [x]x under the mapping of the network onto (Py, Pt); the orange area (pe > Pτ) indicates the region where the adversarial variants are misclassified, or say successful attack, while the blue area (p+ < Py) indicates the region where the adversarial variants do not attack successfully.
Reliability. To achieve reliability, we investigate the reasons behind the failure PGD. We identify that CE loss, which is based on the probability of the true label py. is not an appropriate measure to the quality of adversarial examples. Figure 2, we provide a simple demonstration to this issue, in which we consider one targeted false label t. As we can see, it is much more reasonable to measure the quality of adversarial examples in terms of the margin of probability Py - Pt. The most adversarial example in Figure 2 then corresponds to the one with the minimum margin of probability instead of the minimum probability Py. Detailed study of the rationality of minimum-mais magrin is provided in Section 3.1. Since the search space B, of high dimensional images is large (grey area), previous studies use gradient descent methods to generate the adversarial example that maximizes the loss function (Goodfellow al., 2015; Carlini & Wagner, 2017b; Madry et al., 2018).
Though it looks promising to generate adversarial examples via minimizing the margin of probability, find that there are still two issues: (a) The probability p is kind of rescaling method to the logits Croce & Hein (2020) heuristically rescaled the logits using their proposed DLR (defined at (6). We investigate the performance of different rescaling methods in Section 1. We numerically find that the method using natural logits 2y 2 (the meaning of margin) with no rescaling performs the best; (b) For the problem of multi-class and untargeted attacks, the margin is then By max 24- However, -E, m 2;) ) is not appropriate loss function, because the max function only considers one target the current step while those unconsidered targets may lead to more adversarial examples. Hence, the reliable method is to minimize 2, for each & + y and take the most adversarial one, which is a widely used solution (Croce & Hein, 2020).
Computational Efficiency. Although running the attack for each false target is reliable, the com- putational cost depends on the number of classes. For datasets with a large number classes, e.g., CIFAR- 100 (100 classes) and Imagenet (1,000 classes), the computational cost will increase accordingly. To achieve computational efficiency, we propose a Sequential TArget Ranking Selection (STARS) method in Section 3.2 make the computational time independent of the number of classes. According to the ranking of predicted probability, STARS method only selects a few highest targets and runs a sequential attack. Experiments show that, benefited from STARS method, MM attack can save 76.36% of the computational time on CIFAR-10, 98.51% % on CIFAR-100 and 77.78% on SVHN.
By taking all the above factors into consideration, we propose novel method, the minimum- margin (MM) attack. Its detailed realization is provided in Section 3. We present extensive experi- mental results in Section 4, which verify that our MM attack can fast and reliably evaluate adversarial robustness. In particular, MM attack achieves comparable performance but only costs 3% of the computational time compared with the current benchmark AA. Hence, our proposed MM attack provides a new direction of evaluating adversarial robustness and contributes a feasible and reliable method to generate high-quality adversarial examples AT.
2 PRELIMINARY
Neural Networks. A neural network a function fo R= - 1],, where 0 is the parameters contained in so and K is normally the number of classes. The output of the network is computed
3

---

Under review as a conference paper at ICLR 2022
using the softmax function, which ensures that the output is valid probability vector. Namely, given input RD, fa(x)) [pI. :p...... ......-PP Pk] = p where Σ P; 1 and pt is the probability that input x has on class i. Before the softmax function, the output of the network is called logits z, i.c., p softmax(z). The classifier assigns the label y arg max; (z);
Projected Gradient Descent Attack (PGD). Madry et al. (2018) proposed the projected gradi- ent descent (PGD) attack to generate adversarial examples to mislead a well-trained classifier fa. Specifically, they start with setting x" = and then in each iteration:
(((+1) =Il,, [m) (0/1) + sign V *(1) ((fol(())) ()),)))))))) 0, 1,,
(1)
where
[x = {z' do, x') S c}
the closed ball of radius c >> 0 centered at z; the (0) refers the starting point which corresponds to the natural example (or the natural example perturbed by a small Gaussian or uniformly random
noise); .²(() is the adversarial example at step i; [. (.) is the projection function that projects the adversarial variant back the e-ball centered at (0)) if necessary; the foo distance metric is do (x) = || - x' oooooo: and & is cross entropy (CF) loss:
K Σ 6
CE(F, 2. v) -log(p)) -y + +log
K et, where p; = = -
I,..., K, and is the logits of the model outputs.
Carlini and Wagner attack (CW). Carlini & Wagner (2017b) observed the phenomenon of gradient vanishing in the widely used CE loss for potential failure. The gradient w.r.t x in Eq. (3) is given by
V V CE(,..)))) - (-1 p.) + Σ V224 1t1
(4)
If Py ~ 1 and consequently p: ~ 0 for i + y. then V, CE,,, 3) = 0 in Eq. (4). This gradient vanishing issue can lead to the failure of attacks. Motivated by this phenomenon, Carlini Wagner (2017b) replaced the CE loss with several possible choices. Among these choices, the widely used one for the untargeted attack is
CW CW(,I,y)) -zy(x') + max (xx).
(5)
AutoAttack (AA). Croce & Hein (2020) claimed that the fixed step size and the lack of diversity in attack methods are the main reasons for the limitations of previous studies. Motivated by the line search method (Grippo et al., 1986), they put forward adaptive projected gradient descent attack (APGD). They showed that using adaptive step size significantly improves the adversarial robustness compared with using fixed step size. For the loss function at Eq. (5), they claim that scale invariance w.r.t. is necessary, and they proposed an alternative loss function:
DLR(f,I.9) 2y(x') max 23 (x')
(6)
2x, (x') Zerg (30)
where π is the permutation of the components of in decreasing order. For the targeted attack, they propose another alternative loss function:
2y(x²) - 2x(2²) 2 (xx) 1/2 (Zng (xx) Zxa 2x, (x'))
(opeteddDDR(ffxp.pp =
For the lack of diversity, they claimed that diverse attacks are beneficial for reliability. and then they put forward ensemble of various parameter-free attacks called AutoArtac (AA) to test adversarial robustness, where AA contains APGD-CE. APGD-DLR, FAB and Square. Targeted AutoAtrack (T-AA) replaces APGD-DLR with the targeted APGD-DLR, and replaces FAB with targeted FAB.
4

---

Under review as a conference paper at ICLR 2022
Table I: Attack success rate (%) of different loss functions.
Loss function
CIFAR-10
CIFAR-100
Diff.
SVHN
Diff.
Tiny-Imagenet
Dift.
PGD
(Py)
48.27
-3.09
73.60
-2.92
41.89
-5.67
78.78
-4.19
(xy) muxicty (xr))
49.13
-2.23
74.55
-1.97
45.08
-2.48
-1.78
MM
-By + Re
51.36
0.00
76.52
0.00
47.56
0.00
82.97
0.00
THE REALIZATION OF MINIMUM-MARGIN ATTACK
In this section, we discuss the realization of our minimum margin (MM) Attack. In Section 3.1, we verify the rationality of using Kinimum-margin as the loss function and discuss the influence of different logits rescaling methods on robustness. In Section 3.2, we propose effective Sequential TArget Ranking Selection (STARS) method to improve computational efficiency. Section 3.3, we provide the detailed descriptions of MM attack.
3.1 THE RATIONALITY OF MINIMUM-
To understand the rationality of our minimum-margin. we first look into the situation where no adversarial attack can succeed. Then show that the formulation of minimum-margin is naturally derived from such a situation. We say that classifier f is completely robust if Vax' E B, [x]. arg max; f(x'); arg max, fx), The following condition is necessary and sufficient to the complete robustness:
Condition 1. Given natural example with its true label y, the K-class classifier satisfies
Va' E B,(x), by (x')
max 2,(x') 2 0,
where [x = {x' (2, s c): 2y (x') = f(x)) (4''); f(x'),
According this condition, to reliably evaluate the complete robustness, the adversarial attacks should find the adversarial examples with minimum my =,('") maxigy =,(x'), i.e., the most non-robust data point. It indicates that we should replace the CE loss in Eq. (3) with -(z (1²) - max (a))) the loss function in adversarial attacks. However, as mentioned before, -(2y max 2,) in Eq. (5) is not an appropriate loss function since it only focuses on the current step. The reliable method is to minimize by 2, for each target * y and take the most adversarial one. To verify the rationality of minimum-margin we conduct experiments on different datasets with -(z, - 2,) being the loss function (MM). In Table MM performs more reliable evaluation than PGD and CW.
The comparison with Targeted-DLR, Targeted-DLR (Croce & Hein, 2020) heuristically rescales the logits in Eq. (7). For the logit of true target 2y and false target zt, the relative magnitude of by and is constant under different rescaling methods. However, though the rescaling methods preserve the sign of 2, 2g, they will lead different adversarial variants. Clearly, there is no perfect rescaling method that can perform the best in any situation but inappropriate rescaling methods will hinder the searching for the most adversarial example.
We conduct experiments to investigate the difference of using different rescaling methods, The experimental setting follows (Madry et al., 2018), and we replace the CE loss with seven different logits rescaling methods. In Table 2, the successful set denotes the number of examples that can be attacked successfully' in the test set of CIFAR- 10 (10, 000 examples). As shown Table inappropriate logits rescaling methods (e.g., Targeted-DLLR in AA) reduce the reliability, and the method using natural logits (no rescaling) performs the best among the seven methods. The results motivate
the loss function. Note we do not deny that there could be better rescaling method, but still, a reasonably good result can be obtained by no rescaling (natural logits in Table 2).
We also investigate the difference among different successful sets. The non-empty difference sets A U B, - A and B, B, in Table 2 suggest that diverse logits rescalings can be considered when considerable computational resources are available, we analyze it in Appendix C.
5

---

Under review as a conference paper at ICLR 2022
Table 2: The successful set of different rescaling methods.
ID
Rescaling method
Formalation
Set size
Ranking
diff.
Au B:
AU B; B;
A
Natural logits
(8y 4x)
5219
B1
Softmax
cfa
5172
2
-47
4
51
B2
Max
Py Fyy
5165
-54
59
Bo
Sum
By Fy th
5165
=3
-54
5
59
84
Min-Max
Sy
5121
-98
101
Fm1g
Bs
DLR
5078
6
-141
2
143
stj - (#og Hamgg)
Bs
Sigmoid
1+ 7
4820
7
-399
400
Algorithm MM Attack
1:
Input: natural data æ, true label y, set of false labels C, model f. loss function êM maximum number of PGD steps N, perturbation bound initial step size a, the number of classes K, targets selection number K,, checkpoints set W;
2: Output: adversarial data 20;;
while K. do
4: 26 + x; I Fmax f x: fmaz + [ab)): c= arg max (()
5: for k = to N do
6:
# [T (2's asign(V laM (f(a)))))
7:
if ((((+)) fmaz then I'maz Z'4zi Smar +- f(1
8:
if E W and (Condition 2 or Condition then a/2; a amaxi
9:
end for
10:
C+ CV C+CV {c};
11:
arg max,ec f(x'); # then K. + +00;
12: K, + K, 1;
13:
end while
3.2 SEQUENTIAL TARGET RANKING SELECTION METHOD
As mentioned in the introduction, for multi-class and untargeted attacks, the widely used and reliable method is to minimize the loss function -(2y 21) for each target t + y. and then take the most adversarial one. Since the computational cost of such solution depends is unacceptable for practitioners with limited computational resources. Here, we propose a fast solution which saves large amount of running time with little-to-no performance lost. We call solution the Sequential TArget Ranking Selection (STARS), which consists of the following strategies.
the number of classes,
Given natural input and K-class classifier f. denoting the predicted probability as f(x); for false label i, a natural intuition is that the false target with a higher value of f(x); is more likely lead to successful attack. To verify this intuition, in Figure between only selecting the false targets with K, highest predicted probabilities and selecting all the
compare the performance
K 1 false targets. The results show that only selecting K₃ (e.g.. K, 3) highest targets achieves comparable performance, Combining this strategy with MM attack, we then denote MM3 (or MM5) as the MM attack with K, (or K, = 5). For the ranking of predicted probability f(x)); we also investigate the difference of replacing the natural input with adversarial examples in Appendix D. which shows that the replacement only has limited improvements. For the sake of efficiency, we recommend to the ranking of f f(x), of the natural input a, which does not need extra computation.
Figure 3 also motivates us to perform a sequential attack based on the ranking of predicted probability on false targets: first, consider the false target with the highest predicted probability f(z); ; if the attack succeeds, terminate MM attack on other targets; otherwise, continue to consider the false target with the second highest predicted probability, and on.

---

Under review as a conference paper at ICLR 2022
Accuray (%6
Accunes (%6)
Acturay (1%
8
22.51
68.22
21
48
(a) Acc on CIFAR-10
(b) Acc on CIFAR-100
(c) Acc SVHN
300
1400
111
1000
Time:
300
Time
800
02
160
MAM
30
99
2
3
(d) Time on CIFAR-10
(e) Time on CIFAR-100
(f) Time on SVHN
Figure 3: Reliability and computational time different selection number. Regarding performance for all targets as benchmarks. as shown in the subfigure (a), (b) and (c), pre-selecting (or 5) targets can achieve 99.13% (or 99.78%) reliability on CIFAR-10, 99.49% (or 99.82% reliability on CIFAR-100: 98.789 38.8 (or 99.67%) reliability on SVHN; shown in the subfigure (d), (c) and (f), pre-selecting 3 (or 5) targets only costs 33.58%
(or 55.72%) computational time on CIFAR-10, 3.1 % (or 5.07%) computational time CIFAR-I00; 33.33% (or 55.49%) computational time on SVHN.
Our experiments show that with the help of STARS method, MM3 attack saves 76.36% of the computational time on CIFAR-10, 98.51 on CIFAR-100 and 77.78% on SVHN.
3.3 MINIMUM-MARGIN ATTACK
With the above considerations, we summarize our scheme of MM attack Algorithm 1. We follow the setting of the adaptive step size selection in Croce & Hein (2020) and specify checkpoints W (wo = 0, ..., wn) ) at which the MM attack decides whether to halve the current step size. The two conditions in Algorithm are:
w,-1 Σ
Condition
(x(41)))(()) < B. (wj wi-1),
(9)
imwry
Condition 3. a - = " and [ s Iaaz.
(10)
EXPERIMENTS
In this section, we justify the efficacy of our MM attack. In the experiments, we consider -nrm bounded perturbation (i.e., ||æ" -x|loc S e) in both training and evaluations. The images of all datasets normalized into [].
4.1 BASELINES
We evaluate 3 versions of our MM attack and compare them with 8 baselines. MM3 (or MM5) is the version of MM attack with maximum step number = 20 and targets selection number K, = (or K2 5); MM+ is the version of MM attack with maximum step number K 100 and targets selection number K, = 9. Baselines consist of PGD (Madry et al., 2018), CW (Carlini & Wagner, 2017b), APGD-CE, APGD-DLR, FAB, Square, AA and T-AA (Croce & Hein, 2020).
7

---

Under review as a conference paper at ICLR 2022
Table 3: Test accuracy (%) of adversarial training.
Methods
PGD
Difr
CW
Ditt.
MM3-F10
Dif..
MM3-F20
DuI.
MM3
Diff.
PGD (Test)
51.14
-4.00
51.47
-3.77
54.96
-0.28
55.24
0.00
55.04
-0.20
(Test)
53.26
0.00
51.18
51.84
A-CB (Test)
48.58
-3.92
48.16
-..4
51.55
00.5
52.50
0.00
52.22
40.28
A-DLR (Test)
52.76
50.29
FAB (Test)
47.28
-1.22
47.13
-1.37
47.83
-0.67
48.28
-0.22
48.50
-0.00
Square (Test)
54.46
-0.66
55.32
0.00
54.80
54.83
55.12
-0.20
AA (Test)
46.43
-1.85
46.36
-1.92
47.62
-0.66
47.84
-0.44
48.28
-0.00
46.12
45.26
-1.83
46.39
40.36
47.09
MM3 (Test)
46.69
-1.17
46.77
-1.09
47.20
-0.66
47.48
-0.38
47.86
-0.00
MM9 (Test)
46.21
-0.95
45.36
-1.80
46.49
-0.34
47.16
-0.00
MM+ (Test)
46.12
-0.90
45.22
-1.80
46.39
-0.63
46.68
-0.34
47.02
-0.00
4.2
EXPERIMENTAL SETUP
We verify our methods on different neural networks including ResNet-18 (He et al., 2016) and Wide-ResNet-34 (WRN-34) (Zagoruyko & Komodakis, 2016) using different datasets: CIFAR- 10, CIFAR-100, SVHN. The adversarial training (AT) method in this paper follows (Madry et al., 2018). The training setup follows previous works (Madry et al., 2018; Zhang et al., 2019) that all networks are trained for 100 epochs using SGD with 0.9 momentum. The initial learning rate is 0.1 (0.01 for SVHN), and divided by 10 at epoch 60 and 90, respectively. The weight decay is 0.0002 (0.0035 for SVHN). The previous work (Rice et al., 2020) observed that overfitting in robust adversarial training hurts set performance. Thus, following Rice et (2020), we compare different methods based on the performance of the best checkpoint model (the early stopping results at epoch 60). For generating the adversarial data for updating the network, we set the -nom bounded perturbation Graii = 8/ /555; the maximum number of PGD steps is K = 10; step size = Crrin/ 11.. In testing, unless otherwise specified, we set norm bounded perturbation Étest = 8/25 /25 we follow the setting in (Zhang et al., 2020a): the maximum number of steps K 20, and the step size a e/4. For APGD-CE, APGD-DLR, FAB and their corresponding targeted version, we follow the setting in (Croce & Hein, 2020): the maximum number of steps K 100. There is random start in training and testing, i.e., uniformly random perturbations (( (trrinn +Ctaiin and [-Ctest, + added to natural instances.
PGD and CW,
4.3 PERFORMANCE EVALUATION
We report the performance of our MM attack and all baselines on CIFAR-10, CIFAR-100, SVHN with the model structure chosen ResNet-18. As shown in Figure 4, first, our MM attack can perform better than any single attack of PGD, CW, A-DLR, A-CE and FAB; second, compared with the ensemble of diverse attacks AA and T-AA, our MM attack achieves comparable performance but only incurs very small amount of computational time. Extensive experiments on the large-capacity network WRN-34 are provided Appendix
4.4 ADVERSARIAL TRAINING WITH MM ATTACK
By injecting adversarial examples into the training data, adversarial training (AT) methods seek to train an adversarial-ron deep neural network whose predictions are locally invariant in a small neighborhood of its inputs. Existing empirical defense methods formulate the adversarial training a max optimization problem (Madry et al., 2018). Since a large number adversarial examples need be generated during training, practitioners pay great attention the time cost of adversarial
8

---

Under review as a conference paper at ICLR 2022
6000
54
E22
4000
50
.0.55
.1.5
2000
4512
50
285
136
J82
Squard FGD
CW A
FAB MM4
AA
MMS
T-M. MB
Square
PGD
CW A
FAB
MM3
MMS
TAA MM
Attacke
Attacks
(a) Evaluation CIFAR-10
(b) Computational time on CIFAR-10
3000
30
37.34
Accuacy (%)
256
2115
2136
2A53 22A3
20
15000
500
217
MXS
MM3
Attacks
Attacks
(c) Evaluation on CIFAR-100
(d) Computational time on CIFAR-100
58
77
25000
50
Accuracy (00
3.54
20000
51.55
Time
617B
5000
431
46
.7L
202
Squore PCD
UN A-DRAACE
FAB
MMS AA
MMS
F-A MM
Square
PUD
CW
A-ULR A-UE
FAB
MM3
AA
MMS
T-AA
MM
Allacks
Attacks
Evaluation SVHN
(1) Computational time SVHN
MM attack (MM3, MM5 and MM+ mentioned Scction 4.1) with 8 baselines. In subfigure (a), (c) and (c), the Y-axis the accuracy of the attacked model, which means that the lower the accuracy, the stronger the attack (or to say the better evaluation). In subfigure (b), (d) and (f), the Y-axis is computational time, which means the less the time, the higher the computational efficiency.
Figure 4: Comparison of reliability and computational cost. We compare three versions
example generation. Compared with AA, MM attack contributes a feasible and reliable method to generate high-quality adversarial examples in AT. Table 3 reports the performance. MM3-F10 (MM3-F20) denotes AT using adversarial examples generated by MM attack with 10 (20) fixed steps. MM3 denotes AT using adversarial examples generated MM attack with 20 adaptive steps. We choose 11 testing methods to evaluate the robustness of adversarial trained models. As shown in Table by replacing PGD generated adversarial examples with MM attack generated adversarial examples, the robustness of AT model is significantly improved.
5 CONCLUSION
In this work, we proposed MM attack, which can reliably and efficiently evaluate adversarial robustness. For its reliability, we identified minimum- -magiin (MM) as the key evaluation criterion for the most adversarial example. For its computational efficiency, we proposed an effective STARS method to ensure that computational time independent of the number of classes. Our experiments showed that MM attack achieves comparable performance compared with AA, but only costs 3% of the computational time. Its reliability and efficiency further allow us to extend MM attack into AT, which significantly improves the quality of adversarial examples in AT.

---

Under review as a conference paper at ICLR 2022
ETHICS STATEMENT
This paper does not raise any ethics concerns. This study does not involve any human subjects, practices to data set releases, potentially harmful insights, methodologies and applications, potential conflicts of interest and sponsorship, dliscimination/bissfaieess concerns, privacy and security issues, legal compliance, and research integrity issues.
REPRODUCIBILITY STATEMENT
To ensure the reproducibility of experimental results, we will provide a link for an anonymous repository about the source codes of this paper in the discussion phase.
REFERENCES
Daniel Andor, Chris Alberti, David Weiss, Aliaksei Severyn, Alessandro Presta, Kuzman Ganchev, Slav Petrov, and Michael Collins. Globally normalized transition-based neural networks. arXiv preprint a6Xiv:1603.cmm 06042, 2016.
Nicholas Carlini and David Wagner. Adversarial examples are easily detected: Bypassing ten detection methods. In Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, 2017a.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In CVPR, 2017b.
Davide Castelvecchi. Deep learning boosts google translate tool. Nature News, 2016.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In ICML, 2020.
Li Deng and Dong Yu. Deep learning: methods and applications. Foundations and trends in signal processing, 2014.
Phoebe MR DeVries, Fernanda Viégas, Martin Wattenberg, and Brendan Meade. Deep learning of aftershock patterns following large earthquakes. Nature, 2018
Andre Esteva, Brett Kuprel, Roberto A Novoa, Justin Ko, Susan M Swetter, Helen M Blau, and Sebastian Thrun. Dermatologist-level classification of skin cancer with deep neural networks. Nature, 2017.
Samuel G Finlayson, John Bowers, Joichi Ito, Jonathan L Zittrain, Andrew Beam, and Isaac Kohane. Adversarial attacks on medical machine learning. Science, 2019.
Ruize Gao, Feng Liu, Jingfeng Zhang, Bo Han, Tongliang Liu, Gang Niu, and Masashi Sugiyama. Maximum mean discrepancy test is aware of adversarial attacks. In ICML, 2021.
Goodfellow, Yoshua Bengio, and Aaron Courville. Deep leurning. MIT press, 2016.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In ICLR, 2015.
Luigi Grippo, Francesco Lampariello, and Stephano Lucidi. A nonmonotone line search technique newton's 's method. SIAM journal on Numerical Analysis, 1986.
Shuang Hao, Bin Tang, Zhenyu Wu, Kerstin Ure, Yaling Sun, Huifang Tao, Yan Gao. Akash J Patel, Daniel Curry, Rodney Samaco, et al. Forniceal deep brain stimulation rescues hippocampal memory in rett syndrome mice. Nature, 2015.
Kaiming He. Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016.
10

---

Under review as a conference paper at ICLR 2022
Geoffrey Hinton, Li Deng, Dong Yu, George E Dahl, Abdel-rahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Tara N Sainath, et al. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal processing magazine, 2012.
Hengfeng Huang, Bowen Xiao, Huixin Xiong, Zeming Wu, Yadong Mu, and Huichao Song. Appli- cations of deep learning to relativistic hydrodynamics. Nuclear Physics A, 2019.
Ling Huang, Anthony Joseph, Blaine Nelson, Benjamin IP Rubinstein, and Doug Tygar. Ad- versarial machine learning. In Proceedings of the 4th ACM workshop on Security and artificial intelligence, 2011.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolu- tional neural networks. In NeurlPS, 2012.
Alexey Kurakin, lan Goodfellow, Samy Bengio, et al. Adversarial examples in the physical world. In ICLR, 2017.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 1998.
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. Nature, 2015.
Yoav Levine, Or Sharir, Nadav Cohen, and Amnon Shashua. Quantum entanglement in deep learning architectures. Physical review letters, 2019.
Aleksander Madry. Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant adversarial attacks. In ICLR, 2018.
Amy Maxmen. Deep learning sharpens views of cells and genes. Nature, 20.
Amy Maxmen. Machine learning spots natural selection at work in human genome. Nature, 2018b.
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing atari with deep reinforcement learning. arXiv preprint 1rXiv:1312.5602,22 2013.
Anh Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks easily fooled: High confidence predictions for unrecognizable images. In CVPR, 2015.
Leslie Rice, Eric Wong, and Zico Kolter. Overfitting adversarially robust deep learning. In ICML, 2020.
David Silver, Aja Huang, Chris Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schritwieser,, Ioannis Antonoglou, Veda Panncershelvam, Marc Lanctot, et al. Mastering the game go with deep neural networks and tree search. Nature, 2016.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In ICLR, 2014.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurIPS, 2017.
Yisen Wang, Xingjun Ma, James Bailey, Jinfeng Yi, Bowen Zhou, and Quanquan Gu. On the convergence and robustness of adversarial training. In ICML, 2019.
Sarah Webh. Deep learning for biology. Nature, 2018.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In BMVC, 2016.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P. Xing, Laurent El Ghaoui, and Michael I. Jordan. Theoretically principled trade-off between robustness and accuracy. In ICML, 2019.
Jingfeng Zhang, Xilie Xu, Bo Han, Gang Niu, Lizhen Cui, Masashi Sugiyama, and Mohan Kankan- halli. Attacks which do not kill training make adversarial learning stronger. In ICML, 2020a.
Yonggang Zhang. Ya Li, Tongliang Liu, and Xinmei Tian. Dual-path distillation; unified framework to improve black-box attacks. In ICMI., 2020b.
11

---

Under review as a conference paper at ICLR 2022
A THE COMPUTATIONAL TIME IN THE WORST CASE
In this section, we discuss the computational time in the worst case. According to their attack mechanism as an ensemble of diverse attacks, AA and T-AA consider one attack first. If the attack succeeds, stop other attacks the current example; else, continue to consider the next attack in the ensemble. According to the strategy of our STARS method, MM attack considers the false target with the largest predicted probability first, if the attack succeeds, stop attacks other false targets; else, continue to consider the next target the ranking of the predicted probability. The computational time of these methods is influenced by different datasets and models. Hence, in the worst attacks inside fail to succeed, the computational time is the sum of the individual time of each attack. Hence, the computational cost of AA (or T-AA) 109 times (or 440 times) more than PGD, and 34 times (or 139 times) more than MM3 in this case.
that all
B THE REALIZATION OF ADVERSARIAL TRAINING OF MM ATTACK
We summarize the adversarial training of MM Attack Algorithm We use MM3 attack to generate adversarial examples, and the computational time is about times much as PGD (Madry et al., 2018), which can be acceptable for most practitioners.
C POTENTIAL BENEFITS OF DIVERSE RESCALINGS
We investigate the difference among different successful sets of seven rescaling methods mentioned above. In Table 2, the setting follows (Madry et al., 2018) (with 20 fixed steps). In Table 4, the setting follows (Croce & Hein, 2020) (with 100 adaptive steps). In Table 2 and Table 4, the non-empty difference sets A U B; A and A U B; B; suggest that diverse rescaling methods can complement each other. Hence, when considerable computational resources are available, we recommend practitioners to consider diverse logits rescaling on strong attack (e.g., our MM attack) rather than diverse weak attacks. Note that we do not argue that diverse weak attacks is unnecessary but rather that when a reliable enough attack exists, most relatively weak attacks have limited benefits other than increased computational cost.
D THE REPLACEMENT OF NATURAL DATA FOR THE RANKING IN STARS
In STARS method, also investigate the difference of replacing the natural input with adversarial examples. Table 5 shows that the replacement has limited improvements.
E DETAILED EXPERIMENTAL RESULTS
To verify the rationality of minimum mariin, we conduct experiments on different step size, different step number and different B, [x] in Table 6 and Table 7. We compare the reliability and the computa- tional time between MM attacks and baselines. In Table 8 and Table 9, unless specified, the model structure is ResNet- 18. The experiments verify that our MM attack achieves comparable performance but only incurs a very small amount of computational time.
F EXPERIMENTAL RESOURCES
We implement all methods on Python 3.7 (Pytorch 1.7.1) with an NVIDIA GeForce RTX 3090 GPU with AMD Ryzen Threadripper 3960X 24 Core Processor. The CIFAR-IO 10 dataset, the SVHN and the CIFAR- 100 dataset can be downloaded via Pytorch. Given the 50, 000 images from the CIFAR-10 and CIFAR-100 training set, 73, 257 digits from the SVHN training set, we conduct the adversarial training on ResNet-18 and Wide ResNet-34 for classification.
12

---

Under review as a conference paper at ICLR 2022
Algorithm 2 Adversarial Training of MM attack.
1: Input: network architecture ff parametrized 8, training dataset S, loss function (, learning rate n, number epochs T. batch size nt;
2: Output: Adversarial robust network fo; do
3: for epoch = 1,
4: for mini-batch .,, do
Sample mini-batch {(2), y6) )=))== from S; for 2.2.....
7:
Obtain adversarial data of MM attack v', of ; by Algorithm I; end for
8:
9:
A 4
Vol (Je (2'2) 2..2),.0
end for
11: end for
Table 4: The successful set of different rescaling methods.
ID
Rescaling method
Formulation
Successful set
Ranking
dift.
A B,
A
A B. B.
A
Natural lngits
- (Zy #()
5379
B1
Softmax
meRt S.N cff -1
5377
3
5
B2
Ry Fyy
Ba
Sum
Ry
5374
sy Sy-ti
F3
Min-Max
Fy
S376
Sxy F9i10
B5
DLR
5372
2
Sey 2 (Fing +
Bg
Sigmoid
5311
-68
2
70
Table 5: Test accuracy (%): Replacing natural data with adversarial data in STARS method.
Dataset
Reference attack
Select-e
MM3
Diff.
MM9
Diff.
CIFAR-10
8/255
48.23
47.81
CTPAR- 10
FGSM
8/255
48.05
-0.24
47.81
0.00
CIFAR-10
PGD-20
8/255
47.92
-0.11
47.81
0.00
PGD-20
6/255
47.81
CIFAR 10
PGD.20 20
4/255
48.04
.0.23
47.81
0.00
SVHN
None
8/255
52.45
-0.61
51.84
0.00
FGSM
8/255
$2.07
51.84
SVHI
PGD-20
8/255
51.97
-0.13
51.84
0.00
SVHN
PGD-20
6/255
$2.00
-0.16
51.84
0.00
SVHN
PGD-20
4/255
52.07
-0.23
51.84
0.00
CIFAR 100
None
8/255
23.92
.0.41
23.51
0.00
CIFAR-100
FOSM
8/255
23.63
-0.12
23.51
0.00
CIFAR-100
PGD-20
8/255
23.57
23.51
CIFAR-100
PGD-20
6/255
23.57
-0.06
23.51
0.00
CIFAR- 100
PGD-20
4/255
23.63
.0.12
23.51
0.00
13

---

Under review as a conference paper at ICLR 2022
Table Test accuracy (%): the rationality of MM under different step sizes and step numbers.
Step
Step
CIFAR-I0
0.003
20
51.14
-3.33
49.95
-2.14
48.23
-0.42
47.81
1/255
40
50.16
-3.15
49.13
-2.12
47.46
-0.45
47.01
0.00
1/255
20
50.28
-3.22
49.19
-2.13
47.50
-0.44
47.06
0.00
1/255
40
49.30
-2.92
48.45
-2.07
-0.50
46.38
2/255
50.54
-3.26
49.38
-2.10
46.70
-0.42
47.28
0.00
2/255
20
49.36
-2.93
48.48
-2.05
46.92
-0.49
46.43
0.00
4/255
10
49.52
-2.97
48.60
-2.05
47.02
-0.47
46.55
0.00
SVHN
0.003
20
57.68
-5.84
54.42
-2.58
52.45
-0.61
51.84
0.00
1/255
40
56.03
-5.78
$2.90
-2.65
50.91
40.66
50.25
0.00
1/255
20
-5.74
$3.69
-2.62
51.72
40.65
51.07
0.00
1/255
40
55.49
-5.50
52.59
-2.60
50.65
-0.66
49.99
0.00
2/255
10
57.30
-5.71
54.12
-2.53
52.19
-0.60
51.59
0.00
2/255
-5.32
-2.57
-0.66
4/255
10
56.16
-5.13
53.52
-2.49
51.62
-0.59
51.03
0.00
Table 7: Test accuracy (%): the rationality MM under different [xx.].
PGD-20
Dift.
CW
Dift.
MM3-F
Diff.
MM9-F
Dift.
ResNet 18
67.90
-0.70
67.20
51.14
-3.33
49.95
-2.14
48.23
-0.42
47.81
0.00
12
45,53
-4.62
43.85
-2.94
41.86
-0.95
40.91
0.00
WRN-34
70.23
-0.30
70.55
-0.62
69.94
-0.01
69.93
0.00
53.69
-2.07
53.89
-2.27
51.95
-0.33
51.62
0.00
46.76
14

---

Under review as a conference paper at ICLR 2022
Table 8: Evaluation: test accuracy (%) different datasets and model structures.
Methods
CIFAR-10
CIFAR-100
Diff.
Diff.
[WRN34] CIFAR-10
DKf..
PGD
-3.92
57.68
-10.39
CW
49.95
-3.84
25.60
-3.07
54.50
-7.21
53,90
-4.08
A-CR
48.58
-2.47
24.71
-2.18
51.55
-4.26
51.00
-1.18
A-DLR
48.85
-2.74
24.85
-2.32
50.64
-3.35
52.24
-2.42
FAB
-1.17
23.16
52.19
-4900
51.04
-1.22
Square
54.46
-8.35
27.94
-5.41
53.80
-6.51
-8.22
AA
46.43
-0.32
23.07
-0.54
48.44
-1.15
50.21
-0.39
T-AA
46.12
-0.01
22.53
0.00
47.36
-0.07
49.82
0.00
MM3
46.69
-0.58
22.98
-0.45
49.15
-1.86
50.26
-0.4
-0.19
MM+
46.11
0.00
22.53
0.00
47.29
0.00
49.82
0.00
Table 9: Evaluation: the computational time (s) different datasets and model structures.
Methods
CIFAR-10
Diff.
CIFAR-100
Diff.
SVHN
Diff.
[WRN34] CIFAR-10
Diff.
10
CW
62
-2
4
164
0
406
A.CE
289
-229
215
-155
777
-613
1910
-1504
A-DLR
305
-255
222
-162
871
-707
1901
-1495
FAB
2181
-2121
1980
-1920
6178
-6014
13809
-13403
Square
-3708
-2468
-9342
-22187
AA
3885
-3825
2187
-2127
11146
-10982
29637
-29231
T-AA
5970
-5910
2967
-2907
25116
-24952
40178
-39772
MM3
126
-66
-31
332
1668
796
-390
MM+
1421
-1311
746
686
4431
-266
10773
-00377
15


Reviews:
[{"review_id": "f_xT_PzdYef", "reviewer": "Reviewer_pCJy", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Major contributions:\n\nThe main idea has been illustrated in Figure 2. Traditional PGD attack minimizes the probability of the true label (by maximizing the loss), and the proposed minimum margin attack minimizes the margin of the probabilities between the true label and the most confusing label. To the best of my knowledge, the idea is novel in adversarial machine learning.\n\nThe computational efficiency of the MM attack is amazing. Nowadays, researchers are still using mainly PGD for training but AA for evaluation because AA is more than 100 times slower than PGD. The MM attack is 20 or even 30 times faster than the AA attack, making it possible for training with stronger adversarial examples besides faster evaluation of the adversarial robustness of given deep learning models. In my opinion, the results are significant.\n\nConcerns:\n\nThe paper lacks some theoretical analysis, for example, how would the attack converge, how to guarantee the minimum probability margin example is stronger than the minimum probability example (and thus more informative for both evaluating and training), and whether the iterative MM attack algorithm is as stable as the PGD and AA algorithms.\n\nThe experiments mainly focused on the evaluation part and then the training part is quite weak. Although researchers believe stronger adversarial examples lead to more robust models, it is not always the case because adversarial examples can be generated by quite different underlying principles as minimum probability vs. minimum probability margin. It is better to concretely show that MM is almost as fast as PGD and almost as strong as AA for training besides for evaluation. This is quite critical for the significance of the paper. ", "summary_of_the_paper": "The paper proposed a strong adversarial attack, i.e., an attack that can generate strong adversarial examples and thus can better evaluate the adversarial robustness of given deep learning models. Compared with the SOTA attack, the proposed attack is much faster and thus easier to be applied in practice. The idea is novel and the results are solid.\n", "summary_of_the_review": "This is an overall well-executed paper, with good novelty and solid experiments. Some points should be clarified and stregnthened in the revision. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "ayTXEckd31H", "reviewer": "Reviewer_BW92", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "### Comments:\n\n- The threat model is not stated anywhere. There is no definition of adversarial robustness/robust accuracy. It is not clear then if (1) the attack is a minimum-distance or maximum-confidence attack, (2) it is a targeted or untargeted attack in the common sense used in the field (as opposed to the \"targeted\" version of APGD, that is instead just using the \"targets\" for reducing the number of adversarial classes to consider in the optimization), and (3) if the attack is only defined in the $\\ell_\\infty$ norm.\n\n\n### Flaws in the experimental evaluation\n\n- the evaluation does not consider a state-of-the-art attack such as [brendel2020]. This is a pity, as [brendel2020] is presented as a \"fast and reliable method\" for evaluating robustness, and has similar desired characteristics as this attack.\n- The parameters of the attacks used seem sub-optimal. There is no choice of the hyperparameters, and using 10 steps for PGD seems to be limiting the capabilities of the attack. The same can be true for the CW attack, especially as there is no mention on how many binary-search steps are being used. It is OK to test the attacks with limited resources, but a more detailed asymptotic analysis (e.g. with 1k steps) would concretely support the claims of the paper that the attack remains comparable to the other attacks while reducing computational time.\n- The runtimes are computed in an uneven scenario. The total time per-step, or better, per-query to the model should be used instead of the total cumulative time. This makes no sense. As an alternative, one should compare the capabilities of the PGD attack (depicted here as fast but not reliable) with a fixed computational time, i.e., by increasing the number of steps performed by PGD until it spends the same amount of time as the MM attack.\n- the authors did not state if they used some available implementations of the attack, or implemented their own versions. Since the computational time depends on the implementation, this might be a problem when using the runtime as a benchmark.\n\n### Incorrect statements and unsupported claims\n\n**Abstract**\n\n- there is no definition of the \"most adversarial example\", even though there are several references of this in the paper. This is also used in the abstract. Depending on the objective, a stong adversarial example can be seen in different ways. I suggest to expand this with a definition.\n- there is no evidence suggesting that the PGD attack is 100 times slower than AA. The comparison is performed in uneven scenario, where AA uses 100 iterations while PGD uses 10. Moreover, this is stated in the abstract, which makes the statement easy to take and quote, without knowing the context. This statement should be removed.\n\n\n**Introduction**\n\n- \"for practitioners who need real-time evaluation at each epoch of the training process of a robust model\". Is there real cases that require this kind of evaluation? This is missing a reference.\n- \"Unfortunately, PGD fails to reliably evaluate adversarial robustness of a robust DNN\". This sentence is over-generalistic and not true for the majority of the cases. PGD was succesfully used against many defenses, just by making it adaptive to the defense [tramer2020].\n- \"CE loss, which is based on the probability of the true label $p_y$, is not an appropriate measure to the quality of adversarial examples\". There is no definition in the paper for \"quality of adversarial examples\", which makes this statement very confusing.\n- \"Hence, the reliable method is to minimize $z_y - z_t$ for each $t \\neq y$ and take the most adversarial one, which is a widely used solution\". This is not widely-used, as for now it seems only used in [croce2020].\n\n**Preliminary**\n\n- \"$x^{(0)}$ refers to the starting point which corresponds to the natural example (or the natural example perturbed by a small Gaussian or uniformly random noise)\". The statement within parentheses makes the definition of the closed ball in eq. 2 makes the ball centered in $x^{(0)}$. This does not correspond to the adversarial robustness measured in the original clean sample.\n- many equations (see Eqs. 3-7) depend on f, x, y, but they often don't appear inside the equations.\n- \"They showed that using adaptive step size significantly improves the adversarial robustness\". Should be \"improves the adversarial examples\" or \"improves the adversarial evaluation\". The attacks are not improving robustness.\n\n**Realization**\n\n- Eqs. 8 and 9 use variables ($\\alpha$, $\\beta$) never introduced in the text.\n\n\n### Minor issues\n\n- the comparison with targeted-dlr loss in sect. 3 should be clarified. It is very difficult to read, and it does not really capture the advantage of using different methods for rescaling. This might be better supported by some evidence or toy example, and surely by adding some insight on which the hypothesis is based on. Moreover, the authors should then explain what is the difference from the CW loss, as it seems that they are using that one.\n- Figures and tables need descriptive captions that clarify what is being depicted. In particular, tables need improvements in the headers and some highlighting of the results. It is also a good practice to mention them in the text (Figure 1b). Figure 2 is difficult to understand, and contains a legend with unclear definitions (see \"classified area\"). In table 2, it is impossible to understand what are the values presented in the cells. The algorithm needs some hints/comments/description.\n\n\n### References:\n\n- [tramer2020] Tramer, Florian, et al. \"On Adaptive Attacks to Adversarial Example Defenses.\" Advances in Neural Information Processing Systems 33 (2020).\n\n- [croce2020] Croce, Francesco, and Matthias Hein. \"Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks.\" International conference on machine learning. PMLR, 2020.\n\n- [brendel2020] Brendel, W., et al. \"Accurate, reliable and fast robustness evaluation.\" Thirty-third Conference on Neural Information Processing Systems (NeurIPS 2019). Curran, 2020.", "summary_of_the_paper": "The paper proposes an attack for testing adversarial robustness that is reportedly faster than the state-of-the-art attacks but still produces reliable results. The advantage in speed is obtained by using a sequential target ranking selection method, while reliability is achieved by using a minimum-margin loss.\n", "summary_of_the_review": "### Strengths: \n- Tries to improve efficiency of adversarial attacks\n\n### Weaknesses:\n- the paper is missing some definition that should not be taken for granted\n- the evaluation is not entirely convincing and might be unfair\n- results should be presented better, as they are very difficult to read and understand", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "RxbXZXcSIFI", "reviewer": "Reviewer_Zhx4", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n- This paper is well-written, especially with detailed descriptions and empirical results on the effects of different attacking loss functions.\n- The improvements shown in Figure 1 seems promising with significant saving on computation.\n\nWeaknesses:\n- Although several attacking baselines are considered, they are all only evaluated against PGD-AT (Madry et al. 2018). This could cause a biased evaluation of the attacking performance. Namely, as a potential substitute for AA, the proposed MM should be widely tested against different defenses, just as done in the AA paper (Croce & Hein, 2020). This should not be computationally hard, considering the efficiency of MM and many existing defenses (and their checkpoints) provided in, e.g., RobustBench.\n- The multi-target attacking strategy has already been proposed in [a], but I'm surprised that [a] is not even cited in this paper. For me, the proposed STAR strategy is just a top-K variant for the original multi-target attack. Besides, using logits rather than softmax outputs is also not a new discovery since Carlini & Wagner (2017b). Thus, the technical contribution and novelty of MM are quite limited.  \n\nMinors:\n- What is the definition of $l\\_{MM}$ in Algorithm 1?\n\nReferences:\n[a] Gowal et al. An alternative surrogate loss for pgd-based adversarial testing, 2019.", "summary_of_the_paper": "This paper proposes a minimum-margin (MM) attack to evaluate defenses. The authors report detailed results on the effects of different loss functions. Experiments are done on CIFAR-10/100 and SVHN, against the adversarially trained models.", "summary_of_the_review": "Limited technical contribution, lack of evaluations against more defenses.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "JjjvbSLcC1j", "reviewer": "Reviewer_s3fo", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n\n-- The paper is well-written and the preliminaries are described clearly. \n\n-- The proposed method presents significantly low computational complexity. \n\nWeaknesses:\n\n-- The proposed method is only compared against PGD and CW. \n\n-- The authors have mentioned that: \"For reliability, we evaluate the quality of adversarial examples using the margin between two targets for precisely identifying the most adversarial example.\". Can you please explain about \"most adversarial example\" ?  \n\n-- $\\beta$ in Equation 9 is not defined.", "summary_of_the_paper": "The authors propose minimum-margin (MM) attack to provide comparable performance with AutoAttack while significantly decreasing the computational cost. They propose Sequential TArget Ranking Selection (STARS) to make the computational cost independent of the number of classes. ", "summary_of_the_review": "Although, I believe the proposed method has the potential for a good publication, I do not recommend the acceptance of the paper in the current form.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper focuses on the strong adversarial attack, i.e., an attack that can generate strong adversarial examples and thus can better evaluate the adversarial robustness of given deep learning models. One review gave a score of 8 while the other 3 reviewers gave negative scores. The main issue lies in the limited experiments, as a potential substitute for AA, the proposed MM should be widely tested against different defenses, just as done in the AA paper. The writing of the paper is somehow is not rigorous including many incorrect statements and unsupported claims which should be well addressed in the revision. Thus, it cannot be accepted to ICLR for its current version.
Decision: Reject