A2: Efficient Automated Attacker for Boosting Adversarial Training
Authors: Zhuoer Xu
Guanghui Zhu
Changhua Meng
shiwen cui
Zhenzhe Ying
Weiqiang Wang
Ming GU
Yihua Huang
Venue: NeurIPS 2022 Accept (2022)
Keywords: Adversarial Training
Automated Machine Learning

Abstract: Based on the significant improvement of model robustness by AT (Adversarial Training), various variants have been proposed to further boost the performance. Well-recognized methods have focused on different components of AT (e.g., designing loss functions and leveraging additional unlabeled data). It is generally accepted that stronger perturbations yield more robust models.
However, how to generate stronger perturbations efficiently is still missed. In this paper, we propose an efficient automated attacker called A2 to boost AT by generating the optimal perturbations on-the-fly during training. A2 is a parameterized automated attacker to search in the attacker space for the best attacker against the defense model and examples. Extensive experiments across different datasets demonstrate that A2 generates stronger perturbations with low extra cost and reliably improves the robustness of various AT methods against different attacks.

A Efficient Automated Attacker for Boosting Adversarial Training
Zhuoer Xu Guanghui Zhu Changhua Meng", Shiwen Cui,, Zhenzhe Ying",
Weigiang Wang', Ming Gu², Yihua Huang'
State Key Laboratory for Novel Software Technology, Nanjing University TTiansua Lab, Ant Group
zhuoer. www.maul........ jnju. edu. en, zghōnju. edu. {changhua. mch, donn csw zhenzhe. yzz, weiqi www. mggta)gggruupcco com, yhuangen jwr.ggdnu...uu.crr edu.cr.
edu. en,
Abstract
Based the significant improvement of model robustness by AT (Adversarial Training), various variants have been proposed to further boost the performance. Well-recognized methods have focused on different components of AT (e.g., de- signing loss functions and leveraging additional unlabeled data). It is generally accepted that stronger perturbations yield more robust models. However, how generate stronger perturbations efficiently is still missed. In this paper, we propose an efficient automated attacker called A to boost AT by generating the optimal perturbations on-the-fly fly during training. A is a parameterized automated attacker to search the attacker space for the best attacker against the defense model and examples. Extensive experiments across different datasets demonstrate that 2 generates stronger perturbations with low extra cost and reliably improves the robustness of various AT methods against different attacks.
Introduction
DNNs (Deep Neural Networks) are extremely vulnerable to imperceptible perturbations despite their success in wide variety of applications [He et al.) 2016 Kenton and Toutanova] 2019 Guo et al. 2017]. In particular, adding small but carefully chosen deviations to the input, called adversarial perturbations, can cause DNNs make incorrect predictions with high confidence. indicates that models trained by minimizing the empirical risk are not intrinsically robust. To explicitly improve robustness, AT (Adversarial Training), where a defense model trained on worst-case adversarial perturbations generated by an attacker, was developed and proved to be effective.
Based on the significant improvement of models' robustness, various methods have been pro- posed, which focus on different components of AT: analyzing the robustness of neural architectures Zagoruyko and Komodakis (2016). designing loss functions such as TRADES [hhng et al. 2019 and MART Wang et al.) 2019). perturbing the model to regularize the loss landscape's flatness (i.e., AWP Wu al. [2020] and leveraging unlabeled data (i.e., RST [Cammon al. 2019 Gowal et al.)
(220) compares the performance of each combination of most components and achieves the SOTA (state-of-the-ee performance.
All the above AT methods use PGD* [Madry et al. 2018 which is K K-step stack of FGSM (Good fellow al. 2013]. as the attacker to generate perturbations for each example against the defense model. As the key of AT, stronger perturbations yield more robust models. However, there is trade-off between the strength of the perturbation and the training efficiency. Increasing the attack step K strengthens the perturbations, but linearly increases the training overhead (Gowal et al. 2020
Cooresponding author.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

Likewise, the huge overhead prevents the use of SOTA adversarial attackers [Croce and Hein 2020 Yao al. 202T]. To achieve a balance of robustness and efficiency, manually tuning the attacker (e.g., step size and attack method in each step) is of great concern. R+FGSM [Wong et al. 20199 first randomly initializes a small perturbation, and then applies FGSM with the tuned step size. Surprisingly, AT with R+FGSM is as effective as PGDK but has significantly lower cost.
However, given the novel dataset, tuning the attacker manually is a challenging task requiring expert knowledge. Moreover, the best attacker is fine-grained to each example and current model during adversarial training. Manual coarse- -ersiied tuning for the whole training (e.g., fixed attack method and step size for all examples) is sub-optimal and prevents further improvement of robustness.
Inspired by AutoML (Automated Machine Learning (Zoph and Le 2017 Liu al. 2018).
propose an efficient automated attacker called A to boost AT. A is parameterized attacker, which automatically tune itself on-the-fly during training to generate worst-case perturbations for each example. First, we design a general attacker space by referring to existing attackers. The attacker space is stacked by one-step attacker cells. Each cell consists of a perturbation block and step size block. Then, we employ parameterized attacker to search for operations in each block and construct the attacker for each example that maximizes the model loss. Specifically, leverage the attention mechanism to calculate the score of each operation. For continuous operations, we sum up the operations using the normalized scores as weights. For discrete operations. we use the reparameterization trick (Jang et al. [2017] to sample an operation from the corresponding block. In this way. the constructed attacker generates worst-case adversarial perturbations to train the defense model. Meanwhile, A is differentiable and can be optimized with respect to the model loss by gradient descent.
We conduct extensive experiments to verify the effectiveness and efficiency of A2, Compared with PGD, A can find better perturbations for different models trained with various AT methods. The results demonstrate that 20-step A² generates better perturbations than PGD Moreover, combine A with other AT variants, and the robustness of models with different architectures on various datasets is generally improved under strong attacks (e.g., classical C& W and SOTA AutoAttack). We also show that A is insensitive its hyperparameters.
To summarize, our main contributions can be highlighted as follows:
We propose an efficient automated attacker called A which can generate worst-case perturbations on-the-fly during training to improve robustness.
In A2, we design an attacker space by summarizing the existing attackers and employ a differentiable method construct the most adversarial attacker for each example according to the attention mechanism.
Extensive experimental results across different datasets and neural architectures demonstrate that A improves the model's robustness by generating stronger perturbations in the inner maximization. Moreover, 2 can be flexibly combined with different AT methods, showing good generality.
2 Preliminary: Adversarial Training
Let D (X,Y) = yx), be dataset with x; E Rd as a natural example and yr (l....,Q C) as its associated label. We measure the performance of DNN classifier parametrized with 0 using a suitable loss function denoted as B(x,yieD saddle point problem whose goal is to find the model parameters 0 that minimize the adversarial risk in the outer minimization (the example's index is omitted for brevity):
(x)) AT (Madry et al.) 2018) formulates a
inner maximization
min E ED
max (So (x+6),,) 863
(I)
outer minimizatiinn
where defines the set of allowed perturbations. The perturbation is usually constrained by Lp norm with bound E, i.c. S [6]]|0]I 5 E).

---

The inner maximization aims to find an adversarial perturbation against the example that achieves high loss for the defense model. However, is NP-hard to find the optimum of the inner maximization. Various gradient-based attackers have been proposed to approximate its solution, and we classify them according to the number of steps gradient ascent. One-step attackers [Goodfellow et al. 2015 Miyato et al. 2017] generate adversarial perturbations as:
x flg 17 d/ (Vx)
where Vx is short for (fo (x) y), 17 is the step size, is transformation function (e.g., sgn in FGSM and Identity in FGM) and II is the projection. Since such linearization attacks tend to be trapped in the non-smooth vicinity of the data point, R+FGSM initializes small random perturbation to escape the vicinity and then applies FGSM. As typical multi-step attacker, PGD [Madry et al. 2018 can find better perturbations by K step gradient ascent:
(k) -xx+ (x(--l) +1- vp (V (V(a..))
(3)
3 Methodology
3.1 Motivation
The key of AT is to generate perturbations in the inner maxinization. Strong perturbation helps to improve robustness. It is generally accepted that the step K used to solve the inner maximization correlates with the attacker's ability to generate stronger perturbation. However, larger K leads to linear increase in training overhead. |Wong al. 2019) suggests that with appropriate step size tuning and early stopping, one-step attackers yield models with the robustness that is comparable to much more expensive multi-step attackers. It indicates that hyperparameters, such as random initialization. step size, momentum, and carly stopping, affect perturbation generation. From the perspective of effectiveness and efficiency, it is valuable further improve robustness by tuning the attacker to strengthen perturbations.
However, manual tuning of perturbation generation for each example on-the-fly during training is impractical. To address this problem, we propose an efficient automated attacker to boost adversarial training by generating optimal perturbations on-the-fly during training.
3.2 Problem Formulation
Inspired by AutoML, we first design a general attacker space A by referring to existing attackers. Then, employ automated attacker parameterized by a to search in A and further construct an attacker against the example and the defense model (x, y. fa). ). We abbreviate the perturbation generated by the constructed attacker 8a. Therefore, the goal of A is train robust model using the perturbation generated by the constructed attackers through a bilevel optimization problem:
min (X [1[o + 8,)
s.t. arg max ()D [I (fe (x+
On the attack side, we train by SGD to make the defense model misclassify. On the defense side, we use a" to construct the best attacker for each example and then generate perturbations to adversarially train the defense model.
3.3 Attacker Space
Revisiting most attackers, find that the attacker can be viewed as stack of one-step attackers consisting of an attack method and step size. Thus, as shown in Figure [(a) we design a general
attacker space A consisting of K-step cells. The k-th cell denoted as (R) which is a one-step attacker consisting of the following two blocks:
Perturbation Block Op- Typical attack methods (i.e., FGM and FGSM), attack methods with momentum (i.e., FGMM and FGSMM), random perturbations (i.e., Gaussian and Uniform), and the special Identity which enables the attacker to automatically early stop a certain step like FAT Zhang et al. 2020
3

---

Col
Attack Call K
Cropened Claks
Upae
Cal
Attack Cell
keys
%
leys
S
7
Altack Cell
x
Pormaraation Black
Skpp Nor diack
Altack Call 1
(a)
(b)
Figure 1: (a) Attacker Space A (b) PGD FGSM and a fixed step size 2/255 in each cell; (c) R+FGSM: Gaussian in the first cell, FGSM in the second cell and Identity in the other cells.
Step Size Block O,. {10- - 17. 10 -7,, 10 n, 10 n,.), where n is hyperparameter related to the space allowed perturbations
Each block O contains multiple operations. Let o(-) denote the operation, y = 11 (2) (k) denote
the choice of operation in the k-th cell. The attack methods within 0%²²) mutually exclusive. Thus, (k)
(k) is one-hot vector. In contrast, the operations within 0²²) are continuous. Yo, is normalized Yop
continuous vector, where each element represents the selection probability. To unify the categorical
choice of attack methods and the probabilities over step sizes, the output of ((k), is expressed as mixture based on (R).
Ōly!) (k) V x===-)=
Σ Yo o(Vx(-11)
(5)
oc01R)
where Yo denotes the weight of the operation o in yo Correspondingly, the one-step attacker the k-th cell can be expressed as the joint of two blocks:
V ((y())vm_)) = 0,
(2) Vaa-n) Öp Op(rop-
(k) Vza-.))
(6)
Moreover, the constructed attacker is a composition of attackers from each cell. In this way, we can cover common attackers in space. For example, shown in Figure (bb) PGD* is obtained by selecting FGSM in each perturbation block. R+FGSM in Figure [(c) is a case of selecting Gaussian in the first cell, FGSM in the second cell and Identity in the other cells.
Analysis of A Considering there exist 7 attack methods in Op of each step, there are 7 combinations of attack methods in the K-step attacker space. The exponential increasing combinations prevent the brute-force search. Moreover, the continuous step size O is also part of the attacker space. Thus, we propose 2 to search for the best attacker in A and generate adversarial perturbations efficiently.
3.4 Automated Attacker A
2 is used construct the best attacker against (x, y. fo) and its trainable parameters include (x) w(kk and W (k) where & E example query and the candidate operations as keys. Thus, the attention mechanism can be used to calculate the scores of operations within each block and the operations are selected based on their scores. Specifically, in the k-th cell, we take the gradient of the last step Viien/ (fa (x(*-1) y) as
As shown in Figure ((a). we treat the current model and
input and project it to vector space as the query using (:(k) Then, we use the trainable embedding table ('( to convert the individual operations within (k) to continuous keys. With the Scaled Dot-Product Attention (Vaswani et al. 2017), compute the dot products of the query with each key as the score of the operation o E O in the k-th cell:
(k) e, = =(V (Vxu-1) (((k) (R)
(

---

Perturbation Block. The operations within (k) are mutually exclusive. We sample operation with the normalized scores as probabilities, i.e., ( Yop ~ softmax(e (k)). o,,
Step Size Block. As the operations within 0₂²0) are continuous values, we sum up the individual step sizes with the normalized scores weights. For Or !!!), the weight can be expressed
(R) exp (Eo,) (8)
Yo,
(8)
En' exp (e)
3.5 Training of Automated Attacker
As mentioned above, we train A to minimize the following objective by gradient descent:
a' arg min (x [/ (fo (x+ 6x)
However, as result of constructing the attacker by sampling in each perturbation block, the gradient of the loss w.r. Yo, is zero. To train a, 2013) to transfer the randomness of sampling to the auxiliary noise and reformulate the objective function. For brevity, omit the step index k.
the reparameterization trick [Kingma and Welling
Let yo, = ((K,((Ke)) eo,.) be a diffcrentiable transformation where is an auxiliary noise variable with independent marginal p*)). In 2 we sample noise from Gumbel Distribution, i.c., ~ Cumbel(O) [Gumbel 1954), and use Gumbel Softmax (Jang al., 2017 as & smoothly approximate the expectation of loss [Maddison et al. 2014). Specifically, (xx, eo,)) softmax (eeo + +x/TT where T is the temperature parameter. When - O, the generated samples have the same distribution as one_hot(arg max (eop Kop)).
Using the reparameterization trick, we can now form MC (Monte Carlo) estimates of the expectation of A loss / for each example, which is differentiable, as follows:
I ([fo(x66a..)
( vw).)
=Byo sofmaarleo,)
=Bp(x)
[(fxxx+C((((xxeo))..oo vx).y)
(10)
M
Σ (x (((((((( (x) M m==
where M is the number of samples. In practice, M can achieve good performance. In this way, we reformulate the MC approximation in Equation (10) of / the objective function i.
Moreover, training a to convergence in each epoch can be prohibitive due to the expensive inner maximization in Equation (4) We use simple approximation scheme following the common
methods [Finn et al. 2017 Liu et al. 2018):
a' a + VV ((.y) [ī (fe (x)
(11)
where a denotes the current weights of the attacker and & is the learning rate.
3.6
Framework of Adversarial Training with 2
The overall procedure is shown in Algorithm M As in normal adversarial training, we generate perturbations in K steps every batch and update the model parameters. The key difference is in Line Benefiting from a parameterized automated attacker, we tune the discrete attack methods and continuous step sizes to generate adversarial perturbations. After optimizing the model parameters, we use Equation [1] to update a as an approximation to a². Since A² focus on the inner maximization, it can be compatible with most adversarial training methods. For example, it is flexible to use the loss function of TRADES or MART for outer minimization Line 10 (i.e., TRADES-A and MART-A2),
or include carly stopping in Line 5 as FAT.
5

---

Algorithm 1 Adversarial Training with Automated Attacker (AT-A?)
Input: Training examples D, perturbation bound e, the number of attack steps
1: Initialize 0.0:
2: for epoch I...., ..ep do
3: for minibatch (X, Y) C D do
4: X X;
5: for lark=,,.... K do
6:
Calculate the gradient Vxi--i;
Construct ((k) E according to Vx by gai ( = ( +
7:
8:
9:
end for
10: Update 0 with V Eix, (fo
11: Update by Equation III
end for
13:
end for
4
Experiments
We conduct extensive experiments public datasets to answer the following questions: 1) Can A generate stronger adversarial perturbations? 2) How effective the adversarial training with A?? 3) Is 22 robust to hyperparameters? All experiments are using GeForce RTX 3090 (GPU) and Intel(R) Xeon(R) Silver 4210 (CPU) instances.
4.1 Effectiveness of Automated Attacker (RQ1)
In this part, we fix the model fo. train the automated attacker alone and investigate whether A can generate more powerful perturbations compared to the commonly used PGD.
10-step A
10-step A
20-step A
20-step A
POP
K0
Bjsh
(a) ResNet-18 trained by MART
(b) WRN-34-10 trained by TRADES-AWP
4330
P23
(c) WRN-34-10 trained by MART-AWP
(d) WRN-28-10 trained by RST-AWP
Figure 2: Effect of adversarial perturbations generated by A with the training epoch.
Experimental Settings. To demonstrate the generality, choose different neural architectures (i.e., R8,Net-18, WRN-34-10 and WRN-28-101 trained on CIFAR-10 by various AT methods: TRADES, MART, RST, and AWP. All trained models open-source checkpoints. We choose P with random start ((0) Uni form(-e, e) as baseline. All attacks are boo-ddddd with total perturbation scale of 8/255. Since more attack steps generally improve the attack effect, compare the automated attacker with PGD at different steps. Moreover, try different step

---

Table 1: Comparison of attack effects (%, the lower the better) of multi-step PGD and A in robust models. We run each method 5 times and show the average. The standard deviations are omitted as they are very small. The architecture of all defense models is WideResNet, except for MART whose architecture is ResNet-18.
10-step
20-step
Defense
Natural
PGD
A²
A
PGD
A²
A2
PGD 100
MART
83.07
54.78
54.09
53.65
53.76
53.52
53.24
53.28
TRADES-AWP'
85.36
60.22
59.67
59.60
59.64
59.38
59.34
59.49
MART-AWP'
85.60
60.38
59.76
59.51
59.52
59,42
59.25
59.29
RST-AWPI
88.25
64.68
64.27
64.17
64.14
64.02
63.97
64.03
size blocks, i.e., 17. in A 1) = 2/255, which is the same as the setting of step size in PGD; 2) 7 = 8/255, which indicated by A² and allows A² to search the whole e bound each step. A is trained using Adam [Kingma and Ba 2015 with learning rate 10-3, weight decay 10-2 and other default hyperparameters.
Attack Effect. The training process of A2 is shown in Figure 2 We take the first 20 epochs of the attack effects, compare them with PGD, and observe whether A2 converges. In the early training stage, the random combinations of attack operations are much less effective. After 10-20 epochs, the effect of generated attacks is much more stable and effective. In practical automated adversarial training, the model and the automated attacker are trained iteratively. The fast convergence of A ensures that the generated perturbations are strong enough. In addition, a larger 17 achieves better attacks. As the steps increase, the effect diminishes and the training may fluctuate.
Table reports the attack effects of PGD and A with different steps. In the comparison of A2 and PGD with different steps K (10, 20), 2 stably outperforms PGD. A2, constructs stronger attacks in the expanded search space. With the increase of steps, A is more effective due to the combination of attack methods and the automated step size tuning. Due to the diminishing marginal effect, PGD 100 PGD20 achieves less improvement than PGD v.s. PGD At 1/5 of the cost, the 20-step A2, finds better attacks using the optimized a than PGD In summary, Table ] verifies that A stably outperforms PGD for the same step. and obtains better attacks compared to PGD, whose step size and attack method are fixed, with significantly lower cost.
Overhead Analysis. The overhead of A is not significant compared PGD. Both methods are close in terms of clock time. For WRN-34, PGD takes 19.75/147 0....77997777666 seconds to generate 1/10/20 step attacks respectively. demonstrates that more inner steps lead linear increase in time. Meanwhile, A takes 157.61/302.51 seconds to generate the 10/20 step attack respectively. The main overhead remains in the forward computation and backward propagation of the defense model. Moreover, Section A.3 in Appendix shows that the total parameter size of A is also acceptable.
4.2 Effectiveness of Adversarial Training with 2 (RQ2)
In this part, we evaluate the robustness of our proposed AT-A² on different datasets against white- box and ensemble attacks. To verify that the stronger attacks generated by A² on-the-fly during training can improve robustness, we consider various adversarial training methods (i.e., AT, TRADES, MART, and AWP) without additional data across different datasets.
Benchmark. We conduct experiments on the baseline AT and the SOTA AWP with A across three benchmark datasets to verify the generalization of 2 We follow the settings in AWP: PreActResNet- 18 trained for 200 epochs, é = 8/255 and y 10-2 for AWP. The step size is 1/255 for SVHN and 2/255 for CIFAR-10 and CIFAR-100. For AT and AWP, the attacker used in training is LG The 10-step 22 trained with the same setting as in RQ1. PGD used for testing, and the test robustness is reported in 2 It shows that A as component focusing on the inner maximization.
hhtts://iihub.com//iiennnii
Baunss//iittbbocommm/iieeeeetttt Adversarial Training
7

---

Table 2:
Test robustness (%, the higher the better) using PreActResNet-18 under Loo threat model ("Best" means the highest robustness while "Last" means the robustness at the last epoch). Std. of runs is omitted due to being small.
SVHN
CIFAR-10
CIFAR-100
Defense
Best
Last
Best
Last
Best
Last
AT
53.36
44.49
52.79
44.44
27.22
20.82
AT-A2
56.76
44.75
52.96
44,59
28.14
20.28
AWP
59.12
55.87
55.39
54.73
30.71
30.28
AWP-A2
61.42
58.45
55.71
55.31
31.36
30.73
Table 3: Test robustness (%, the higher the better) on CIFAR-10 using WRN-34-10 under Loo threat model ("Natural" denotes the accuracy on nature examples, and other columns indicate the accuracy on adversarial examples generated by different attacks). Std. of 5 runs is omitted due to being small.
Defense
Natural
FGSM
PGD20
CW
AutoAttack
AT
87.30
56.10
52.68
50.73
47.04
AT-A²
84.54
63.72
54.68
51.17
48.36
TRADES
84.65
61.32
56.33
54.20
53.08
TRADES-A
85.54
65.93
59.84
56.61
55.03
MART
84.17
61.61
57.88
54.58
51.10
MART-A2
84.53
63.73
59.57
54.66
52.38
AWP
85.57
62.90
58.14
55.96
54.04
AWP-A2
87.54
64.70
59.50
57.42
54.86
achieves better results on most datasets. Moreover, 2 is generic and can boost the robustness of both baseline and SOTA AT methods.
Robustness on Wete Furthermore, we train WRN-34-10 on CIFAR-10 with various AT methods (i.e., AT, TRADES, MART, and AWP) following their original papers and open-source codes?. All defense models are trained using SGD with momentum 0.9, weight decay 5 10-4, and an initial learning rate of 0.1 that is divided by 10 at the 50%-th and 75%-th epoch. Except for 200 epochs AWP, other AT methods train the model for 120 epochs. Simple data augmentations (i.e., 32x32 random crop with 4-pixel padding and random horizontal flip) are applied.
For white-box attack, test FGSM, 2GD and CW., (Carlini and Wagner 2017 In addition, we test the robustness against the standard AutoAttack [Croce and Hein 2020]. which is strong and reliable attacker to verify the robustness via an ensemble of diverse parameter- free attacks including three white-box attackers and a black-box attacker. 3able that A² reliably boosts AT variants against white-box and ensemble attacks. This verifies that A is general for AT and improves adversarial robustness reliably rather than gradient obfuscation or masking.
Additionally, given the nature examples, AT performs better than AT-A?. The main reason is that 2 generates stronger perturbation for better robustness, which decreases the accuracy (i.e., 84.54). Many works (e.g., TRADES, MART, and AWP) regularization achieve the trade-off between robustness and accuracy. The regularization also used to optimize the automated attacker. Thus, for other AT methods in Table 3 combining A can achieve higher accuracy. Moreover, for WRN-34, the training time of AWP-A is 970 s/epoch while the training time of AWP is 920 s/epoch. Thus, the additional overhead of A2 not significant.
4.3 Hyperparameters of 2 (RQ3)
The hyperparameters of A include the training hyperparameters and the design of the attacker space. The comparison of the attack effect with different hyperparameters is shown in Table 4 Overall, A2 is robust to hyperparameters and performs better than PGD and closely to 2GD

---

Table Comparison of attack effects (%, the lower the better) of 10-step A with different hyperpa- rameters (The 10-step is omitted, 22 p.a is short for training the attacker using learning rate 5 = 10 with the step size block 17 = q/ ). Std. of runs is omitted due to being small.
Attack
TRADES-AWP
MART-AWPI
RST-AWPI
PGD
60.22
60.38
64.68
PGD20
59.64
59.52
64.14
323,2
59.67
59.76
64.27
222,2
59.93
59.87
64.34
A
59.76
59.78
64.29
323.5
59.49
59.62
64.11
A²3,8
59.53
59.53
64.17 17.17
Training of A The effect of attacks with different learning rates is shown in the middle two rows of Table 4 Although Adam uses a dynamic learning rate, an excessive initial learning rate (i.e., 10-2) leads to sub-optimal.
Attacker Space. The influence of the attack step K has been investigated in RQ1. As shown in the last two rows of Table 4 larger step size increases the effectiveness of A2, However, as shown in the training curve in Figure 2 larger 77 introduces instability the training of the attacker.
Related Work
5.1 Adversarial Learning
Croce and Hein 2020) have shown that DNNs are vulnerable to adversarial examples. Various defense strategies and models have been proposed to deal with the threat of adversarial examples. However, as proved in C&W [Carlini and Wagner 2017]. many works mistake gradient obfuscation or masking for adversarial robustness. [adry al.) 2018) formulates a class of adversarial training methods for solving saddle point problem (i.e., Equation (I) and improves robustness reliably.
Many recent works [Goodfellow et 2015 Carlini and Wagner 2017
Based AT, many works [Zhang al., 2019 Wang al. 2019 Wu al. 2020 focusing the components of outer minimization are introduced to further enhance performance. The inner maximization is also the goal of the adversarial attack, where / is the 0-1 loss. Many works, c.g.. FGSM [Goodfellow et al.] 2015), C&W [Carlini and Wagner] [017]] and AutoAttack (Croce and Hein have been proposed to attack DNNs and facilitate the development of adversarial training.
2020
5.2
Automated Machine Learning
AutoMI. |Bergstra et al. automate the parts of the machine learning pipeline that require expert solutions. For particular domain, it common practice to summarize a large search space of parameters and configurations based expert experience and search for the optimal solutions using methods such black-box optimization. During the search process, a certain metric is required to evaluate each solution. The same idea can be applied adversarial learning. A Yao al. 2021), which is also closely related to AutoML, automatically discovers an effective attacker on a given model.
2011 Zoph and Le 2017 Liu et al. 2018 Cubuk et al. 2019] aims to
Conclusion
In this work, we proposed A2, to the best our knowledge, the first adversarial training method which focuses automated perturbation generation. In A?, the attacker space is designed by summarizing the existing perturbations. Moreover, the parameterized automated attacker leverages the attention mechanism to choose the discrete attack method and the continuous step size and further generates adversarial perturbations. During training, the one-step approximation the optimal automated attacker is used to generate the optimal perturbations on-the-fly for the model. The experimental

---

results show that A generates stronger attacks with low extra cost and boosts the robustness of various AT methods reliably.
For future work, we plan to add the target loss of the inner maximization to the attacker space. We also plan to apply A to enhance adversarial training for Natural Language Processing.
Acknowledgments and Disclosure of Funding
This work was supported by the National Natural Science Foundation of China (#62102177 and #U181 1461), the Natural Science Foundation of Jiangsu Province (#BK20210181), the Key Research and Development Program of Jiangsu Province (#BE2021 1729), Ant Group through Ant Research Program, and the Collaborative Innovation Center of Novel Software Technology and Industrialization, Jiangsu, China.
References
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference computer vision and pattern recognition, 2016.
Jacob Devlin Ming-Wes Chang Kenton and Lee Kristina Toutanova. Bert: Pre-training of deep
bidirectional transformers for language understanding. In Proceedings of NAACI-HLT, 2019.
Huifeng Guo, Ruiming Tang, Yumming Ye, Zhenguo Li, and Xiugiang He. Deepfmm a factorization- machine based neural network for etr prediction. In Proc. of IJCAI, 2017.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In BMVC, 2016.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In Proc. of ICML., 2019.
Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improving adversarial robustness requires revisiting misclassified examples. In Proc. of ICLR, 2019.
Dongxian Wu, Shu-Tao Xia, and Yisen Wang. Adversarial weight perturbation helps robust general- ization. Advances in Neural Information Processing Systems, 2020.
Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, Percy Liang, and John Duchi. Unlabeled data improves adversarial robustness. In Proc. of ICONIP, 2019.
Sven Gowal, Chongli Qin, Jonathan Uesato, Timothy Mann, and Pushmeet Kohli. Uncovering the limits of adversarial training against norm-bounded adversarial examples. arXiv preprint arXiv:2010.03593, 2020.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. Proc. of ICLR, 2018.
Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In Proc. of ICLR, 2015.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International conference on machine learning, 2020.
Chengyuan Yao, Pavol Bielik, Petar Tsankov, and Martin Vechev. Automated discovery of adaptive attacks on adversarial defenses. arXiv preprint arXiv:2 1860, 2021.
Eric Wong, Leslie Rice, and Zico Kolter. Fast better than free: Revisiting adversarial training. In Proc. of ICLR, 2019.
Barret Zoph and Quoe V. Le. Neural architecture search with reinforcement learning. In Proc. of ICLR, 2017.
Hanxiao Liu, Karen Simonyan, and Yiming Yang. Darts: Differentiable architecture search. In Proc. of ICLR, 2018.
10

---

Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmas 2017.
Takeru Miyato, Andrew M Dai, and lan Goodfellow. Adversarial training methods for semi-supervised text classification. In Proc. of ICLR, 2017.
Jingfeng Zhang, Xilie Xu, Bo Han, Gang Niu, Lizhen Cui, Masashi Sugiyama, and Mohan Kankan- halli. Attacks which do not kill training make adversarial learning stronger. In ICML, 2020.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, 2017.
Diederik P Kingma and Max Welling. arXiv:1312.6114, 2013.
Auto-encoding variational bayes.
arXiv preprint
Emil Julius Gumbel. Statistical theory of extreme values and some pructical applications: a series of lectures. US Government Printing Office, 1954.
Chris J Maddison, Daniel Tarlow, and Tom Minka. A sampling. Advances in Neural Information Processing Systems, 2014.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. Proc. of ICML, 2017.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Proc. of ICLR, 2015.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp), 2017.
James Bergstra, Rémi Bardenet, Yoshua Bengio, and Balázs Kégl. Algorithms for hyper- parammee optimization. Advances in neural information processing xystems, 2011.
Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoe V Le. Autoaugment: Learning augmentation strategies from data. In Proc. of CVPR, 2019.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [No]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either the supplemental material or URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes]
(c) Did you report error bars (e.g., with respect to the random seed after running exper- iments multiple times)? [Yes] The standard deviations are omitted as they are very small
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, cloud provider)? [Yes] See Appendix.

---

4. If you are using existing assets (e.g., code, data, models) or curating/releasnng new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [Yes]
(c) Did you include any new assets either in the supplemental material or as URL? [N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
12


Reviews:
[{"review_id": "xyZSjb7QBcp", "reviewer": "Reviewer_rb9H", "summary": "This paper presents a more efficient way for adversarial training – deciding attack algorithm parameter based on the idea of AutoML. In this framework, attack steps are split into step-wise cells, and the proposed method aims to predict the best parameter for each cell using attention mechanism. By fixing the parameters of each cell, the proposed framework degenrates into existing adversarial training method. The proposed method has been evaluated on three commonly used datasets: SVHN, CIFAR-10, CIFAR-100. Comprehensive experimental results demonstrate the effectiveness of the proposed method.", "questions": "My questions are listed in the weaknesses above. Generally this is a good paper and I did not find many questions.", "limitations": "By searching the keyword \"limitation\" I found no result in the submission, but I consider this a minor issue.\nThis paper focuses on defense side, and hence negative impact is not of concern.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "# Strengths\n\n1. Traditional defense methods leverages multi-step PGD for creating adversarial examples to adversarially train a model. Such method is promising but poses very significant computational overhead. I've also gone through the pain of slow adversarial training. Hence, the motivation to create adversarial examples more effectively for training is valid and clear.\n\n2. The proposed method is novel – introducing ideas of automl into the manual parameter tuning of attack algorithms for better results. Meanwhile, the method is clearly described and is not difficult to understand.\n\n3. The proposed method is effective and clearly improves robust accuracy. Nowdays it's very hard to improve adversarial robustness on these standard benchmarks. Compared to the recent related works, the robustness improvement introduced by this method (such as Table 2) is already non-trivial.\n\n4. The proposed method is compatible with other state-of-the-art defense methods like AT, TRADES, and AWP.\n\n# Weaknesses\n\n1. [important] Is the resulting adversarial perturbation clipped into the perturbation bound epsilon? The perturbation bound epsilont is reuiqred by Algorithm 1, but not explicitly used. Please clarify this and make it less confusing.\n\n2. [minor suggestion.] It would be better to investigate wether the proposed method is still effective on larger scale datasets like ImageNet.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "x966UKTEVRE", "reviewer": "Reviewer_SX8F", "summary": "- The authors propose a new approach for the adversarial training of deep neural networks using their $A^2$ automated attacker. By using an efficient and stronger attack algorithm, the adversarial perturbations can result in more robust models trained using adversarial training. Their inner attack algorithm is inspired by AutoML and can select the attack parameters that generate the worst-case perturbations. The authors’ attack incurs a small increase in runtime compared to PGD.", "questions": "- Are the results from tables 2, 3, and 4 also run 5 times and averaged as well? Can we be certain that the $A^2$ approach shows a consistent improvement over normal adversarial training (and were not cherry-picked results)?\n- Did the authors investigate the efficacy of their $A^2$ adversarial training approach against transferable black box attacks?\n- I’m curious how frequently each attacker parameter is selected. Is there a combination of attack types or step sizes which are clearly selected a vast majority of the time? If so, does $A^2$ provide a considerable improvement over just using this combination of attack parameters? Or are there classes or datasets that are more vulnerable to certain attack combinations?\n", "limitations": "- The main limitations of this work can be addressed with the following suggestions:\n\t- I would suggest the authors log the distribution of selected attack parameters for each dataset and class.\n\t- The authors should ensure that the the results presented in tables 2-5 are reproducible. Though, it is understandable these experiments were run only once, given the expensive runtime of adversarial training.\n\t- Emphasizing or explaining the significance of the relatively small performance improvements afforded by $A^2$\n", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\n- A new attack type that can be used to improve the robustness of models defended using adversarial training. The attack increases runtime by 5-7%, a rather acceptable trade-off.\n- The results and experiments show a consistent improvement over previous work in both attacking models and using $A^2$ for adversarial training.\n- Included analyses and explanations of certain results (e.g., natural vs. adversarial accuracy trade-off).\n\nWeaknesses\n\n- Most results only indicate small improvements of less than 2 percentage points.\n- It is unclear whether results from tables 2, 3, and 4 are also run and averaged 5 times.\n- Various minor typos and grammar issues: is developed -> was developed, PDG -> PGD, such linearization attack -> attacks tend, becompatible -> be compatible, etc. I would recommend the authors perform a thorough proofreading check.\n- No investigation of whether different datasets, classes, or samples actually require a significant amount of fine-tuning to find optimal attack parameters.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "83pv1_ycmAZ", "reviewer": "Reviewer_WxDd", "summary": "This paper proposes a novel attack method that efficiently generates strong adversarial perturbations. Specifically, based on the idea of AutoML, the authors design an automated attacker $A^2$ that finds the best perturbation for each iteration. The main idea is to use an attention mechanism to score possible attacks in the attacker space, then sample the attack to perform based on the assigned scores. The authors challenge the problem of training attention mechanism via reparameterization trick. By training both the model parameter and the automated attacker, the authors propose an improved adversarial training method using $A^2$.\n\nFrom a set of experiments, the paper answers three questions about $A^2$: the power of the attack method, its effect in adversarial training, and the robustness of $A^2$ training. The experimental results demonstrate that $A^2$ can increase the attack power and improve the adversarial training performance without too much overhead.\n", "questions": "1. This paper specifically uses $\\nabla_{\\mathbf{x}^{(k-1)}} l(f_\\theta(\\mathbf{x}^{(k-1)}, y)$ to produce a query. Is it a necessary choice, or is your design flexible enough to take other values as input? To provide more context, I’m considering a gradient-free version of $A^2$ and expect the same design to work with a different attacker space and a different type of query.", "limitations": "The authors discussed limitations and potential negative societal impact in the paper appropriately.", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Originality: To the best of my knowledge, the paper contains novel ideas.\n\nQuality: \n\n[[Strength]]\n1. The construction looks sound, and the reparametrization trick seems reasonable to handle similar situations in model training.\n2. The authors used various adversarial training methods, datasets, and attack methods for the adversarial training experiments, showing the proposed method's effectiveness.\n\n[[Weakness]]\n1. The authors can improve the experiment on the attack effectiveness by more comparisons to other attacks (other than PGD) such as CW and AutoAttack.\n\nClarity: The paper writing is clear, and there is no issue with the clarity of the paper.\n\nSignificance: \n\n[[Strength]]\n1. The result on the attack effectiveness shows that we can save the number of iterations by adapting $A^2$ in adversarial example generation.\n2. The experiments contain a practical overhead analysis for people adapting $A^2$ in practice. Considering the additional power of the attack, I believe the overhead is reasonably small.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept