A 2 : Efficient Automated Attacker for Boosting Adversarial Training
Abstract
Based on the significant improvement of model robustness by AT (Adversarial
Training), various variants have been proposed to further boost the performance.
Well-recognized methods have focused on different components of AT (e.g., designing loss functions and leveraging additional unlabeled data).
It is generally accepted that stronger perturbations yield more robust models.
However, how to generate stronger perturbations efficiently is still missed.
In this paper, we propose an efficient automated attacker called A 2 to boost AT by generating the optimal perturbations on-the-fly during training.
A 2 is a parameterized automated attacker to search in the attacker space for the best attacker against the defense model and examples.
Extensive experiments across different datasets demonstrate that Agenerates stronger perturbations with low extra cost and reliably improves the robustness of various AT methods against different attacks.
Introduction
DNNs (Deep Neural Networks) are extremely vulnerable to imperceptible perturbations despite their success in a wide variety of applications [He et al., 2016, Kenton and Toutanova, 2019, Guo et al., 2017].
In particular, adding small but carefully chosen deviations to the input, called adversarial perturbations, can cause DNNs to make incorrect predictions with high confidence.
It indicates that models trained by minimizing the empirical risk are not intrinsically robust.
To explicitly improve robustness, AT (Adversarial Training), where a defense model is trained on worst-case adversarial perturbations generated by an attacker, was developed and proved to be effective.
Based on the significant improvement of models’ robustness, various methods have been proposed, which focus on different components of AT: analyzing the robustness of neural architectures
Zagoruyko and Komodakis [2016], designing loss functions such as TRADES [Zhang et al., 2019] and MART [Wang et al., 2019], perturbing the model to regularize the loss landscape’s flatness (i.e.,
AWP Wu et al.
and leveraging unlabeled data (i.e., RST [Carmon et al., 2019]).
Gowal et al. [2020] compares the performance of each combination of most components and achieves the SOTA (state-of-the-art) performance.
All the above AT methods use PGD 𝐾
[Madry et al., 2018], which is a 𝐾 -step stack of FGSM [Goodfellow et al., 2015], as the attacker to generate perturbations for each example against the defense model.
As the key of AT, stronger perturbations yield more robust models.
However, there is a trade-off between the strength of the perturbation and the training efficiency.
Increasing the attack step 𝐾 strengthens the perturbations, but linearly increases the training overhead [Gowal et al., 2020].
Likewise, the huge overhead prevents the use of SOTA adversarial attackers [Croce and Hein, 2020,
Yao et al., 2021].
To achieve a balance of robustness and efficiency, manually tuning the attacker (e.g., step size and attack method in each step) is of great concern.
R+FGSM
[Wong et al., 2019] first randomly initializes a small perturbation, and then applies FGSM with the tuned step size.
Surprisingly, AT with R+FGSM is as effective as PGD 𝐾 but has a significantly lower cost.
However, given the novel dataset, tuning the attacker manually is a challenging task requiring expert knowledge.
Moreover, the best attacker is fine-grained to each example and current model during adversarial training.
Manual coarse-grained tuning for the whole training (e.g., fixed attack method and step size for all examples) is sub-optimal and prevents further improvement of robustness.
Inspired by AutoML (Automated Machine Learning [Zoph and Le, 2017, Liu et al., 2018]), we propose an efficient automated attacker called A 2 to boost AT.
A 2 is a parameterized attacker, which can automatically tune itself on-the-fly during training to generate worst-case perturbations for each example.
First, we design a general attacker space by referring to existing attackers.
The attacker space is stacked by one-step attacker cells.
Each cell consists of a perturbation block and a step size block.
Then, we employ a parameterized attacker to search for operations in each block and construct the attacker for each example that maximizes the model loss.
Specifically, we leverage the attention mechanism to calculate the score of each operation.
For continuous operations, we sum up the operations using the normalized scores as weights.
For discrete operations, we use the reparameterization trick
[Jang et al., 2017] to sample an operation from the corresponding block.
In this way, the constructed attacker generates worst-case adversarial perturbations to train the defense model.
Meanwhile, A 2 is differentiable and can be optimized with respect to the model loss by gradient descent.
We conduct extensive experiments to verify the effectiveness and efficiency of A 2 .
Compared with PGD, A 2 can find better perturbations for different models trained with various AT methods.
The results demonstrate that 20-step A 2 generates better perturbations than PGD 100 .
Moreover, we combine A 2 with other AT variants, and the robustness of models with different architectures on various datasets is generally improved under strong attacks (e.g., classical C&W and SOTA
AutoAttack).
We also show that A 2 is insensitive to its hyperparameters.
To summarize, our main contributions can be highlighted as follows:
• We propose an efficient automated attacker called A 2 , which can generate worst-case perturbations on-the-fly during training to improve robustness.
• In A 2 , we design an attacker space by summarizing the existing attackers and employ a differentiable method to construct the most adversarial attacker for each example according to the attention mechanism.
• Extensive experimental results across different datasets and neural architectures demonstrate that A 2 improves the model’s robustness by generating stronger perturbations in the inner maximization .
Moreover, A 2 can be flexibly combined with different AT methods, showing good generality.
Preliminary: Adversarial Training
Let 𝐷 =
( X ,𝑌 )
= {( x 𝑖 , 𝑦 𝑖 )}
𝑛𝑖 = 1 be a dataset with x 𝑖 ∈ R 𝑑 as a natural example and 𝑦 𝑖 ∈ { 1 , . . .
, 𝐶 } as its associated label.
We measure the performance of a DNN classifier 𝑓 parametrized with 𝜃 using a suitable loss function 𝑙 , denoted as E ( x 𝑖 ,𝑦 𝑖 )∈ 𝐷 [ 𝑙 ( 𝑓 𝜃 ( x 𝑖 ) , 𝑦 𝑖 )] .
AT [Madry et al., 2018] formulates a saddle point problem whose goal is to find the model parameters 𝜃 that minimize the adversarial risk in the outer minimization (the example’s index 𝑖 is omitted for brevity): min [Equation]
E ( [Equation]
𝐷 inner maximization (cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)
[Table]
outer minimization (1) where S defines the set of allowed perturbations.
The perturbation is usually constrained by 𝐿 𝑝 norm with a bound 𝜖 , i.e. S = { 𝛿 |∥ 𝛿 ∥ 𝑝 ≤ 𝜖 } .
The inner maximization aims to find an adversarial perturbation against the example that achieves a high loss for the defense model.
However, it is NP-hard to find the optimum of the inner maximization .
Various gradient-based attackers have been proposed to approximate its solution, and we classify them according to the number of steps in gradient ascent.
One-step attackers [Goodfellow et al., 2015,
Miyato et al., 2017] generate adversarial perturbations as:
[Equation]
) where ∇ x is short for ∇ x 𝑙 ( 𝑓 𝜃 ( x ) , 𝑦 ) , 𝜂 is the step size, 𝜓 is a transformation function (e.g., 𝑠𝑔𝑛 in
FGSM and 𝐼𝑑𝑒𝑛𝑡𝑖𝑡𝑦 in FGM) and Π is the projection.
Since such linearization attacks tend to be trapped in the non-smooth vicinity of the data point, R+FGSM initializes a small random perturbation to escape the vicinity and then applies FGSM.
As a typical multi-step attacker, PGD 𝐾 [Madry et al., 2018] can find better perturbations by 𝐾 step gradient ascent:
[Equation]
Methodology 3.1 Motivation
The key of AT is to generate perturbations in the inner maximization .
Strong perturbation helps to improve robustness.
It is generally accepted that the step 𝐾 used to solve the inner maximization correlates with the attacker’s ability to generate stronger perturbation.
However, larger 𝐾 leads to a linear increase in training overhead.
Wong et al.
[2019] suggests that with appropriate step size tuning and early stopping, one-step attackers yield models with the robustness that is comparable to much more expensive multi-step attackers.
It indicates that hyperparameters, such as random initialization, step size, momentum, and early stopping, affect perturbation generation.
From the perspective of effectiveness and efficiency, it is valuable to further improve robustness by tuning the attacker to strengthen perturbations.
However, manual tuning of perturbation generation for each example on-the-fly during training is impractical.
To address this problem, we propose an efficient automated attacker to boost adversarial training by generating optimal perturbations on-the-fly during training.
3.2 Problem Formulation
Inspired by AutoML, we first design a general attacker space A by referring to existing attackers.
Then, we employ an automated attacker parameterized by 𝛼 to search in A and further construct an attacker against the example and the defense model ( x , 𝑦, 𝑓 𝜃 ) .
We abbreviate the perturbation generated by the constructed attacker as 𝛿 𝛼 .
Therefore, the goal of A 2 is to train a robust model using the perturbation generated by the constructed attackers through a bilevel optimization problem:
[Equation]
On the attack side, we train 𝛼 by SGD to make the defense model misclassify.
On the defense side, we use 𝛼 ∗ to construct the best attacker for each example and then generate perturbations to adversarially train the defense model.
3.3 Attacker Space
Revisiting most attackers, we find that the attacker can be viewed as a stack of one-step attackers consisting of an attack method and a step size.
Thus, as shown in Figure 1(a), we design a general attacker space
A consisting of 𝐾 -step cells.
The 𝑘 -th cell is denoted as C ( 𝑘 ) , which is a one-step attacker consisting of the following two blocks:
Perturbation Block O 𝑝 .
Typical attack methods (i.e., FGM and FGSM ), attack methods with momentum (i.e., FGMM and FGSMM ), random perturbations (i e., Gaussian and Uniform ), and the special Identity which enables the attacker to automatically early stop at a certain step like FAT [Zhang et al., 2020]; [Figure]
in the second cell and Identity in the other cells.
Step Size Block O s .
[Equation] [Equation] · 𝜂,
[Equation]
· 𝜂, [Equation] · 𝜂, 10 − 1 · 𝜂, 𝜂 } , where 𝜂 is a hyperparameter related to the space of allowed perturbations S .
Each block O contains multiple operations.
Let 𝑜 (·) denote the operation, 𝛾 ( 𝑘 ) =
[ 𝛾 ( 𝑘 ) O 𝑝 , 𝛾 ( 𝑘 )
O 𝑠 ] denote the choice of operation in the 𝑘 -th cell.
The attack methods within O ( 𝑘 ) 𝑝 are mutually exclusive.
Thus, 𝛾 ( 𝑘 ) O 𝑝 is a one-hot vector.
In contrast, the operations within O ( 𝑘 ) 𝑠 are continuous.
𝛾 ( 𝑘 )
O 𝑠 is a normalized continuous vector, where each element represents the selection probability.
To unify the categorical choice of attack methods and the probabilities over step sizes, the output of O ( 𝑘 ) is expressed as a mixture based on 𝛾 ( 𝑘 ) O :
[Equation]
where 𝛾 𝑜 denotes the weight of the operation 𝑜 in 𝛾 ( 𝑘 ) O .
Correspondingly, the one-step attacker of the 𝑘 -th cell can be expressed as the joint of two blocks:
[Equation]
Moreover, the constructed attacker is a composition of attackers from each cell.
In this way, we can cover common attackers in our space.
For example, as shown in Figure 1(b), PGD 𝐾 is obtained by selecting FGSM in each perturbation block.
R+FGSM in Figure 1(c) is a case of selecting Gaussian in the first cell, FGSM in the second cell and Identity in the other cells.
Analysis of A Considering there exist 7 attack methods in O 𝑝 of each step, there are 7 𝐾 combinations of attack methods in the 𝐾 -step attacker space.
The exponential increasing combinations prevent the brute-force search.
Moreover, the continuous step size O 𝑠 is also part of the attacker space.
Thus, we propose A 2 to search for the best attacker in A and generate adversarial perturbations efficiently.
3.4 Automated Attacker A
A 2 is used to construct the best attacker against ( x , 𝑦, 𝑓 𝜃 ) and its trainable parameters 𝛼 include 𝑊 ( 𝑘 ) O 𝑝 , 𝑊 ( 𝑘 )
O 𝑠 , and 𝑊 ( 𝑘 ) 𝑞 where 𝑘 ∈ { 1 , . . .
, 𝐾 } .
As shown in Figure 1(a), we treat the current model and example as a query and the candidate operations as keys.
Thus, the attention mechanism can be used to calculate the scores of operations within each block and the operations are selected based on their scores.
Specifically, in the 𝑘 -th cell, we take the gradient of the last step ∇ x ( 𝑘 − 1 ) 𝑙 (cid:0) 𝑓 𝜃 (cid:0) x ( 𝑘 − 1 ) (cid:1) , 𝑦 (cid:1) as input and project it to a vector space as the query using 𝑊 ( 𝑘 ) 𝑞 .
Then, we use the trainable embedding table 𝑊 ( 𝑘 ) O to convert the individual operations within O ( 𝑘 ) to continuous keys.
With the Scaled
Dot-Product Attention [Vaswani et al., 2017], we compute the dot products of the query with each key as the score of the operation 𝑜 ∈ O in the 𝑘 -th cell:
[Equation]
Perturbation Block.
The operations within O ( 𝑘 ) p are mutually exclusive.
We sample an operation with the normalized scores as probabilities, i.e., 𝛾 ( 𝑘 ) O 𝑝 ∼ softmax ( 𝑒 ( 𝑘 ) O 𝑝 ) .
Step Size Block.
As the operations within O ( 𝑘 ) s are continuous values, we sum up the individual step sizes with the normalized scores as weights.
For 𝑜 𝑠 ∈ O ( 𝑘 ) 𝑠 , the weight can be expressed as:
[Equation]
3.5 Training of Automated Attacker
As mentioned above, we train A 2 to minimize the following objective by gradient descent:
[Equation]
However, as a result of constructing the attacker by sampling in each perturbation block, the gradient of the loss w.r.t 𝛾 O 𝑝 is zero.
To train 𝛼 , we use the reparameterization trick [Kingma and Welling, 2013] to transfer the randomness of sampling to the auxiliary noise and reformulate the objective function.
For brevity, we omit the step index 𝑘 .
Let 𝛾 O 𝑝 = 𝜙
( 𝜅, 𝑒 O 𝑝 ) be a differentiable transformation where 𝜅 is an auxiliary noise variable with independent marginal 𝑝 ( 𝜅 ) .
In A 2 , we sample noise from Gumbel Distribution, i.e., 𝜅 ∼ Gumbel ( 0 )
[Gumbel, 1954], and use Gumbel Softmax [Jang et al., 2017] as 𝜙 to smoothly approximate the expectation of loss [Maddison et al., 2014].
Specifically, 𝜙 ( 𝜅, 𝑒 O 𝑝 ) = softmax (cid:16) (
𝑒 O 𝑝 + 𝜅 )/ 𝜏 (cid:17) where 𝜏 is the temperature parameter.
When 𝜏 → 0 , the generated samples have the same distribution as one_hot ( arg max 𝑜 𝑝 ∈O 𝑝 ( 𝑒 𝑜 𝑝 + 𝜅 𝑜 𝑝 )) .
Using the reparameterization trick, we can now form MC (Monte Carlo) estimates of the expectation of A 2 ’s loss 𝑙 for each example, which is differentiable, as follows:
[Equation]
where 𝑀 is the number of samples.
In practice, 𝑀 = 1 can achieve good performance.
In this way, we reformulate the MC approximation in Equation (10) of 𝑙 as the objective function ˆ 𝑙 .
Moreover, training 𝛼 to convergence in each epoch can be prohibitive due to the expensive inner maximization in Equation (4).
We use a simple approximation scheme following the common methods [Finn et al., 2017, Liu et al., 2018]:
[Equation]
(11) where 𝛼 denotes the current weights of the attacker and 𝜉 is the learning rate.
3.6 Framework of Adversarial Training with A
The overall procedure is shown in Algorithm 1.
As in normal adversarial training, we generate perturbations in 𝐾 steps every batch and update the model parameters.
The key difference is in Line 7.
Benefiting from a parameterized automated attacker, we tune the discrete attack methods and continuous step sizes to generate adversarial perturbations.
After optimizing the model parameters, we use Equation (11) to update 𝛼 as an approximation to 𝛼 ∗ .
Since A 2 focus on the inner maximization , it can be compatible with most adversarial training methods.
For example, it is flexible to use the loss function of TRADES or MART for outer minimization in Line 10 (i.e., TRADES-A 2 and MART-A 2 ), or include early stopping in Line 5 ∼ 9 as FAT.
Input : Training examples 𝐷 , perturbation bound 𝜖 , the number of attack steps 𝐾 1: Initialize 𝜃, 𝛼 ; 2: for epoch = 1 , . . .
, 𝑁 𝑒𝑝 do 3: for minibatch ( X ,𝑌 ) ⊂
𝐷 do 4:
X ( 0 ) ← X ; 5: for k = 1 , . .
, 𝐾 do 6:
Calculate the gradient ∇ X ( 𝑘 − 1 ) ; 7:
Construct 𝛿 ( 𝑘 ) 𝛼 ∈ S according to ∇ X ( 𝑘 − 1 ) by 𝑔 𝛼 ; 8:
X ( 𝑘 ) =
X ( 𝑘 − 1 ) + 𝛿 ( 𝑘 ) 𝛼 ; 9: end for 10:
Update 𝜃 with ∇ 𝜃 (cid:205)
( x ,𝑦 ) 𝑙 (cid:0) 𝑓 𝜃 ( x ( 𝐾 ) ) , 𝑦 (cid:1) ; 11:
Update 𝛼 by Equation (11); 12: end for 13: end for
Experiments
We conduct extensive experiments on public datasets to answer the following questions: 1) Can Agenerate stronger adversarial perturbations?
2) How effective is the adversarial training with A 2 ? 3) Is A 2 robust to hyperparameters?
All experiments are run using GeForce RTX 3090 (GPU) and
Intel(R)
Xeon(R)
Silver 4210 (CPU) instances.
4.1 Effectiveness of Automated Attacker (RQ1)
In this part, we fix the model 𝑓 𝜃 , train the automated attacker alone and investigate whether A 2 can generate more powerful perturbations compared to the commonly used PGD. 图例
[Table]
(a)
[Table]
18 trained by MART
[Table]
[Figure]
A tt ac k E ff ec t
[Figure]
Experimental Settings.
To demonstrate the generality, we choose different neural architectures (i.e., ResNet-18, WRN-34-10, and WRN-28-10) trained on CIFAR-10 by various AT methods:
TRADES, MART, RST, and AWP.
All trained models are open-source checkpoints.
We choose
PGD 𝐾 with a random start 𝛿 ( 0 ) ∼ 𝑈𝑛𝑖 𝑓 𝑜𝑟𝑚 (− 𝜖, 𝜖 ) as baseline.
All attacks are 𝐿 ∞ -bounded with a total perturbation scale of 𝜖 = 8 / 255 .
Since more attack steps generally improve the attack effect, we compare the automated attacker with PGD at different steps.
Moreover, we try different step
[Table]
Attack Effect.
The training process of A 2 is shown in Figure 2.
We take the first 20 epochs of the attack effects, compare them with PGD, and observe whether A 2 converges.
In the early training stage, the random combinations of attack operations are much less effective.
After 10 ∼ 20 epochs, the effect of generated attacks is much more stable and effective.
In practical automated adversarial training, the model and the automated attacker are trained iteratively.
The fast convergence of Aensures that the generated perturbations are strong enough.
In addition, a larger 𝜂 achieves better attacks.
As the steps increase, the effect diminishes and the training may fluctuate.
Table 1 reports the attack effects of PGD and A 2 with different steps.
In the comparison of A 2 and
PGD with different steps 𝐾 ∈ { 10 , 20 } , A 2 stably outperforms PGD.
A 2 † constructs stronger attacks in the expanded search space.
With the increase of steps, A 2 is more effective due to the combination of attack methods and the automated step size tuning.
Due to the diminishing marginal effect, PGDv.s.
PGD 20 achieves less improvement than PGD 20 v.s. PGD 10 .
At 1 / 5 of the cost, the 20-step A 2 † finds better attacks using the optimized 𝛼 than PGD 100 .
In summary, Table 1 verifies that A 2 stably outperforms PGD for the same step, and obtains better attacks compared to PGD, whose step size and attack method are fixed, with significantly lower cost.
Overhead Analysis.
The overhead of A 2 is not significant compared to PGD.
Both methods are close in terms of clock time.
For WRN-34, PGD takes 19.75/147.09/287.76 seconds to generate 1/10/20 step attacks respectively.
It demonstrates that more inner steps lead to a linear increase in time.
Meanwhile, A 2 takes 157.61/302.51 seconds to generate the 10/20 step attack respectively.
The main overhead remains in the forward computation and backward propagation of the defense model.
Moreover, Section A.3 in Appendix shows that the total parameter size of A 2 is also acceptable.
4.2 Effectiveness of Adversarial Training with A 2 (RQ2)
In this part, we evaluate the robustness of our proposed AT-A 2 on different datasets against white-box and ensemble attacks.
To verify that the stronger attacks generated by A 2 on-the-fly during training can improve robustness, we consider various adversarial training methods (i.e., AT, TRADES, MART, and AWP) without additional data across different datasets.
Benchmark.
We conduct experiments on the baseline AT and the SOTA AWP with A 2 across three benchmark datasets to verify the generalization of A 2 .
We follow the settings in AWP: PreActResNet18 trained for 200 epochs, 𝜖 = 8 / 255 and 𝛾 = 10 − 2 for AWP.
The step size is 1 / 255 for SVHN and 2 / 255 for CIFAR-10 and CIFAR-100.
For AT and AWP, the attacker used in training is PGD 10 .
The 10-step A 2 is trained with the same setting as in RQ1.
PGD 20 is used for testing, and the test robustness is reported in Table 2.
It shows that A 2 , as a component focusing on the inner maximization , [Table] [Table] achieves better results on most datasets.
Moreover, A 2 is generic and can boost the robustness of both baseline and SOTA AT methods.
Robustness on WideResNet.
Furthermore, we train WRN-34-10 on CIFAR-10 with various AT methods (i.e., AT, TRADES, MART, and AWP) following their original papers and open-source codes 2 .
All defense models are trained using SGD with momentum 0.9, weight decay 5 × 10 − 4 , and an initial learning rate of 0.1 that is divided by 10 at the 50% -th and 75% -th epoch.
Except forepochs in AWP, other AT methods train the model for 120 epochs.
Simple data augmentations (i.e., 32x32 random crop with 4-pixel padding and random horizontal flip) are applied.
For white-box attack, we test FGSM, PGD 20 and CW ∞
[Carlini and Wagner, 2017].
In addition, we test the robustness against the standard AutoAttack [Croce and Hein, 2020], which is a strong and reliable attacker to verify the robustness via an ensemble of diverse parameter-free attacks including three white-box attackers and a black-box attacker.
Table 3 shows that A 2 reliably boosts
AT variants against white-box and ensemble attacks.
This verifies that A 2 is general for AT and improves adversarial robustness reliably rather than gradient obfuscation or masking.
Additionally, given the nature examples, AT performs better than AT-A 2 .
The main reason is that
A 2 generates stronger perturbation for better robustness, which decreases the accuracy (i.e., 84.54).
Many works (e.g., TRADES, MART, and AWP) use regularization to achieve the trade-off between robustness and accuracy.
The regularization is also used to optimize the automated attacker.
Thus, for other AT methods in Table 3, combining A 2 can achieve higher accuracy.
Moreover, for WRN-34, the training time of AWP-A 2 is 970 s/epoch while the training time of AWP is 920 s/epoch.
Thus, the additional overhead of A 2 is not significant.
4.3 Hyperparameters of A 2 (RQ3)
The hyperparameters of A 2 include the training hyperparameters and the design of the attacker space.
The comparison of the attack effect with different hyperparameters is shown in Table 4.
Overall, Ais robust to hyperparameters and performs better than PGD 10 and closely to PGD 20 .
[Table]
Training of A 2 .
The effect of attacks with different learning rates 𝜉 is shown in the middle two rows of Table 4.
Although Adam uses a dynamic learning rate, an excessive initial learning rate (i.e., 10 − 2 ) leads to sub-optimal.
Attacker Space.
The influence of the attack step 𝐾 has been investigated in RQ1.
As shown in the last two rows of Table 4, a larger step size 𝜂 increases the effectiveness of A 2 .
However, as shown in the training curve in Figure 2, larger 𝜂 introduces instability in the training of the attacker.
Related Work 5.1 Adversarial Learning
Many recent works [Goodfellow et al., 2015, Carlini and Wagner, 2017, Croce and Hein, 2020] have shown that DNNs are vulnerable to adversarial examples.
Various defense strategies and models have been proposed to deal with the threat of adversarial examples.
However, as proved in C&W [Carlini and Wagner, 2017], many works mistake gradient obfuscation or masking for adversarial robustness.
AT [Madry et al., 2018] formulates a class of adversarial training methods for solving a saddle point problem (i.e., Equation (1)) and improves robustness reliably.
Based on AT, many works [Zhang et al., 2019, Wang et al., 2019, Wu et al., 2020] focusing on the components of outer minimization are introduced to further enhance performance.
The inner maximization is also the goal of the adversarial attack, where 𝑙 is the 0-1 loss.
Many works, e.g.,
FGSM [Goodfellow et al., 2015], C&W [Carlini and Wagner, 2017] and AutoAttack [Croce and Hein, 2020], have been proposed to attack DNNs and facilitate the development of adversarial training.
5.2 Automated Machine Learning
AutoML [Bergstra et al., 2011, Zoph and Le, 2017, Liu et al., 2018, Cubuk et al., 2019] aims to automate the parts of the machine learning pipeline that require expert solutions.
For a particular domain, it is common practice to summarize a large search space of parameters and configurations based on expert experience and search for the optimal solutions using methods such as black-box optimization.
During the search process, a certain metric is required to evaluate each solution.
The same idea can be applied to adversarial learning.
𝐴
[Yao et al., 2021], which is also closely related to AutoML, automatically discovers an effective attacker on a given model.
Conclusion
In this work, we proposed A 2 , to the best of our knowledge, the first adversarial training method which focuses on automated perturbation generation.
In A 2 , the attacker space is designed by summarizing the existing perturbations.
Moreover, the parameterized automated attacker leverages the attention mechanism to choose the discrete attack method and the continuous step size and further generates adversarial perturbations.
During training, the one-step approximation of the optimal automated attacker is used to generate the optimal perturbations on-the-fly for the model.
The experimental results show that A 2 generates stronger attacks with low extra cost and boosts the robustness of various AT methods reliably.
For future work, we plan to add the target loss of the inner maximization to the attacker space.
We also plan to apply A 2 to enhance adversarial training for Natural Language Processing.
Acknowledgments and Disclosure of Funding
This work was supported by the National Natural Science Foundation of China (#62102177 and #U1811461), the Natural Science Foundation of Jiangsu Province (#BK20210181), the Key Research and Development Program of Jiangsu Province (#BE2021729), Ant Group through Ant Research
Program and the Collaborative Innovation Center of Novel Software Technology and Industrialization,
Jiangsu, China.
Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes] (c) Did you discuss any potential negative societal impacts of your work?
[No] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[N/A] (b) Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] The standard deviations are omitted as they are very small (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] See Appendix. 4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes] (b) Did you mention the license of the assets?
[Yes] (c) Did you include any new assets either in the supplemental material or as a URL?
[N/A] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
[N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
[N/A] 5.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]
A Details in AA.1 Unify Magnitude of Perturbations
Perturbations generated by different operations in O 𝑝 have different magnitudes and thus require different magnitudes of step size for different 𝑜 𝑝 .
For example, FGSM generates perturbations with elements belonging to {− 1 , 0 , 1 } , while the perturbation generated by FGM is usually in the magnitude of 10 − 3 .
Obviously, they cannot use the same step size.
To have a uniform effect of the step size block, we normalize the magnitude of other generated perturbations to be the same as FGSM (i.e., 𝛿 𝑜 𝑝 = 𝛿 𝑜 𝑝 · ∥ 𝛿 FGSM ∥
∥ 𝛿 𝑜𝑝 ∥ ).
In this way, we find that other attack methods such as FGM can achieve good results with the same step size as FGSM .
A.2 Temperature Parameter in Softmax
Since there is an order of magnitude difference in step size operations, the larger step size with the same score will dominate the output.
For example, 0 . 7 · 10 − 2 𝜂 + 0 . 3 · 𝜂 ≈ 0 .
3 𝜂 .
The output of the step size block is dominated by the operation 𝜂 , despite the greater weight of 10 − 2 · 𝜂 .
To alleviate the problem, we use the temperature parameter 𝜏 in softmax to sharpen the distribution:
[Equation]
(12) where 𝑜 s is an operation in O s , and 𝑒 𝑜 𝑠 is its attention score.
Through experiments, we set 𝜏 = 0 .
1 to distinguish the preference for the step size in most cases.
A.3 Overhead of A
Let the number of steps be 𝐾 , the number of operations be |O| , the image size be 𝑊 × 𝐻 and the embedding size be 𝐸 .
The number of the attacker’s parameter is O
( 𝐾 · 𝐸 · ( 𝑊 × 𝐻 + |O|)) .
Specifically, the number of parameters for the attacker is 7873280 , which is 17% of the model’s parameters (i.e., 46160474 ).
In each batch, there is only 1 forward calculation of all cells withbackpropagation.
In comparison, the model requires 𝐾 forward calculations with backpropagation.
Therefore, the additional computational overhead from the attacker is not significant in terms of the number of parameters and computations.
Moreover, PGD and A 2 are close in terms of clock time.
For WRN-34, PGD takes 19.75/147.09/287.76 seconds to generate 1/10/20 step attacks respectively.
It demonstrates that more inner steps lead to a linear increase in time.
Meanwhile, A 2 takes 157.61/302.51 seconds to generate the 10/20 step attack respectively.
The main overhead remains in the forward computation and backward propagation of the defense model.
For WRN-34, the training time of AWP-A 2 iss/epoch while the training time of AWP is 920 s/epoch.
In summary, the additional overhead of A 2 is not significant.
A.4 Why No Mixture in O 𝑝
Like most NAS methods in AutoML, the discrete selection in the perturbation block is more interpretable and robust (e.g., L1-Norm for feature selection and single path in NAS) than the mixture over possible solutions.
Moreover, the mixture will incur more computational overhead and 7 times memory overhead due to 7 operations in O 𝑝 .
Figure 3 shows an example of the generated attack on
CIFAR-10, which can be migratable.
[Figure]
generated attack on CIFAR-10.
B Addition Experiments
B.1 Why use FGSM-based PGD in RQ1.
There are multiple single-step attack methods in O 𝑝 for stacking as PGD, e.g., FGM-based PGD and FGSM-based PGD.
The experimental results of the attack effect of PGD based on these attack methods demonstrate that FGSM-based PGD outperforms the stacking of other operations.
Thus, we choose FGSM-based PGD with a random start 𝛿 ( 0 ) ∼ 𝑈𝑛𝑖 𝑓 𝑜𝑟𝑚 (− 𝜖, 𝜖 ) as a baseline for comparison with the automated attacker.
B.2 Number of samples 𝑀 in MC Approximation.
𝑀 is an important hyperparameter that dictates the quality of MC approximation and the training overhead.
We test the cases with 𝑀 ∈ { 1 , 2 , 5 } and achieve similar performance.
Thus, we set 𝑀 toand achieve good results with a significantly lower overhead.
B.3 Generality of A 2 in White-Box Attacks
In this part, we investigate whether A 2 is general to white-box attacks.
As a more powerful attack method, CW ∞ -based attacks [Carlini and Wagner, 2017] stably outperform PGD-based attacks. For
[Table] comparison with CW ∞ , we propose a variant of A 2 that uses CW ∞ loss to generate perturbations and denote it as CW ∞ -A 2 .
The results in Table 5 show that A 2 is general and can improve the attack effect of PGD and CW ∞ by combining attack methods and tuning the step size.
Moreover, the additional overhead of A 2 is 5% to 10%, which is a rather acceptable trade-off.
B.4 Robustness Against Transferable Black-Box Attacks
We investigate the robustness of A 2 against transferable black-box attacks.
Table 6 provides test robustness on CIFAR-10 using ResNet-18.
We adopt three transferable black-box attack methods:
MI (momentum = 1)
[Dong et al., 2018], DI [Xie et al., 2019], and TI [Dong et al., 2019].
The transferable attacks are generated by an ensemble of the above methods on three surrogate pre-trained models 2 : IncV3 (InceptionV3), VGG19, and DN201 (DenseNet201).
Table 6 shows that AT boosts the robustness against transferable black-box attacks, and A 2 can further improve the adversarial robustness. [Table] [Table] [Table]
[Table]
ResNet-18-AT
[Table] [Table] [Table]
ResNet
[Table]
[Table]
[Table]
[Table]
B.5 A Closer Look at Selected Attacks
echarts.svg ﬁle:///Users/rex/Downloads/echarts.svg 1/
P e r ce n t a g e
Epoch
[Figure] [Figure]
ﬁle:///Users/rex/Downloads/echarts (1)
[Figure]
[Figure] [Figure]
[Figure]
[Figure]
[Figure]
[Figure]
[Figure] We analyze the selected attacks from the perspective of perturbation blocks with different steps and datasets.
The first and final perturbation blocks of 10-step A 2 in CIFAR-10 are chosen for analysis.
Figureshows the distribution of selected attacks of different perturbation blocks.
• Perturbation Block 1: A 2 tends to choose FGM , FGSM , and partially random methods as initialization in the first step.
The momentum-based attack methods are quickly discarded as the gradient of the previous step is absent.
FGSM is chosen more frequently due to its stronger attack on both foreground and background.
• Perturbation Block 10: The optimization of the victim model leads to changes in the distribution of selected attacks in the last block.
In the early stage of training, the victim model is vulnerable.
A 2 retains the diversity and plays the role of friendly attackers like FAT [Zhang et al., 2020].
At the end of training, A 2 prefers the momentum-based attacks (i.e., FGSMM and FGMM ).
From the perspective of datasets, SVHN and CIFAR-10 prefer different attack methods.
As shown in
Figure 4(c), SVHN discards FGSMM , which is most frequently used in CIFAR-10, and pays more attention to FGMM .
Moreover, SVHN rarely uses Identity compared with CIFAR-10 as its higher robustness accuracy requires more powerful perturbations.
In summary, A 2 ’s preference for selecting attacks in blocks varies according to the block step, dataset, and victim model.