Model-Agnostic Meta-Attack: Towards Reliable Evaluation of Adversarial Robustness
Authors: Xiao Yang
Yinpeng Dong
Wenzhao Xiang
Tianyu Pang
Hang Su
Jun Zhu
Venue: ICLR 2022 Submitted (2022)
Keywords: Adversarial attacks
robust evaluation

Abstract: The vulnerability of deep neural networks to adversarial examples has motivated an increasing number of defense strategies for promoting model robustness. However, the progress is usually hampered by insufficient robustness evaluations. As the de facto standard to evaluate adversarial robustness, adversarial attacks typically solve an optimization problem of crafting adversarial examples with an iterative process. In this work, we propose a Model-Agnostic Meta-Attack (MAMA) approach to discover stronger attack algorithms automatically. Our method learns the optimizer in adversarial attacks parameterized by a recurrent neural network, which is trained over a class of data samples and defenses to produce effective update directions during adversarial example generation. Furthermore, we develop a model-agnostic training algorithm to improve the generalization ability of the learned optimizer when attacking unseen defenses. Our approach can be flexibly incorporated with various attacks and consistently improves the performance with little extra computational cost. Extensive experiments demonstrate the effectiveness of the learned attacks by MAMA compared to the state-of-the-art attacks on different defenses, leading to a more reliable evaluation of adversarial robustness. 

Under review as a conference paper at ICLR 2022
MODEL-AGNOSTIC META-ATTACK: TOWARDS RELI- ABLE EVALUATION OF ADVERSARIAL ROBUSTNESS
Anonymous authors Paper under double-blind review
ABSTRACT
The vulnerability of deep neural networks to adversarial examples has motivated an increasing number of defense strategies for promoting model robustness. How- ever, the progress is usually hampered by insufficient robustness evaluations. As the de facto standard evaluate adversarial robustness, adversarial attacks typi- cally solve an optimization problem of crafting adversarial examples with an iter- ative process. In this work, we propose a Model-Agnostic Meta- Attack (MAMA) approach to discover stronger attack algorithms automatically. Our method learns the optimizer in adversarial attacks parameterized by a recurrent neural network, which is trained over class of data samples and defenses to produce effective up- date directions during adversarial example generation. Furthermore, we develop model-agnostic training algorithm improve the generalization ability of the learned optimizer when attacking unseen defenses. Our approach can be flexibly incorporated with various attacks and consistently improves the performance with little extra computational cost. Extensive experiments demonstrate the effective- ness of the learned attacks by MAMA compared to the state-of-the-an art attacks on different defenses, leading to more reliable evaluation of adversarial robustness.
INTRODUCTION
Deep neural networks are vulnerable to maliciously crafted adversarial examples (Biggio et al., 2013: Szegedy al., 2014; Goodfellow et al., 2015), which aim induce erroneous model predic- tions by adding small perturbations to the normal inputs. Due to the threats, multitude of defense strategies have been proposed to improve adversarial robustness (Kurakin et al., 2017b; Madry al., 2018; Liao et al., 2018; Wong & Kolter, 2018; Zhang et al., 2019b; Dong et al., 2020a; Pang et al., 2020). However, many defenses have later been shown to be ineffective due to incomplete or incor- rect robustness evaluations (Athalye et al., 201 8a; Uesato al., 2018; Carlini et al., 2019; Croce & Hein, 2020b; Dong et al., 2020b; Tramer et al., 2020), making it particularly challenging to under- stand their effects and identify the actual progress of the field. Therefore, developing methods that can evaluate adversarial robustness accurately and reliably is of profound importance.
Adversarial attacks are broadly adopted as indispensable solution to evaluate adversarial robust- ness for different models. One of the most prominent attacks is the projected gradient descent (PGD) method (Madry et al., 2018), which generates an adversarial example by performing iterative gra- dient updates to maximize a classification loss (e.g.. cross-entropy loss) w.r.t. the input. Although recent methods improve upon PGD by introducing different loss functions (Carlini & Wagner, 2017; Gowal et al., 2019; Croce & Hein, 2020b; Sriramanan et al., 2020) and adjusting the step size (Croce & Hein, 2020b), most of them adopt hand- -esigned optimization algorithms, such as vanilla gradi- ent descent, momentum (Polyak, 1964), and Adam (Kingma & Ba, 2015). However, it has been shown that these attacks can be sub-optimal (Croce & Hein, 2020b; Tramer al., 2020), arous- ing an overestimation of adversarial robustness for some defenses. It is thus imperative to develop more effective optimization algorithms for improving adversarial attacks. Nevertheless, designing generic optimization algorithm in hand-crafted manner is non-trivial, considering the different defense models with varying network architectures, defense strategies, and training datasets.
To overcome this issue and develop stronger adversarial attacks, in this paper, we propose Model- Agnostic Meta-Attack (MAMA) approach to learn optimization algorithms in adversarial attacks automatically. In particular, parameterize the optimizer with recurrent neural network (RNN)

---

Under review as a conference paper at ICLR 2022
to mimic the behavior of iterative attacks, which outputs update direction at each time step during adversarial example generation. By learning to solve a class of optimization problems with different data samples and defense models, the RNN optimizer could benefit from long- term dependencies of iterative attacks and exploit common structures among the optimization problems. To improve and stabilize training, we then propose prior-guided refinement strategy, which imposes prior given by the PGD update rule on the outputs of the RNN optimizer, whose parameters are trained via the maximum a posteriori (MAP) estimation framework. Consequently, the RNN optimizer can learn to yield more effective update directions than dend-des ones.
Despite the effectiveness, the learned optimizer may not generalize well attacking unseen defenses due to potential overfitting to the defenses used for training. Although training a different optimizer for each defense is possible, the training process can be time-consuming and inconvenient in practice when it is utilized to benchmark adversarial robustness. To endow the learned optimizer with a better generalization ability, we develop model-agnostic training algorithm with a gradient-based meta- train and meta-test process simulate the shift from seen to unseen defenses. Therefore, the learned optimizer can directly be deployed to attack unseen defenses without retraining or fine-tuning.
Extensive experiments validate the effectiveness and generalization ability of the learned optimizers for attacking different defense models. We also demonstrate the ficxibility of our method integrating with various attacks, including those with different loss functions (Carlini & Wagner, 2017; Gowal et al., 2019; Croce& & Hein, 2020b) and initialization (Tashiro et al., 2020). Our method consistently improves the attacking performance over baselines, while introducing little extra computational cost. By incorporating our method with an orthogonal technique - output diversified initialization (ODI) (Tashiro et al., 2020), we achieve lower robust test accuracy on all 12 defense models that we study than other state-of-the-art attacks, including the MultiTargeted (MT) attack (Gowal et al., 2019) and AutoAttack (Croce& & Hein, 2020b), leading to a more reliable evaluation of adversarial robustness. Moreover, MAMA provides a plug-and- -pla module in adversarial attacks, which may benefit future adversarial attacks if new improvements have been made in different aspects.
2 PRELIMINARIES AND RELATED WORK
2.1 ADVERSARIAL ATTACKS
Let f(x) : x E RDD + + RK denote a classifier, which outputs the logits # := [21, Zk] over K classes for an input image The prediction of f is Cy(x) arg = K 2; Given the true label y of x, adversarial attacks aim to generate an adversarial example & that is misclassified by (i.e., ( * y). while the distance between the adversarial example & and the natural one measured by the Ex norm is smaller than a threshold c as | - =I|>0 I e. Although we introduce our approach based on the lo norm only, the extension other f, norms is straightforward. The adversarial example a can be generated by solving a constrained optimization problem as
s arg max L(I((') v))) s.t. ||a.' 001100 S,
x²
where L is a loss function on top of the classifier f(x) For example, L could be the cross-entropy loss as ())() - logp, Py where Py = - <>""/ e denotes the predicted probability of class
y. or the margin- CW loss (Carlini & Wagner, 2017) as (f y) = max.
Zy.
A lot of gradient-based methods (Goodfellow et al., 2015; Kurakin et al., 2017a; Carlini & Wagner, 2017: Madry et al., 2018; Dong et al., 2018) have been proposed to solve this optimization problem. The projected gradient descent (PGD) method (Madry et al., 2018) performs iterative updates as
1.+1 = [Ig. (w) (i; + ae sign( V (( v))),
(2)
where (x) = (ax: ||ar' - a<| ao|lo S }) denotes the txx ball centered at x with radius c, IT(-) the projection operation, and a is the step size at the t-th attack iteration. PGD initializes do randomly by uniform sampling within B, (x) and adopts a fixed step size a in every iteration.
To develop stronger attacks for reliable robustness evaluation, recent improvements upon PGD in- clude adopting different loss functions (Gowal et al., 2019; Croce & Hein, 2020b; Sriramanan et al., 2020), adjusting the step size (Croce& & Hein, 2020b), and designing sampling strategies of initializa- tion (Tashiro et al., 2020). For the update rules, besides the vanilla gradient descent adopted in PGD, recent attacks (Dong et al., 2018: 8; Gowal et al., 2019) suggest use the momentum (Polyak, 1964)
2

---

Under review as a conference paper at ICLR 2022
and Adam (Kingma& & Ba, 2015) optimizers. In contrast, we aim to enhance adversarial attacks by learning optimization algorithms in an automatic way.
2.2 ADVERSARIAL DEFENSES
To build robust models against adversarial attacks, numerous defense strategies have been proposed, but most defenses can be evaded by new adaptive attacks (Athalye et al., 201 8a; Tramer et al., 2020). Among the existing defenses, adversarial training (AT) is arguably the most effective defense tech- nique, in which the network is trained on the adversarial examples generated by attacks. Based on the primary AT frameworks like PGD-AT (Madry et al., 2018) and TRADES (Zhang et al., 2019b), improvements have been made via ensemble learning (Tramèr et al., 2018; Pang et al., 2019), met- ric learning (Mao et al., 2019; Pang et al., 2020), ami--uuervised learning (Alayrac et al., 2019; Carmon et al., 2019; Zhai et al., 2019), and self-supervised learning (Chen et al., 2020a; Hendrycks et al., 2019; Kim et al., 2020). Due to the high computational cost of AT, other efforts are devoted to accelerating the training procedure (Shafahi et al., 2019; Wong et al., 2020; Zhang et al., 2019a). Recent works emphasize the training tricks (c.g.. weight decay, batch size, etc.) in AT (Gowal et al., 2020; Pang et al., 2021). However, it has been shown that the robust test accuracy of numerous adversarially trained models can be degraded significantly by using stronger attacks (Croce& & Hein, 2020b; Tramer et al., 2020), indicating that the reliable evaluation of adversarial robustness remains an imperative yet challenging task. In this paper, we focus on evaluating adversarial robustness of AT models due to their superior robustness over other defense techniques.
2.3 META-LEARNING
Meta-learning (learning to learn) studies how to learn learning algorithms automatically from a meta perspective, and has shown promise in few-shot learning (Finn et al., 2017) and learning optimizers (Andrychowicz et al., 2016). The latter is called learning to optimize (I.2O). The first I.20 method leverages a coordinate-wif long short term memory (LSTM) network as the optimizer model, which is trained by gradient descent optimizer can also be trained by reinforcement learning (Li & Malik, 2017). Bello et al. (2017) pro- pose to adopt an RNN controller to generate the update rules of optimization algorithms, which can be described as a domain specific language. L20 is then extended to efficiently optimize black-box functions (Chen et al., 2017). Recent methods focus improving the training stability and general- ization of the learned optimizers by designing better optimizer models and training techniques (Lv et al., 2017; Wichrowska al., 2017; Chen et al., 2020b).
class of optimization problems (Andrychowicz et al., 2016). The
The general idea of meta-learning has been investigated in adversarial attacks. Most related meth- ods learn generate adversarial examples by using convolutional neural networks (CNNs), which usually take clean images as inputs and return adversarial examples/perturbation (Baluja & Fischer, 2017: Poursaeed et al., 2018). Differently, our method adopts an RNN model as the attack optimizer to mimic the behavior of iterative attacks. Empirical results validate the superiority of our approach compared to the CNsed generators. Meta-learning has also been used in black-box attacks (Du et al., 2020) and AT defenses (Xiong & Hsieh, 2020; Jiang et al., 2021). ). It is noteworthy that Xiong & Hsieh (2020) propose similar approach to ours that uses an RNN optimizer to learn attacks for AT. Although the RNN optimizer is also adopted in our approach, we improve the attacking perfor- mance and generalization ability of the RNN optimizer with new techniques, as presented in Sec. 3. More detailed comparisons between our work and Xiong & Hsieh (2020) are shown in Appendix A.
METHODOLOGY
In this paper, we propose Model-Agnostic Meta-Attack (MAMA) approach to improve adver- sarial attacks by learning optimization algorithms automatically. MAMA permits learning to exploit common structures of the optimization problems (1) over a class of input samples and defense mod- els. We first present the instantiation of the learned optimizer with a recurrent neural network (RNN) and the corresponding training procedure in Sec. 1. The learned optimizer is expected to not only perform well on the data samples and defenses for which it is trained but also possess excellent generalization ability to other unseen data samples, defenses, and more optimization steps (i.e., at- tack iterations) than training. Therefore, we propose a model-agnostic training algorithm simulating train/test shift to ensure generalization of the learned optimizer, as detailed in Sec. 3.2. An overview of our proposed MAMA approach is provided in Fig. 1.
3

---

Under review as a conference paper at ICLR 2022
Meta-Train
MyC1s105)) (1) w2C1)-(8)) (") ( fo... (w
%
Updae
#0
Meta-Test
V.did (5.43)
algecithe gradien flow mete grddien flow upábte direction
(a)
(b)
Figure 1: An overview of the MAMA approach. (a) shows the architecture of meta-attack with an RNN optimizer. (b) represents the model-agnostic optimization process. As an example, given defense mod- els f2: fa); meta-train/metta-es could include different divisions as {f1, f23/ f23/ (fa), (f,, {f2, fa)/ The parameters & of the optimizer will be updated by accumulating the gradients from meta- train and meta-gradients from meta-test.
or
3.1 META-ATTACK WITH RECURRENT NEURAL NETWORKS
Different from the existing attack algorithms, we propose a meta-attack that learns an optimizer in adversarial attacks parameterized by RNN model ms with parameters ó to mimic the behavior of iterative attacks. The learned optimizer is capable of optimizing the loss function L in Eq. (1) for different data samples (xx y) and defense models f. At each step, me takes gradient information of the loss function (1) at the current adversarial example as input and outputs an update direction for generating the adversarial example. This procedure be expressed as
#++1 = [Ix.(z)) (#: + a .g.) [gt, h+41] = my (V.,h.)
(3)
where de is the step size, V, V (( y) denotes the gradient of the loss function L at Φt, and h; denotes the hidden state representation of the optimizer, which contains rich temporal gradient information to capture long-term dependencies. The output 9t of the RNN model is used update the adversarial example œ;. We apply a tanh transformation to the last layer of the RNN model to normalize and smoothen the update direction g: since the adversarial perturbations are required to be smaller than the threshold c. In particular, we adopt the coordinate-wist long short term memory (LSTM) network (Andrychowicz et al., 2016) as the RNN model mg. which can significantly reduce trainable parameters, making training much easier and faster.
Given a distribution FF over defenses and data distribution DD over (ar, y) pairs, the parameters can be trained by maximizing the expected loss over the entire trajectory of optimization
T
Jo: F. = B5-FF(
w. C. (.
(4)
where i; is obtained by the optimizer model my through Eq. (3), w. E 12 is the weight for the loss at the 1-th step to emphasize different importance on the loss at different steps, and T is the total number of optimization steps. For example, at the final adversarial example since would be used to attack the classifier, but using a different setting for w, make training easier. For training the optimizer network m.. can maximize the objective (4) using gradient ascent on Øb. The expectation in Eq. (4) is approximated by mini-batches of randomly sampled defense models f and data points (ax, y). The gradient w.r.t. é calculated by truncated backpropagation. Besides the general framework of learning to optimize (Andrychowicz al., 2016; Chen al., 2020b; Xiong Hsieh, 2020), there exist particular challenges in training the optimizer due to the optimization problem of adversarial attacks as illustrated below.
could let w, = 1 7] to focus on the loss
Vanishing gradient problem. Note that the optimization problem (1) is constrained one, which is different from unconstrained problems commonly studied in previous works (Andrychowicz et al., 2016; Chen et al., 2020b; Lv et al., 2017: Wichrowska al., 2017). As Eq. (3), we need project the adversarial example onto the fox ball B, (x)) after every optimization step, which is essentially the clip function. However, optimizing Eq. (4) requires backpropagation through the clip function, which can result in number of zero gradient values when the update exceeds the lx ball. This can consequently lead to the vanishing gradient problem for a long optimization trajectory. To address this problem, we apply the straight-
estimator (Bengio et al., 2013) to get approximate gradient of the clip function as var (=)(x') (x') = I, where denotes the identity matrix. In this way, the optimizer can effectively be trained optimize the constrained problem (1).
4

---

Under review as a conference paper at ICLR 2022
Algorithm 1 Model-Agnostic Meta-Attack (MAMA) algorithm
Require: A data distribution over (ær, y) pairs, set of defense models F ト the number of defenses n the meta-test set, the objective function I in Eq. (5), attack iterations T, batch size B, learning rates 9, 7, balancing hyperparameter, μ.
Ensure: The parameters di of the learned optimizer.
1: for iter = to Maxiterations do
Randomly draw N n defenses from FF to get and JF FVF consist of the other a defenses;
3: for in F do Randomly draw a mini-batch B, = {(x" ) from D; 5: For (x",²"") 2/ E Br, compute ,;" from & for defense fr by Eq. (3) fort to;
Meta-train
6: end for
7: Compute J(;;F, D) using the defenses fr in F and the corresponding mini-batches By;
8:
o' = Fd,;;
9:
for fo in do
b Meta-test
10:
Randomly draw mini-batch B, = ((x* )] from For (ax .g"" ) E Be, compute #;" from ø for defense fe by Eq. (3) for = 1 to T;
12: end for
13: Compute [dd:FF F, D) using the defenses fo in F and the corresponding mini-batcher Ba: F
14: Φ 4 Φ + Vo
15:
end for
Training instability problem. Training the learned optimizer may suffer from an instability prob- lem (Chen et al., 2020b), leading unsatisfactory performance. To further stabilize training of the learned optimizer my and improve its performance, we propose a prior-guided refinement strategy informative prior refine the outputs of the optimizer. In particular, the prior is given by the PGD update rule (Madry et al., 2018) which performs generally well on many defenses. We specify the prior distribution of 9: to an isotropic Gaussian distribution as (sign( Ve), 2 ). whose mean is the PGD update direction sign( (V.). The variance of the distribution 1/2x, with d being a hyperparameter. Under the maximum posteriori (MAP) estimation framework, the ob- jective function (4) can be viewed as the likelihood while the prior is equivalent to minimizing the squared &a distance as Àzllg: sign( I12 We provide formal derivation of MAP in Appendix B. Therefore, the overall objective function of training the optimizer becomes
which leverages
J(d:F,D) = -F-
w. wrC(f(()).)) Aelgg sign (v))()))))
(5)
where {>e)== can be seen as another of weights corresponding to the regularization terms at each step. In this way, the optimizer can learn from the informative prior given by the PGD update rule and explore potential better directions simultaneously. To maximize the objective function (5), a basic technique is using gradient ascent on ø. We call Basic Meta-Attack (BMA).
3.2 MODEL-AGNOSTIC META-ATTACK ALGORITIIM
To make robustness evaluation more convenient after training, the learned optimizer should general- ize well new data samples. defense models, and more optimization steps that are not seen during training. In practice, we observe that the data-driven process of optimizing Eq. (5) (i.e., BMA) en- ables to obtain generalizable optimizer on new data samples with more optimization steps, but the learned optimizer may not necessarily generalize well to new defenses, as shown in Sec. 4.3. This because the defenses usually adopt different network architectures and defense strategies, making it hard for attack optimizer to generalize well across a variety of defenses (Tramer et al., 2020).
To improve the generalization ability of the learned optimizer when attacking unseen defenses, we propose model-agnostic training algorithm for meta-attack (i.e., MAMA). This method simulates the shift between defenses during training with a gradient- based meta-train and meta-test procedure, inspired by model-agnostic meta-learning (Finn al., 2017: Li et al., 201 8). After MAMA training, the learned optimizer can directly be used to attack unseen defenses, which will be more convenient and friindiyy for carrying out robustness evaluation
novel defense methods presented.
Specifically, assume that we have a set of N defenses (denoted as F) in training, where we reuse the notation F without ambiguity. To simulate the shift between defense models in training and testing.
5

---

Under review as a conference paper at ICLR 2022
Table I: Classification accuracy (%) of four defense models CIFAR-10 against various white-box attacks
with different optimizers, including vanilla gradient descent (PGD), momentum (MI-FGSM), nesterov acceler- ated gradient (NI-FGSM), Adam, Adv-CNN, and the learned optimizer (BMA).
Defense
PGD
MI-FGSM
NI-FGSM
Adam
Adv-CNN
BMA (Ours)
HYDRA
58.73
59.27
59.61
58.69
59.42
58.57
Pre-training
56.77
57.12
57.20
56.77
57.40
56.63
Overfitting
54,27
54.67
54.85
54.30
54.95
54.13
FastAT
46.69
47.33
47.30
46.69
47.58
46.54
at each training iteration we split FF into a meta-train set F of N - n. defenses and a meta-test set F of the other ni defenses. The model-agnostic training objective is to maximize the loss [d;; D) on the meta-train defenses, while ensuring that the update direction of Φ also leads to maximizing [ó; F, D) on the meta-test defenses, such that the learned optimizer will perform well on true test defenses. During meta-train, the optimizer parameters are updated by the one-step gradient ascent
6' = + BV I (o; FD),
where 33rr is the meta-train learning rate. During meta-test, we expect the updated parameters o' to ex- hibit a good generalization performance on the meta-test defenses, which indicates that J (F is also maximized. The meta-traie and meta-test can be optimized simultaneously as
max [b;; F,D) + ( 3V [o;; F);))
(6)
where y a thyperparameter balance the meta-train loss and the meta-test loss. The objective (6) is optimized by gradient ascent. After accumulating the gradients from meta-train and meta- vetaaggaddeens from meta-test, the optimizer trained perform well on attacking both meta-train and meta-test defenses. The gradient and meta-grades weta-gradient flows in the computation graph are illustrated in Fig. 1(b).
We summarize the overall MAMA algorithm in Alg. 1.
4 EXPERIMENTS
In this section, we present the experimental results to validate the effectiveness of BMA and MAMA for adversarial attacks. We follow a simple-to-complex procedure to conduct experiments. See. 4.2 first shows the advantages of the learned optimizer in BMA when considering one defense at time. Sec. 4.3 then proves that the learned optimizer possesses good cross-data, cross-iteration, and cross-model generalization abilities, respectively. Sec. 4.4 demonstrates that BMA and MAMA can achieve state-of-the- performance by comparing with various top- aop-pefforming attacks on 12 defense models. More details about the baseline attacks and defenses are provided in Appendix C and D, respectively. We mainly present the results based on the &o norm in this section. The results based on the & norm and standard deviations of multiple runs are provided in Appendix E.
4.1 EXPERIMENTAL SETUP
The hyperparameters of baseline attacks are set according to the common practice or their original implementations. Specifically, in Sec. 4.2 and Sec. 4.3, we set the attack iterations as = 20 and the fixed step size = 2/255 according to (Madry et al., 2018). The learned optimizer in our method also adopts the same setting for fair comparison. In Sec. 4.4, we adopt the official hy hyperparameter configurations of the baseline attacks since we assume that their hyperparameters have been tuned appropriately. Our BMA and MAMA attacks follow a similar setup with 100 iterations, 20 random restarts, and ODI initialization. In all experiments, we use two-layer LSTM with 20 hidden units in each layer as the optimizer model. We set through an ablation study. We set the learning rates as 0.0001 and 7 0.001 in MAMA with 1.0. The mini-batch size is B = 32.
and X = 0.1 in Eq. (5) and study different x
4.2 EFFECTIVENESS AND ADAPTIVENESS OF BMA
We first train different attack optimizer for each defense by BMA. We consider four typical AT de- fenses on CIFAR-10 10 (Krizhevsky & Hinton, 2009) HYDRA (Sehwag et al., 2020), Pre-Training (Hendrycks et al., 2019), Overfitting (Rice et al., 2020), and FastAT (Wong et al., 2020).
Comparison with hand-designed optimizers. Since most existing adversarial attacks adopt hand- designed optimizers, we first compare the performance of our learned optimizer with these hand- designed ones. Specifically, we adopt the CW loss (Carlini & Wagner, 2017) as L in Eq. (1) and set

---

Under review as a conference paper at ICLR 2022
Table 2: Classification accuracy (%) of four defenses on CIFAR-10 against PGDCE. PGDDLR. MT, ODI-RIO, APGDDER and their extensions by integrating with our proposed BMA method. The default attack iterations are set as T 20 as detailed in Sec. 4.1, and we also involve another 100-step APGDDL as APGDoLx-100.
Fixed Step Size
Automatic See Size
Defense
PGDes
PGD
MT
ODI-RIO
APGDoLK
APGDau-100
Baveline
+BMA
Baseline
+BMA
Baseline
+BMA
Buseline
+BMA
Baseline
+BMA
Buseline
+BMA
HYDRA
60,38
60.27
39.16
59.02
57.63
57.54
57,50=0.01
57,41=0.01
58.89
58.74
58.47
58.37
Pre-training
57.89
$7.69
57.66
55.09
15.11:00:11
55.04:0.001
57.83
57.44
$7.15
Overtiting
56.08
55.87
55.29
55.13
55,71
55.64
52.68=0.02
52.57=0.01
55.19
54.84
54.60
54.45
FastAT
46.82
46.70
48.19
48.01
43.85
43.72
43.76=00001
43.66:20.00
48.02
47.53
47.18
46.85
61.0
80
2=00
1,000
60.5
k=001
2,000
2-003
56
Asauray (%)
60.0
2m0.5
10,000
52
89.5
Accuracy
48
58.5
44
58.0
15
20
25
40
Beration
HYDRA Pre-Tmining Overfitting
FastAT
Figure 2: The accuracy curves of HYDRA w.r.t. training iterations of BMA with different A.
Figure 3: Cross-data generalization evaluation with dif- ferent numbers of training data points on CIFAR-10.
<= 8/255.. We compare our learned optimizer (in BMA) with vanilla gradient descent (used in PGD (Madry et al., 2018)), momentum (Polyak, 1964) (used in MI-FGSM (Dong et al., 2018)), nesterov accelerated gradient (Nesterov, 1983) (used in NI-FGSM (Lin et al., 2019)), and Adam (Kingma & Ba, 2015) (used in (Uesato al., 2018; Gowal et al., 2019)). Besides, we also adopt CNN-based adversarial example generation method (Adv-CNN) (Baluja & Fischer, 2017) for comparison. We report the classification accuracy of the four defenses against these attacks in Table 1. We can see that BMA with the learned optimizer leads to lower accuracy of the defense models than the existing attacks, demonstrating that BMA can discover stronger adversarial attacks automatically.
Ablation study of the prior-guided refinement strategy. We conduct an ablation study to inves- tigate the effects of the prior-guided refinement strategy introduced in Sec. 3. Fig. shows the accuracy of HYDRA along the training iterations of BMA with Xe := À 0.0, 0.01, 0.1, 0.5, and 1.0, respectively. After integrating the PGD update rule as a prior with A > O. the learned opti- mizer achieves faster convergence and better performance than that with À = O. The results indicate that the proposed prior- -ooirggudddd refinement strategy can benefit from the informative prior given by the PGD update rule and explore potential better update directions simultaneously. Moreover, as A increases, to a certain extent the convergence tends to be faster, but the performance tends to be similar. Therefore, we select À = 0.1 for all experiments in the training phase of BMA.
The learned optimizer is generally compatible with previous methods on improving adversarial attacks in different aspects. To verify this, we incorporate our method with PGD vari- ants using the cross-entropy loss (PGDcE) (Madry et al., 2018), the difference of logits ratio loss (PGDDlR) (Croce& & Hein, 2020b) and PGDDLR with varying step size (APGDpLR) (Croce & Hein, 2020b), the Multi (MT) attack (Gowal et al., 2019) that performs targeted attack for each class, and the output diversified initialization method (Tashiro al., 2020) with 10 random restarts (0DI-R10). Note that these methods only change the loss function L or the starting point d.. which do not affect the learning process of BMA. Table 2 shows the results of the baseline methods and their extensions by integrating with the learned optimizer in BMA. The results verify that integrat- ing BMA can consistently enhance the performance of each method. The mechanism of combining BMA is concise and easy to implement, yet steadily improving the different attack methods.
Adaptiveness.
4.3 GENERALIZATION EVALUATION
Cross-data generalization. We first evaluate the cross-data generalization ability of the learned optimizer in BMA on unseen data points. Specifically, we randomly select 1, 000, 2, 000, 5, 000, and all 10, 000 data points from the CIFAR-10 test set as the training set for BMA, respectively, and adopt the entire test set of CIFAR- 10 for evaluation. The performance on the entire test set is shown in Fig. 3. We can that the learned optimizers using different numbers of data points achieve
7

---

Under review as a conference paper at ICLR 2022
Table 3: Cross-model generalization performance of four unseen defenses on CIEAR-IO, while the optimizer
trained on four random defenses from TRADES (T), AWP (A), Unlabeled (U), Hydra (H), and SAT (S).
Training Defenses
Pre- Pre-Training
BoT
PGD-AT
FastAT
BMAs,
BM
MAMA
BMAs,
BM
MAMA
BMAA
BMAm
MAMA
BMA
BMA
MAMA
(TAA
56.84
56.79
56.74
56.01
55.98
55.93
50.82
50.80
50.75
46.77
46.75
46.69
(.A.UUS))
56.84
56.81
56.74
56.02
55.98
55.94
50.82
50.80
50.74
46.78
46.75
46.69
T.U.H.SS
56.85
56.80
56.75
56.02
55.99
55.93
50.82
50.81
50.76
46.78
46.75
46.68
HYDRA
Tre-Tnining
Overttiing
59.4
57.4
54.8
473
MILFOSM
$7.2
47.1
RMA
36.8
58.6
36.6
54.0
46.-
56.4
$5.4
51.5
46
DeeciSoon
Beration
Iteratonn
Figure 4: Comparing the performance of different optimizers based on PGD adversary against four defonse models. The iteration needed for training the automated optimizer (BMA) is also marked at 20, The results also evidence that BMA can directly extend attack iterations from 20 to a longer step.
similar attack performance on the four defense models, indicating that the data-driven procedure enables the learned optimizer obtain a good generalization ability to new unseen data samples.
Cross-iteration generalization. We then evaluate the generalization ability of the learned optimizer in BMA to more attack iterations than training. Fig. shows the curves of model accuracy w.r.t. the number of attack iterations ranging from 1 to 100. We also mark the number of iterations used for training BMA at 20. The results on four defense models verify that BMA can consistently outper- form the baseline attacks with varying attack iterations. It demonstrates that the learned optimizers also have a good generalization ability across attack iterations.
Cross-model generalization. The training process may face inconvenience in practice when train- ing a different optimizer for each defense. A better property of the learned optimizer is more gen- eralizable for adversarial attacks on any unseen defenses. We thus consider the challenging and practical scenario - training the optimizer on several available defense models and evaluating its generalization ability to other unseen defense models. This setting requires the optimizer to learn more general knowledge for adversarial attacks. For choosing the defense models training, we mainly consider using set of diverse defenses. Thus we randomly sample four models to train the optimizer from TRADES (T) (Zhang et al., 2019b) AWP (A) (Wu et al., 2020), Unlabeled (U) (Carmon et al., 2019), Hydra (H) (Sehwag et al., 2020), and SAT (S) (Huang et al., 2020). MAMA randomly selects three defenses as meta-train and the other one as meta-test in each iteration. We compare MAMA with two BMA variants. The first one trains optimizer for each training defense and chooses the best optimizer for evaluation (BMA))) and the second one optimizes Eq. (5) using the training defenses as an ensemble (BMAcn). Table shows that MAMA more effective than the two methods, indicating that the generalization ability of the optimizer can be improved by adopting the -gdel-agn algorithm. The effect of meta-test components listed in Appendix E.3.
4.4 COMPARISON WITH STATE-OF- ATTACKS
In this section, we perform a large-scale experiment compare the performance of our methods with various state-of-the-art attacks for robustness evaluation of 12 defense models on CIFAR-1O. We consider both BMA that learns a different attack optimizer for each defense model and MAMA that only learns one generalizable optimizer adopting the model-agnostic training algorithm to test all defenses. We compare our methods with several typical and state-of-the-ar art attacks, including 1) PGDCE (Madry et al., 201 8), 2) PGDcw. 4) APGD (Croce & Hein, 2020b) with the DLR loss, 3) MT, 5) AutoAttack (AA) (Croce& & Hein, 2020b), and 6) ODI (Tashiro et al., 2020). PGD PGDcw. and ODI use 100 iterations with step size 2/255 and 20 restarts. APGD also uses the same iterations and restarts. MT performs targeted attacks for 9 classes except the true one with restarts and 100 attack iterations. We adopt the official implementation of AA. We integrate methods with ODI to establish stronger attacks. Our methods also use 100 attack iterations with 20 ODI restarts. MAMA is trained on defense models (T.A.UUH)) with the model-agnostic algorithm.
Table 4 shows the classification accuracy of these defense models against different adversarial at- tacks. We also show the clean accuracy and the reported robust accuracy of these models in the
8

---

Under review as a conference paper at ICLR 2022
Table 4: Classification accuracy (%) of different defenses CIFAR-10 under various white-box attacks. The defenses marked with adopt additional unlaheled datasets. The defenses marked with adopt = 0.031 as originally reported, and the others adopt = 8/255. A represents the difference between reported accuracy and testing accuracy by ours. A' represents the difference between accuracy under the best attack and ours. We also present the complexity (forward times) of each attack method in blue.
Defense
Clean Reported
PGDcE
PGDew
APGD
MT
AA
ODI
BMA
MAMA
A
2k
2k
2k
1.8k
7.4k
2k
2k
2k
AWP 88.25
60.04
63.12
60.50
60.46
60.17
60.04
60.10
59.99
59.98
-0.06
-0.06
Unlabeled
89.69
62.50
61.69
60.61
60.55
59.73
59.53
59.65
59.50
59.50
-3.00
-0.03
HYDRA
88.98
59.98
59.53
58.19
58.20
57.34
57.20
57.24
57.11
57.12
-2.87
-0.09
Pre-Training
87.11
57.40
57.07
56.36
56.90
55.00
54.94
54.96
54.82
54.81
-2.59
-0.13
BoT
85.14
54.39
56.15
55.53
55.66
54.44
54.39
54.34
54.24
54.24
-0.15
-0.15
AT-HE
85.14
62.14
61.72
55.27
55.87
53.81
53.74
53.79
53.69
53.70
-8.45
-0.05
Overfitting
85.34
58.00
56.87
55.10
55.60
53.43
53.42
53.42
53.38
53.37
-4.63
-0.05
SAT
83,48
58.03
56.21
54.39
54.54
53.46
53.34
53,42
53.28
53.27
-4.76
-0.07
TRADES
84.92
56.43
55,22
53.95
53,94
53.25
53.09
53.09
53.01
53.02
-3.42
-0.07
LS
86.09
53.00
55.81
54.44
54.90
52.82
53.00
52.72
52.61
52.60
-0.40
-0.12
YOPO
87.20
47.98
45.98
46.88
47.19
44.97
44.87
44.94
44.81
44.82
-3.17
-0.06
FastAT
83.80
46.44
45.93
45.88
46.37
43.53
43.41
43.50
43.29
43.32
-3.15 -0.12
Table 5: Classification accuracy (%) of different defenses on CIFAR-100 100 and ImageNet under various white- box attacks. We directly apply the learned optimizer in MAMA on CIFAR- 10 without further training.
Defense
Dataset
e
Clean
Reported
PGDce
AA
MAMA
A
AWP
8/255
60.38
28.86
33.27
28.86
28.85
-0.01
-0.01
Pre-Trainin Training"
CIFAR-100
8/255
59.23
33.50
33.10
28.42
28.38
-5.12
-0.04
Overfitting
8/255
53.83
28.10
20.50
18.96
18.92
-9.18
-0.04
FreEAT-HE (ResNet-50)
4/255
61.83
39.85
36.70
30.65
30.60
-9.25
-0.05
ImageNet
FreeAT-HE (WRN-50-2)
4/255
65.28
43.47
42,61
32.65
32.65
-10.82
0.00
corresponding papers. Our methods achieve lower robust test accuracy on all 12 defense models than the baseline attacks, including MT and AA, demonstrating that ours discover stronger adversar- ial attacks automatically than the existing attacks with hand-designe desiide optimizers. Mcanwhile, our methods are not only stronger but also less complex than AA. Besides, the results also demonstrate that MAMA can achieve a comparable performance of BMA, showing that the learned optimizer has good generalization ability unseen defense models due to the involved model-agnostic training algorithm, and could be flexibly used to evaluate the robustness of future defenses.
Robustness evaluation on other datasets. We directly apply the learned optimizer in MAMA on CIFAR-10 to evaluate adversar- ial robustness on CIFAR-100 (Krizhevsky & Hinton, 2009) and
UUI
BMA
Tiee pet fuad
ImageNet (Deng et al., 2009), without updating the optimizer by new datasets or defenses. We adopt e = 8/255 on CIFAR- 100 and c 4/255 on ImageNet. During testing, there is significant shift between training and testing defense models for MAMA. Despite these difficulties, MAMA can consistently improve the performance on most defenses in Table 5. The results verify that MAMA obtains a good generalization ability for adversarial at- tacks on unseen datasets, benefiting from the data-driven learning procedure automatically.
FGRAT HYDKA Pre-Trainig TRADES
Figure 5: The running time of AA, ODI, and BMA with a mini-batch of 100 data points on CIFAR-10.
Running time. Fig. 5 shows the running time of AA, ODI, and BMA (MAMA) based on a single RTX 2080-Ti GPU. BMA is slightly slower than the baseline ODI, but significantly faster than AA.
CONCLUSION
In this paper, we propose a Model-Agnostic Meta-Attack (MAMA) approach to automatically dis- cover stronger attack algorithms. Specifically, we learn an optimizer in adversarial attacks param- eterized by an RNN model, which is trained over a class data samples and defense models to produce effective update directions for crafting adversarial examples. We further develop model- agnostic learning approach with a gradient-based meta- train and meta-test procedure to improve the generalization ability of the learned optimizer to unseen defense models. Extensive experimental re- sults demonstrate the effectiveness, adaptiveness, and generalization ability of MAMA on different defense models, leading more reliable evaluation of adversarial robustness.

---

Under review as a conference paper at ICLR 2022
ETHICS STATEMENT
Our work aims at evaluating the adversarial robustness of defense models more accurately and re- liably. It could identify the security problems of deep learning models before they are deployed to avoid potential harm caused by adversarial examples. We are unaware of any direct negative impact of this work. One limitation of our work that the proposed methods (i.e., BMA and MAMA) cannot directly be used to attack defenses with obfuscated gradients. We still need to design some adaptive attack strategies (e.g., Expectation over Transformation (Athalye al., 201 8b) for random- ized defenses) to apply our methods.
REPRODUCIBILITY STATEMENT
We provide the code for reproducing the results in the supplementary material.
REFERENCES
Jean-Baptiste Alayrac, Jonathan Uesato, Po-Sen Huang, Alhussein Fawzi, Robert Stanforth, and Pushmeet Kohli. Are labels required for improving adversarial robustness? In Advances in Neural Information Processing Systems (NeurIPS). pp. 12192 12202, 2019.
Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein. Square at- tack: a query-efficien black-box adversarial attack via random search. In European Conference Computer Vision (ECCV), 2020.
Marcin Andrychowicz, Misha Denil, Sergio Gómez Colmenarejo, Matthew W Hoffman, David Pfau, Tom Schaul, Brendan Shillingford, and Nando de Freitas. Learning to learn by gradient descent by gradient descent. In Neural Information Processing Systems (NIPS), pp. 3988-3996, 2016.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give a false sense se- curity: Circumventing defenses to adversarial examples. In International Conference Machine Learning (ICML), 2018a.
Anish Athalye, Logan Engstrom, Andrew Ilyas, and Kevin Kwok. Synthesizing robust adversarial examples. In International Conference on Machine Learning (ICML), 2018b.
Shumeet Baluja and lan Fischer. Adversarial transformation networks: Learning to generate adver- sarial examples. arXiv preprint arXiv: VrXivv:773.00000 1703.09387 2017.
Irwan Bello, Barret Zoph, Vijay Vasudevan, and Quoc V Le. Neural optimizer search with rein- forcement learning. In International Conference on Machine Learning, pp. 459-468. PMLR, 2017.
Yoshua Bengio, Nicholas Léonard, and Aaron Courville. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv: 308.3432,
Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Pavel Laskov, Giorgio Giacinto, and Fabio Roli. Evasion attacks against machine learning at test time. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 387-402, 2013.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In IEEE
Symposium on Security and Privacy, 2017.
Nicholas Carlini, Anish Athalye, Nicolas Papernot, Wieland Brendel, Jonas Rauber, Dimitris Tsipras, Ian Goodfellow, and Aleksander Madry. On evaluating adversarial robustness. arXiv preprint arXiv: 19xv::902.00705. 2019.
Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, Percy Liang, and John C Duchi. Unlabeled data improves adversarial robustness. Advances in Neural Information Processing Systems (NeurIPS). 2019.
10

---

Under review as a conference paper at ICLR 2022
Tianlong Chen, Sijia Liu, Shiyu Chang, Yu Cheng, Lisa Amini, and Zhangyang Wang. Adversarial robustness: From self-suuer pre-training to fine-tuning. In Conference on Computer Vision and Pattern Recognition (CVPR), pp. 699-708, 2020a.
Tianlong Chen, Weiyi Zhang, Zhou Jingyang, Shiyu Chang, Sijia Liu, Lisa Amini, and Zhangyang Wang. Training stronger baselines for learning to optimize. In Advances in Neural Information Processing Systems (NeurIPS), 2020b.
Yutian Chen, Matthew W Hoffman, Sergio Gómez Colmenarejo, Misha Denil, Timothy P Lilli- crap, Matt Botvinick, and Nando Freitas. Learning to learn without gradient descent by gradient descent. In International Conference on Machine Learning (ICML), pp. 748-756, 2017.
Francesco Croce and Matthias Hein. Minimally distorted adversarial examples with a fast adaptive boundary attack. In International Conference Machine Learning (ICML), 2020a.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with ensemble of diverse parameter-fre free attacks. In International Conference on Machine Learning, pp- 2206-
2216. PMLR, 2020b.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2009.
Yinpeng Dong, Fangzhou Liao, Tianyu Pang. Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boost- ing adversarial attacks with momentum. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR), 2018.
Yinpeng Dong, Zhijie Deng, Tianyu Pang, Hang Su, and Jun Zhu. Adversarial distributional training for robust deep learning. In Advances in Neural Information Processing Systems, 2020a.
Yinpeng Dong, Qi-An Fu, Xiao Yang, Tianyu Pang, Hang Su, Zihao Xiao, and Jun Zhu. Benchmark- ing adversarial robustness on image classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 321-331, 2020b.
Jiawei Du, Hu Zhang, Joey Tianyi Zhou, Yi Yang, and Jiashi Feng. Query-efficient meta attack to deep neural networks. In International Conference Learning Representations (ICLR), 2020.
Logan Engstrom, Andrew lyas, Hadi Salman, Shibani Santurkar, and Dimitris Tsipras. Robustness (python library), 2019. URL https: /tggt//gtttttt com cww//odd////cceecccoooooo/ooo Madrylaa Labb robustness.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the 34th International Conference on Machine Learning, pp- 1126-1135, 2017.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations (ICLR), 2015.
Sven Gowal, Jonathan Uesato, Chongli Qin, Po-Sen Huang, Timothy Mann, and Pushmeet Kohli. pgd-based based adversarial testing. arXiv preprint 19000939,,
An alternative surrogate loss for pgd- 2019.
Sven Gowal, Chongli Qin, Jonathan Uesato, Timothy Mann, and Pushmeet Kohli. Uncovering the limits of adversarial training against norm-bounded adversarial examples. arXiv preprint arXiv:2010.03593, 2020.
Will Grathwohl, Kuan-Chieh Wang, Joern-Henrik Jacobsen, David Duvenaud, Mohammad Norouzi, and Kevin Swersky. Your classifier is secretly an energy based model and you should treat it like one. In International Conference on Learning Representations (ICLR), 2020.
Dan Hendrycks, Kimin Lee, and Mantas Mazeika. Using pre-training can improve model robustness and uncertainty. In International Conference on Machine Learning, pp. 27 12-2222.. PMLR, 2019.
Lang Huang, Chao Zhang, and Hongyang Zhang. Self-adaptive training: beyond empirical risk minimization. arXiv preprint arXiv:2002.1033 10319, 2020.

---

Under review as a conference paper at ICLR 2022
Haoming Jiang, Zhehui Chen, Yuyang Shi, Bo Dai, and Tuo Zhao. Learning to defend by learning to attack. In Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, pp. 577-585, 2021.
Minseon Kim, Jihoon Tack, and Sung Ju Hwang. Adversarial self-supervised contrastive learning- In Advances in Neural Information Processing Systems, volume 33, 2020.
Diederik Kingma and Jimmy Ba. Adam: method for stochastic optimization. In International Conference on Learning Representations (ICLR), 2015.
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Tech- nical report, University of Toronto, 2009.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial examples in the physical world. In International Conference on Learning Representations (ICLR) Workshops, 2017a.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial machine learning at scale. In International Conference on Learning Representations (ICLR), 2017b.
Da Li, Yongxin Yang. Yi-Zhe Song, and Timothy Hospedales, Learning to generalize: Meta-learning for domain generalization. Proceedings of the AAAI Conference on Artificial Intelligence, 2018.
Ke Li and Jitendra Malik. Learning to optimize. In International Conference on Learning Repre- sentations (ICLR), 2017.
Fangzhou Liao, Ming Liang. Yinpeng Dong, Tianyu Pang, Xiaolin Hu, and Jun Zhu, Defense against adversarial attacks using high-level representation guided denoiser. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
Jiadong Lin, Chuanbiao Song. Kun Hc, Liwei Wang, and John E Hopcroft. Nesterov accelerated gradient and scale invariance for adversarial attacks. In International Conference on Learning Representations (ICLR), 2019.
Kaifeng Lv, Shunhua Jiang, and Jian Li. Learning gradient descent: Better generalization and longer horizons. In International Conference on Machine Learning (ICML), pp. 2247-2255, 2017.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations (ICLR), 2018
Chengzhi Mao, Ziyuan Zhong, Junfeng Yang, Carl Vondrick, and Baishakhi Ray. Metric learning for adversarial robustness. Advances in Neural Information Processing Systems (NeurIPS), pp- 478-489, 2019.
Yurii E Nesterov. A method for solving the convex programming problem with convergence (1/k* 2). In Dokl. akad. nauk Sssr, volume 269, pp. 543-547, 1983.
Tianyu Pang, Kun Xu, Chao Du, Ning Chen, and Jun Zhu. Improving adversarial robustness via promoting ensemble diversity. In International Conference on Machine Learning (ICML),201)
Tianyu Pang, Xiao Yang, Yinpeng Dong, Kun Xu, Hang Su, and Jun 7hu. Boosting adversarial training with hypersphere embedding. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
Tianyu Pang. Xiao Yang. Yinpeng Dong, Hang Su, and Jun Zhu. Bag of tricks for adversarial training. In International Conference on Learning Representations (ICLR), 2021.
Boris T Polyak. Some methods of speeding up the convergence of iteration methods. Ussr compu- tational mathematics and mathematical physics, 4(5):1-17, 1964.
Omid Poursaeed, Isay Katsman, Bicheng Gao, and Serge Belongie. Generative adversarial pertur- bations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1422-4431 2018.
12

---

Under review as a conference paper at ICLR 2022
Leslie Rice, Eric Wong, and J Zico Kolter. Overfitting in adversarially robust deep learning. In International Conference on Machine Learning (ICML). 2020.
Vikash Sehwag, Shiqi Wang, Prateek Mittal, and Suman Jana. Hydra: Pruning adversarially robust
neural networks. Advances in Neural Information Processing Systems (NeurIPS), 2020.
Ali Shafahi, Mahyar Najibi, Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry Davis, Gavin Taylor, and Tom Goldstein. Adversarial training for free! In Advances in Neural Information Processing Systems (NeurIPS), 2019.
Gaurang Sriramanan, Sravanti Addepalli, Arya Baburaj, et al. Guided adversarial attack for evalu- ating and enhancing adversarial defenses. Advances in Neural Information Processing Systems, 33, 2020.
Christian Szegedy. Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfel- low, and Rob Fergus. Intriguing properties of neural networks. In International Conference on Learning Representations (ICLR), 2014.
Yusuke Tashiro, Yang Song, and Stefano Ermon. Diversity can be transferred: Output diversification for white-and black-box attacks. Advances in Neural Information Processing Systems, 33, 2020.
Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Dan Boneh, and Patrick McDaniel. Ensemble adversarial training: Attacks and defenses. In International Conference on Learning Representa- fions (ICLR), 2018.
Florian Tramer, Nicholas Carlini, Wieland Brendel, and Aleksander Madry. On adaptive attacks to adversarial example defenses. Advances in Neural Information Processing Systems, 2020.
Jonathan Uesato, Brendan O'Donoghue, Aaron van den Oord, and Pushmeet Kohli. Adversarial risk and the dangers of evaluating against weak attacks. In International Conference Machine Learning (ICML), 2018.
Olga Wichrowska, Niru Maheswaranathan, Matthew W Hoffman, Sergio Gomez Colmenarejo, Misha Denil, Nando Freitas, and Jascha Sohl-Dickstein. Learned optimizers that scale and gen- eralize. In International Conference on Machine Learning (ICML), pp. 3751-3760, 2017.
Eric Wong and Zico Kolter. Provable defenses against adversarial examples via the convex outer adversarial polytope. In International Conference on Machine Learning (ICMI.,, 2018
Eric Wong, Leslie Rice, and J. Zico Kolter. Fast is better than free: Revisiting adversarial training. In International Conference on Learning Representations (ICLR), 2020.
Dongxian Wu, Shu-Tao Xia, and Yisen Wang. Adversarial weight perturbation helps robust gener- alization. Advances in Neural Information Processing Systems, 33, 2020.
Yuanhao Xiong and Cho-Jui Hsieh. Improved adversarial training via learned optimizer. In Euro- pean Conference on Computer Vision (ECCV), pp. 85-100, 2020.
Runtian Zhai, Tianle Cai, Di He, Chen Dan, Kun He, John Hopcroft, and Liwei Wang. Adversarially robust generalization just requires more unlabeled data. arXiv preprint arXiv: 00555, 2019.
Dinghuai Zhang, Tianyuan Zhang, Yiping Lu, Zhanxing Zhu, and Bin Dong. You only propagate once: Accelerating adversarial training via maximal principle. arXiv preprint arXiv: 1905500877 2019a.
Hongyang Zhang, Yaodong Yu. Jiantao Jiao, Eric Xing. Laurent El Ghaoui, and Michael I Jordan. Theoretically principled trade-off between robustness and accuracy. In International Conference on Machine Learning (ICMI.,, 2019b.
Huan Zhang, Hongge Chen, Chaowei Xiao, Sven Gowal, Robert Stanforth, Bo Li, Duane Boning, and Cho-Jui Hsich. Towards stable and efficient training of verifiably robust neural networks. In International Conference on Learning Representations (ICLR), 2020.
13

---

Under review as a conference paper at ICLR 2022
A MORE DISCUSSIONS ON RELATED WORK
Xiong & Hsieh (2020) propose a similar approach to ours that uses an RNN optimizer to learn attacks for adversarial training. The differences between our method and Xiong & Hsieh (2020) are three-fold. First, the motivations are different: we propose to discover stronger adversarial attack algorithms while they aim learn robust models by adversarial training. Second, based on the different goals, the techniques are different: we develop a prior-guided refinement strategy and model-agnostic training algorithm to improve the attacking performance and generalization ability of the RNN optimizer while they do not adopt these techniques and use a relatively smaller number of attack iterations (i.c., T = 10). Third, the empirical evaluations are different: we mainly conduct experiments to show the attack performance on various defense models while they mainly consider the defense performance of adversarial training. Overall, our method differs from Xiong Hsich (2020) in the aspects of motivation, technique, and experiment. Despite using similar Learning- to-Learn framework, our method has its own contributions developing an effective attack method for evaluating adversarial robustness.
B EXPLANATION OF MAXIMUM A POSTERIORI
Assume that the output of the RNN optimizer is random variable I:- The objective of the optimizer is to maximize the loss C at ã, after applying gt, as in Eq. (3). By using the cross-entropy loss as C, we actually want to minimize Py (i.e., the probability of the true class) given &t. We let ply (9)) 1 such that we want to maximize P(ylge) Thus p(ylgt) can be viewed as the conditional likelihood. By assuming the prior distribution g: be a Gaussian one as p(g:) ~ (Nssign(V (V.), 2x ), we can estimate the posterior distribution of g, using Bayes' theorem
p(g.ly) p(vll)))(gg))
Under the maximum posteriori (MAP) framework, we would like to solve
arg max log p(g, (v) = arg max l- >1llg sign( sign(V.))3)) arg max(_(((()))) v) Aellg. sign V.) 12)).
Be accumulating the loss at each step, we can obtain Eq. (5).
C TECHNICAL DETAILS OF ADVERSARIAL ATTACKS
In this section, we introduce more related backgrounds and technical details for reference.
PGD. One of the most commonly studied adversarial attacks the projected gradient descent (PGD) method (Madry al., 2018). PGD iteratively crafts the adversarial example as
1 II, (ar) (#: + a sign V ((f()) v))),
where (x) fx': || x||00 }} denotes the too ball centered with radius e, I1(() is the
projection operation, and at the step size at the t-th attack iteration. could be the cross-entropy loss (x)) y) = log Py where py ~オ /I// e denotes the predicted probability of class y. or the margin-based CW loss (Carlini & Wagner, 2017) as Lew((f y) = max 2; Zy.
MI-FGSM. Dong et al. (2018) propose adopt the momentum optimizer (Polyak, 1964) in adver- sarial attacks, which can be formulated as
gi+1 1 9:
6.41 (I.. (m) (#, + sign(g++1)), V C((())
where is fixed step size and j is hyperparameter.
NI-FGSM. Lin et al. (2019) propose to adopt the Nesterov accelerated gradient (Nesterov, 1983) in adversarial attacks, as
veL(f(2)"")))) ).y)
2t
nes = -; + t.....p.. Dt: 9t+1 4. +
(f(())))) ), :b:41 = 1I. (m) + signnnggggg))))
V
14

---

Under review as a conference paper at ICLR 2022
Adam. Some attacks (Uesato al., 2018; Gowal et al., 2019) propose adopt the Adam opti- mizer (Kingma Ba, 2015) in adversarial attacks. By replacing the sign gradient PGD by the get the attack based on Adam easily.
Adam optimization rule,
Adv-CNN. Baluja Fischer (2017) first propose to generate adversarial examples against target classifier by training feed-forward CNN networks. To efficiently craft adversarial examples under white-box attack, we directly feed the gradients of adversarial images into the CNN, and output the update direction
g = G u)); ăe+1 = (x) (0, + 04 92),
where G indicates the feed-forward CNN.
AutoAttack (AA), Croce & Hein (2020b) first propose the Auto-PGD (APGD) algorithm, which can automatically tune the adversarial step sizes along with the optimization trend. As for the adversarial objective, they develop new difference of logits ratio (DLR) loss as
By max
Cf
2x1 R
where is the logits and is the ordering which sorts the components of z. Finally, the authors propose to group APGDCE and APGDDLR with FAB (Croce & Hein, 2020a) and square attack (An- driushchenko al., 2020) to form the AutoAttack (AA).
ODI attack. The Output Diversified Sampling (ODS) (Tashiro et al., 2020) belongs to a sampling strategy that aims to maximize diversity in the models' outputs when crafting adversarial examples. The formulation of ODI attack derived from ODS can be described as
V f V (w2) (
+ 1l, (m) (2x az Sign(voos((( (+x, f, wa))), VODS (0k, wa)
where v-Ds(-) generates output-diversified starting points, and wa is sampled from the uniform distribution over [|
MultiTarree (MT) attack. MT (Gowal et al., 2019) performs targeted attack for all other classes except the true one, which iteratively crafts the adversarial example as
Lt+1 IIg, (a) ( 0g sign(V (V
where yf belongs to the specific targeted attack class.
D REFERENCE CODE OF ADVERSARIAL DEFENSES
We provide the architectures and code links for the defenses used in our paper in Table 6. Please note that the checkpoints of evaluating adversarial robustness are released in their code implementations.
Table 6: We summarize the architectures and code links for the referred defense methods.
Method
Architecture
Code link
AWP (Wa 2020)
WRN-23-T0
Unlabeled (Cannon et al., 2019)
WRN-28-10
cettub.cem/yaiicarmeee
HYDRA (Schwag et 2020)
WRN-28-10
gathub.come
cittub..om/ingginee group/hydra
Pre mre-iinnng (Hendrycks et al., 2019)
WRN 21.
BoT (Pang et al., 2021)
WRN-34-20
Btiubb.com//23////a.. Har-eaaa-ooooooosoooee -ricks-for-AAT
HEE (Pang al., 2020)
WRN. 34 20
Stltuh..om/Shhhnne YYng//AT HH
Overfitting (Rice et al., 2020)
WRN-34-22
githu..om//ocuslala
overfitring
SAT (Huang al., 2020)
WRN-34-10
TRADES (2hang et al., 2019b)
WRN-34-10
cultub.cem/yaoddonyyaaee RRDDEE
(Pang et al., 2021)
gilub..umm/2333//Bgg Tricks
YOPO (7hang et al., 2019a)
WRN-34.10
gituh.com/al66000
FistAT (Wong al., 2020)
ResNet18
FreeAT-HE: (Pang 2020)
ResNet-50
githuh
Sttuh..om/ShassXYYagyAY HE
E MORE EXPERIMENTS
E. EFFECTIVENESS UNDER THE l NORM
Apart from the results based on the loo norm, we study the effectiveness based on the & norm. The performance of learned optimizer with these hand-designed is presented in this sec-
15

---

Under review as a conference paper at ICLR 2022
tion. Specifically, adopt the CW loss (Carlini & Wagner, 2017) and set 0.1. We compare our learned optimizer (in BMA) with vanilla gradient descent (used in PGD (Madry et al., 2018)), momentum (Polyak, 1964) (used in MI-FGSM (Dong et al., 2018)), nesterov accelerated gradi- ent (Nesterov, 1983) (used in NI-FGSM (Lin et al., 2019)), and Adam (Kingma & Ba, 2015) (used in (Gowal et al., 2019; Uesato al., 2018)). For fair comparison, T = 20 and the fixed step size as a 0.025 in all methods. The learned optimizer in BMA is trained with 20 attack iterations on the entire test set of CIFAR-IO. We report the classification ac- curacy of the two defenses including Overfitting (Rice et al., 2020) and Robustness (Engstrom et al., 2019) against these attacks in Table 7. BMA with the learned optimizer leads lower accuracy of the defense models than the existing attacks with hand-designed optimizers, demonstrating the effectiveness of BMA based on the & norm.
set the attack iterations as
Table 7: Classification accuracy (%) of defense models on CIFAR-10 against various white-box attacks with different optimizers, including PGD, MI-FGSM, NI-FGSM, Adam and the learned optimizer (BMA), based on the & norm.
Defense Overfitting
PGD
MI-FGSM
NI-FGSM
Adam
BMA (Ours)
68.80
69.07
69.02
76.56
68.64
Robustness 70.29
70.46
70.39
78.52
70.04
E.2 RANDOMNESS EXPERIMENTS
To avoid the randomness caused by single experiment, we report the standard deviations of mul- tiple runs on CIFAR-10 in Table 8. We can see that the learned optimizers using different runs achieve similar attack performance on the four defense models including HYDRA (Sehwag et al., 2020), Pre-training (Hendrycks et al., 2019), Overfitting (Rice et al., 2020) and FastAT' (Wong et al., 2020). The results also demonstrate the stability of BMA to existing attacks.
Table 8: Standard deviations of multiple runs CIFAR-10 against PGDcw and its extension be integrating with our proposed BMA method.
Method HYDRA
Baseline
+BMA
+BMA (run-l)
+BMA (run-2)
+BMA (run-3)
+BMA (run-4)
58.73
58.58
0.01
58.57
58,59
58.58
58.57
Pre-training
56.77
56.64 0.01
56.63
56.63
56.65
56.64
Overfitting
54.27
54.1411 0.01
54.15
54.13
54.14
54.13
FastAT
46.69
46.54 0.01
46.54
46.55
46.55
46.54
E.3 ESTIMATE THE EFFECT META- TESTINGG COMPONENT
Effect of different μs, The meta-train and meta-test can be optimized simultaneously as Eq. (6). and μ is the hyperparameter to balance the meta-train and meta-test losses. To estimate the effect of different μ., sample four models to train the optimizer from TRADES (T), AWP (A), Unlabeled (U), Hydra (H), and SAT (S), and test the performance on Pre- reeiirannng for different p.. We can see in Table proper value 1.0 gives the best performance, indicating that the meta-train and meta-test should be equally learned.
Table 9: Testing the performance for different Jr.
0.5
0.8
1.2
1.5
Accuracy (%)
56.79
56.77
56.74
56.76
56.77
Effeet of different numbers meta-train and meta-test. Apart from the results based on different hyparameters, study the effectiveness for different numbers of meta-train and meta-test. In the experiments, we attempt to sample m domains as meta-train, and remaining n domains as meta-test, thus named Sm-Tn. We report the classification accuracy of the defense model different numbers of domains in Table 10. We can see that the best performance can be obtained by setting m as 3 and 1, which is also adopted in all experiments of MAMA.
16

---

Under review as a conference paper at ICLR 2022
Table 10: Testing the performance of different numbers of meta-train and meta-test.
Sampling mode Accuracy (%)
STT3 56.80
S2T2
S3TT 56.74
56.78
E.4 AUTOMATIC STEP SIZE
We also attempt to apply the automatic step size scheduler (Croce & Hein, 2020b) to MAMA in Table 11, which adopts 100 iterations and 20 restarts based on ODI on the different defenses includ- ing TRADES (Zhang et al., 2019b), Pre-training (Hendrycks et al., 2019), HYDRA (Sehwag al., 2020). The results show that integrating the automatic step size does not further improve the overall performance for robustness evaluation. We think that combining MAMA with ODI can already con- verge sufficiently good results for robustness evaluation and may not be improved by integrating other techniques such as automatic step size.
Table 11: Classification accuracy (%) of defense models on CIFAR-10 against MAMA with fixed automatic step sizes.
Method TRADES
Fixed Step Size
Automatic Step Size
53,02
53,01
Pre-Training
54.81
54.81
HYDRA
57.12
57.11
E.5 OTHER TYPES OF DEFENSES
We further conduct experiments by considering other types of defenses rather than adversarial train- ing. Specifically, we include a certified defense - CROWN-IBP (Zhang et al., 2020) and ran- domized defense JEM-1 (Grathwohl et al., 2020). We compare proposed methods BMA and MAMA with AutoAttack (AA), which the most effective baseline. We re-train the attack optimizer for each model in BMA and directly use the learned optimizer (i.e., the same Table 4) MAMA. To counter the randomness JEM-1, we average the gradients over runs (known as Expectation over Transformation (EoT) (Athalye al., 2018b)). The results shown in Table 12. Our methods consistently achieve better performance than AutoAttack.
Table 12: Classification accuracy (%) of different types of defense models on CIFAR-10 against various white-box attacks.
Defense
AA
BMA
MAMA
CROWN-IBP
33.71
33.60
33.64
JEM-1
7.40
6.35
6.57
17


Reviews:
[{"review_id": "fGh0UrXYryE", "reviewer": "Reviewer_Xe9v", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I think the proposed adversarial attack algorithm makes a nice contribution to the adversarial machine learning community, and this method can be considered as the main evaluation for a newly proposed defense model. \n\nThe strengths are listed as follows:\n\n1: The proposed adversarial attack has less computational cost compared with other state-of-the-art attacks (As shown in Table 4).\n\n2: The proposed attack has outperformed other strong attacks under 12 robust defense methods.\n\n3: The proposed attack can be applied to a large-scale dataset (ImageNet as shown in Table 5)\n\nThe weakness of the paper is the lack of novelty. Meta-learning-based adversarial attacks have been considered before and using RNN as an optimizer has also been considered. The contributions of this paper are those improvements made to improve the performance.\n\n", "summary_of_the_paper": "This paper proposes a model-agnostic meta-attack and achieves promising adversarial attack performance compared with state-of-the-art adversarial attack algorithms. The proposed algorithm overcomes many issues, such as the vanishing gradient problem, training instability problem, generalization to other defense models. The detailed experiments support the proposed method.", "summary_of_the_review": "In the perspective of proposing an adversarial attack algorithm, the author has conducted detailed experiments to support the method. However, the contributions of this paper are the improvements to the existing framework.\n\nHence, I support a marginal acceptance of this paper.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "aucQpW2WtDV", "reviewer": "Reviewer_jTw1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The presentation of the paper was very good. Clearly laid out and comprehensively presented, with thorough experiments to back up the claims.\n\nI am not an expert in this particular area, and so my comments may be misguided or have missed key issues (my Confidence score reflects this).\n\nMy main reservation is that the improvement in performance is only minor. That said, since it comes at little extra computational cost, it is certainly not an argument to not use it.\n\nRelated to the above comment, the deltas reported in Tables 4 and 5 are against the Reported accuracies from the source rather than the author’s discovered accuracies from their experiments. This seems to show their method(s) in a better light and so is a bit disingenuous.\n\nSome minor comments:\n-\tRemove “the” (“the maliciously ...”) in first sentence of Introduction. Same at the start of the Ethics Statement\n-\tIn Algorithm 1, n is undefined\n-\t“be integrating” should be “by integrating” in caption for Table 2\n-\tAre there missing bold highlights on some of the data in Table 3?\n-\tThe Ethics Statement section is not an ethics statement. It is more of a summary of the paper.\n-\tIn the Ethics Statement: a few grammatical corrections\n    o\t“effects of” should be “effects on”\n    o\t“and may promote new adversarial attacks” (i.e. add “and”\n-\tThe second line of the two line equation in Annex B is corrupted (norm)\n-\tIn Annex C, discussion of the MT attack refers to 9 classes. More general to say “all other classes” of course.\n", "summary_of_the_paper": "This paper proposes both a Basic and a Model-Agnostic Meta-Attack, designed to automatically produce more effective update directions during adversarial example generation than standard or hand-crafted evasion attacks against deep neural networks.", "summary_of_the_review": "The paper provides only small improvements in performance. But since this is at minimal computational cost, this is not necessarily an issue. Given the importance of effective adversarial example generation in properly assessing the risk in usage of ML solutions, any contribution to improving the state of the art is welcome.\n\nThe authors draw a distinction to a similar Learning to Learn approach from 2020, establishing that though their approach is not therefore strongly novel, is distinct and has merit in its own right.\n\nHence I would recommend this paper for publication.\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "NRrxLoJFNcq", "reviewer": "Reviewer_8AXD", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\n+ the usage of meta learning techniques is interesting to create better attacks\n+ preliminary step ahead for the research in creating better adaptive attacks\n\nCons:\n- results suggest a too modest improvement w.r.t. the computational effort for training the attack\n- only adversarial training is considered\n\nMinor:\n- confusion between MAMA and BMA\n\nThe creation of automatic adaptive attacks is an important step ahead for the adversarial machine learning community.\nHowever, the proposed technique is hindered by the following key issues.\n\n**Results suggest a too modest improvement w.r.t. the computational effort for training the attack**\nThe results proposed in the paper show that MAMA and BMA reduces the robustness accuracy of the strongest automatic attack (AA) by a very low delta (Table 5), but the table do not show all the effort that is needed for optimizing the optimizer.\nHence, I am doubtful about the overall technique, since it behaves similarly to other strategies that do not need to be trained on any models.\nAlso, the deltas shown in Table 5 are computed against the original evaluation, and not the output of the best attack against the considered defense. This is misleading, since the discussion should revolve around the efficacy of the proposed attacks against the other ones, and the table seems to suggest an improvement that is not real.\n\n**Only adversarial training is considered**\nWhile it is true that adversarial training (AT) is becoming widely used, it is not the only defense that has been proposed, or that it will be proposed in the future.\nHence, how this method can adapt to a technique whose loss landscape is very different from an adversarially-trained one? Intuitively, more defenses should be added to the set used in the algorithm, but more experiments should be conducted to understand their effect on the meta learner. Or rather, this should be stated as limitations (that are not discussed).\nFinally, the fact that this strategy only addresses AT should be mentioned in the abstract, since it is now misleading the reader into believing that this method can adapt itself to every defense.\n\n**Minor comment: confusion between MAMA and BMA**\nThe authors might need to better describe BMA and MAMA, maybe by creating a better visual plot.\nAs it is now, the paper starts by describing BMA, and later MAMA, but the paper title is centered on MAMA. So, the discovery of BMA is a bit misleading while reading.\n", "summary_of_the_paper": "This paper proposes an automatic approach for attacking classifiers, by approximating a possible adaptive attack that can take place on a newly-published defense. The methodology (MAMA) applies meta machine learning techniques, by training it on different defenses to let it grasp what might be a good signal to follow when attacking an unknown classifier. The authors then test this strategy and a naive version of it (that is, the one that is only trained on the attacks, and not fine tuned on a test set) against other attacks proposed in literature.\nResults suggest a modest improvement against defenses hardened with adversarial training.", "summary_of_the_review": "I opt for a rejection, since I struggle to see a clear contribution w.r.t. the literature. Other attacks that do not need any optimization have comparable efficacy to the ones scored by BMA and MAMA, but without all the effort that is needed for training the meta learner. Also, the table is reporting misleading results in the delta column, since it is computed against the original evaluation, and not w.r.t. the strongest attack against a particular defense.\nAlso, these techniques only adapt themselves on adversarial trained models, limiting the scope of the meta learner efficacy.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper introduces a "model agnostic" attack to evaluate the robustness
of machine learning models, specifically focusing on adversarial training
style defenses. The attack uses a RNN optimizer and meta-learning to achieve
this goal.

The reviews are mixed. The two more positive reviewers appreciated the
technical ideas of the attack, found the paper well written, and noted
the results were stronger than prior attacks.

However the more negative reviewer raises valid concerns around (a) the
magnitude of the contribution compared to prior attacks, and (b) to what
extent this attack will be useful more generally.

Starting with the first point (also raised by the other reviewers) the
total contribution of this paper is to improve attack success rates by
~0.1% compared to the best prior attacks. This is a fairly limited total
gain, especially because this technique requires much more sophisticated
attack techniques.

More fundamental is the question if this attack is useful to the community.
I tend to agree with reviewer 8AXD here that this contribution is rather
limited for two reasons:
1. As the authors acknowledge, the attack is most effective for adversarial
  training techniques, or others that don't make the gradient hard to
  optimize. This limits the attack to a smaller subset of defenses.
2. Complexity complicates attack evaluations. It's hard enough to get an
  attack working in the first place, and this paper has to use some fairly
  sophisticated tools to just get the attack marginally better than
  prior attacks that are (much) simpler. It's not clear that we would
  expect authors of future defenses to be able to get as good results,
  when prior methods are much simpler to apply.

And so on the whole, this paper's main contribution is improving attack
success rate by a small amount, with a fairly complex method, that only
applies to a class of defenses that don't make gradient descent difficult.
So while there is nothing outright wrong with this paper, in its current
form it seems to be adding unnecessary complexity to achieve something
that can already be done with existing techniques.
Decision: Reject