Learning to Generate Noise for Multi-Attack Robustness
Authors: Divyam Madaan
Jinwoo Shin
Sung Ju Hwang
Venue:  (2021)
Keywords: adversarial learning
robust machine learning
robust optimization
meta learning

Abstract: Adversarial learning has emerged as one of the successful techniques to circumvent the susceptibility of existing methods against adversarial perturbations. However, the majority of existing defense methods are tailored to defend against a single category of adversarial perturbation (e.g. $\ell_\infty$-attack). In safety-critical applications, this makes these methods extraneous as the attacker can adopt diverse adversaries to deceive the system. Moreover, training on multiple perturbations simultaneously significantly increases the computational overhead during training. To address these challenges, we propose a novel meta-learning framework that explicitly learns to generate noise to improve the model's robustness against multiple types of attacks. Its key component is Meta Noise Generator (MNG) that outputs optimal noise to stochastically perturb a given sample, such that it helps lower the error on diverse adversarial perturbations. By utilizing samples generated by MNG, we train a model by enforcing the label consistency across multiple perturbations. We validate the robustness of models trained by our scheme on various datasets and against a wide variety of perturbations, demonstrating that it significantly outperforms the baselines across multiple perturbations with a marginal computational cost.

Under review as a conference paper at ICLR 2021
LEARNING TO GENERATE NOISE FOR MULTI-ATTACK ROBUSTNESS
Anonymous authors Paper under double-blind review
ABSTRACT
Adversarial learning has emerged of the successful techniques circumvent the susceptibility of existing methods against adversarial perturbations. However, the majority of existing defense methods are tailored defend against single category of adversarial perturbation (e.g. lattackkk). In safety-critical applications, this makes these methods extrancous the attacker can adopt diverse adversaries to deceive the system. Moreover, training on multiple perturbations simultaneously significantly increases the computational overhead during training. To address these challenges, we propose novel meta-learning framework that explicitly learns to generate noise to improve the model's s robustness against multiple types of attacks. Its key component is Meta Noise Generator (MNG) that outputs optimal noise stochastically perturb a given sample, such that helps lower the error on diverse adversarial perturbations. By utilizing samples generated by MNG, model by enforcing the label consistency across multiple perturbations. We validate the robustness of models trained by our scheme on various datasets and against wide variety of perturbations, demonstrating that it significantly outperforms the baselines across multiple perturbations with a marginal computational cost.
train
INTRODUCTION
Deep neural networks have demonstrated enormous success on multiple benchmark applica- tions (Amodei et al., 2016; Devlin et 2018), by achieving super-human performance on certain tasks. However, to deploy them to safety-critical applications (Shen et al., 2017; Chen al., 2015; Mao al., 2019), we need to ensure that the model is robust well accurate, since incorrect predictions may lead to severe consequences. Notably, it is well-known that the existing neural networks are highly susceptible carefully crafted image perturbations which imperceptible to humans but derail the predictions these otherwise accurate networks.
The emergence of adversarial examples has received significant attention in the research community, and several defense mechanisms have been proposed (Madry et al., 2017; Zhang et al., 2019; Carmon al., 2019). However, despite large literature to improve upon the robustness of neural networks, most of the existing defenses leverage the knowledge of the adversaries and are based on the assumption of only single type of perturbation. Consequently, many of the proposed defenses were circumvented by stronger attacks (Athalye et al., 2018; Uesato et al., 2018; Tramer et al., 2020).
Meanwhile, several recent works have (Schott et al., 2018; Tramèr & Boneh, 2019) demonstrated the vulnerability of existing defense methods against multiple perturbations. For the desired multi- -tttik robustness, Tramèr & Boneh (2019); Maini et al. (2020) proposed various strategies aggregate multiple perturbations during training. However, training with multiple perturbations comes at an additional cost; increases the training cost by a factor of four over adversarial training, which already order of magnitude more costly than standard training. This slowdown factor hinders the research progress of robustness against multiple perturbations due to the large computation overhead incurred during training. Some recent works reduce this cost by reducing the complexity of generating adversarial examples (Shafahi et al., 2019; Wong et al., 2020), however, they are limited to lx adversarial training.
To address the drawbacks of existing methods, we propose a novel training scheme, Meta Noise Generator with Adversarial Consistency (MNG-AC), which leams instance-dependent noise to

---

Under review as a conference paper at ICLR 2021
Xclean
X eleaan
Dog
X'dv
Sample from
muitipple
X
aing
attacks
Dog
MNG 9s
Classifier fo
Adversarial Consistency Regularization
Figure 1: Overview of Meta-Noise Generator with Adversarial Consistency (MNG-AC). First, we stochastically sample perturbation to generate the adversarial examples X.avv, The generator Jo takes stochastic noise and input Xcllean to generate the noise- augmmened sample X=aug, The classifier fo then minimizes the stochastic adversarial classification loss and the adversarial consistency loss. MNG is learned via meta-learning explicitly minimize the adversarial classification loss.
minimize the adversarial loss across multiple perturbations while enforcing label consistency between them, as illustrated in Figure 1 and explained details below.
First, we tackle the heavy computational overhead incurred by multi-perturbation training by propos- ing Stochastic Adversarial Training (SAT), that samples from distribution of perturbations during training, which significantly accelerates training for multiple perturbations' Then, based on the assumption that the model should output the same predictions for different perturbations of the same image, introduce Adversarial Consistency (AC) loss that enforces label consistency across multiple perturbations. Finally, motivated by the noise regularization techniques (Huang et al., 2016; Srivastava al., 2014; Noh al., 2017; Lee al., 2020) which target generalization, Meta Noise Generator (MNG) that learns to stochastically perturb a given sample in a meta-learning framework explicitly improve the generalization and label consistency across multiple attacks. In particular, MNG-AC utilizes our generated samples to enforce label consistency across the generated samples from our model, adversarial samples, and clean samples. Consequently, pushes the decision boundary (see Figure 4) and enforces a smooth and robust network across multiple perturbations.
formulate
We validate the efficacy and efficiency of our proposed method by comparing against existing, state-of-the-an methods on CIFAR- 10. SVHN, and Ima-eImaeNNet dataset. The experimental results show that our method obtains significantly superior performance over the baseline methods trained with multiple perturbations, generalizes to diverse perturbations, and substantially reduces the computational cost incurred by training with multiple perturbations. In summary, the major contributions of this paper are as follows:
We introduce Adversarial Consistency (AC) loss that enforces label consistency across multiple perturbations enforce smooth and robust networks.
We formulate Meta-Noise Generator that explicitly meta-learns an input-dependent noise generator, such that it outputs stochastic noise distribution to improve the model's robustness and adversarial consistency across multiple types of adversarial perturbations.
We validate our proposed method on various datasets against diverse benchmark adversarial attacks, on which it achieves srate-of-the-aee performance, highlighting its practical impact.
2 RELATED WORK
Robustness against single adversarial perturbation. In the past few years, multiple defenses have been proposed to defend against a single type of attack (Madry et al., 2017: Xiao et al., 2020; Zhang et al., 2019; Carmon et al., 2019) and have been consequently circumvented by subsequent attacks (Athalye et al., 2018; Brendel et al., 2018; Tramer et al., 2020). Adversarial-training based
By factor of four single machine with four GeForce RTX 2080Ti CIFAR-10 1- and SVHN dataset. using Wide ResNet 28-10 (Zagoruyko Komodakis, 2016) architecture.
2

---

Under review as a conference paper at ICLR 2021
defenses (Madry et al., 2017; Zhang al., 2019; Carmon al., 2019) have been the only exceptions that have withstood the intense serutiny and have provided empirical gains in adversarial robustness.
Generative models for adversarial robustness. There have been various attempts that leverage the representative power of generative models to improve model robustness. Samangouei et al. (2018); Jalal et al. (2017) project image onto the generator manifold, which is then classified by the discriminator. Song et al. (2018) uses the sensitivity of generative models to defend against a single perturbation. Yin et al. (2020) proposed detection method based input space partitioning. However, Samangouei et al. (2018); Jalal et al. (2017); Song et al. (2018 8) were shown to be ineffective by stronger attacks (Carlini & Wagner, 2017; Athalye al., 2018). In contrast to learning the generative model to model the adversarial examples, we meta-learn the generator to explicitly learn an inpet-dependent optimal noise distribution to lower adversarial error across multiple perturbations, that does not necessarily correspond to any of the attack perturbations.
Robustness against multiple adversarial perturbations. Schott al. (2018) demonstrated that foc adversarial training is highly susceptible to fa/ mnoormm adversarial perturbations and used multiple VAES to defend against multiple perturbations on the MNIST dataset. However, was not scalable and limited to the MNIST dataset. Tramèr & Boneh (2019) investigated the theoretical/empiriaal trade-offs between multiple perturbations and introduced adversarial training with worst/average perturbations to defend against multiple perturbations. Maini et al. (2020) incorporated multiple perturbations into a single adversary to maximize the adversarial loss. However, computing all the perturbations is impractical for multiple perturbations and large scale datasets. On the other hand, our proposed framework overcomes this limitation, with improved performance over these methods and has a negligible increase in training cost over multi-perturbation adversarial training.
ROBUSTNESS AGAINST MULTIPLE PERTURBATIONS
We first briefly review single/mmuttipertuufee perturbation adversarial training and introduce Stochastic Ad- versarial Training (SAT) to reduce the computational cost incurred by training with multiple per- turbations. We consider dataset 'D over observations Let So Rdd - RCC be a L-layered classifier with parameters 0 and classification loss Lets- Given an attack procedure A(z) with norm-bail boll (2 :) around with radius æ for each example, which introduces a perturbation ò, we let zdvv = x + denote the corresponding adversarial examples. We consider the lp norm distance under the additive threat model (Laidlaw & Feizi, 2019) and adopt the mojjcted-gradieet
Rd and labels y RCC with C classes.
projected- gradient descent (PGD) (Madry et al., 2017) for crafting the lp perturbations:
adv T
(1+1)
proj (a(a))
adv (()
+ arg max o ande Leta (So (())
(2(0)) (1 adv
11 :
(1)
Iillla
where =-ddy is chosen at random within (2(x, e), ag is the step size, proj is the projection operator projecting the input onto the norm ball (A E), and ((+1) denotes the adversarial example the t-th PGD step. We will refer the approximation of the maximum loss by an attack procedure A(x), such that maX Leta (fe (I 2)
ady
Leta (fe (A (x)) y) for the rest of our paper.
Single-pertur bation adversarial training. In the standard pangll-perturbatoon adversarial train- ing (Kurakin al., 2016; Madry et al., 2017), the model optimizes the network using a min-mat formulation. More formally, the inner maximization generates the adversarial perturbation by maxi- mizing the loss, while the outer minimization minimizes the loss on the generated examples.
min E(Za))D Leta (Ja (A (x)) 0
(2)
The majority of existing single-penee pesullepttuurbation defenses are primarily able to defend against a single category of adversarial perturbation. However, this limits the generalization of these methods to perturbations that are unseen during training (Schott et al., 2018; Tramèr & Bonch, 2019), which has been referred as overfitting the partieular type of training perturbation.
Multi- nultiippetturbaion adversarial training. Tramèr Boneh (2019) extended the adversarial training to multiple perturbations by optimizing the outer objective in Eq. (2) the strongest/umion of adversarial perturbations for each input example. Their proposed strategies can more formally be defined as follows:
3

---

Under review as a conference paper at ICLR 2021
1. The maximum over all perturbations: It optimizes the outer objective in Eq. (2) on the strongest adversarial perturbation from the whole set of additive adversarial perturbations
min (a.))>> arg max; Lets (fo (As (a)) w)
(3)
2. The average over all perturbations: It optimizes the outer objective in Eq. (2) on the whole set of n additive perturbations.
kmn
min E (a.y)~ Lehe (fo (Ax (x),
(4)
Re
Recently, Maini et al. (2020) proposed "Multi Steepest Descent" (MSD) by incorporating the different perturbations into the direction of steepest descent. However, the practicality of all these methods is limited due to increased computational overhead for training.
Stochastic Adversarial Training (SAT). To overcome this limitation, we propose Stochastic Adver- sarial Training to defend against multiple adversarial perturbations. Specifically, conjecture that it essential to cover the threat model during training, not utilizing all the perturbations simultaneously. We formulate the threat model as a random attack A(x) sampled uniformly from perturbation set S during each episode (or batch) of training which prevents overfitting on a particular adversarial perturbation. In this work, we consider the ép-bounded perturbation set, and we sample the attack procedure A(r) with its corresponding norm-ball A from the perturbation set S as follows:
S = (An))))..AAA))))
4: Cat ((1/1,...,11n)) 1/n)),
(5) where Cat is the categorical distribution and is the number of attacks in the perturbation set S, Our proposed SAT optimizes the outer objective in Eq. (2) using the sampled attack procedure A(x) and a drastic simplification of the average one in Eq. (4), which makes it highly efficient for multiple perturbations. It is important note that unlike the average and max strategy SAT can be applied to any perturbation set with a constant cost and it promotes generalization and convergence (due to its stochasticity) by preventing over-fitting on a single type perturbation.
A(x) = St(r),
4 LEARNING TO GENERATE NOISE FOR MULTI-ATTACK ROBUSTNESS
In this section, we introduce our framework MNG-AC, which leverages adversarial consistency loss (AC) and mela-noise generator (MNG) to help the model generalize to multiple perturbations. Let 96 Rd + Rd denote the generator with parameters ø and zadv be the adversarial examples generated by SAT for uniformly sampled attack A(z) from a perturbation set with norm-bon BΛ(x,)) e). We sample generate the noise-augmented samples ang projected on the norm-ba ball (2 e). Note that, as MNG learns the noise to minimize the adversarial loss, it is essential to project the generated noise on the norm-bal nall [z =). which is the corresponding norm-ball of the sampled attack procedure A(xx). The total loss function Lrotal for the classifier consists exclusively of two terms: SAT classification loss and an adversarial consistency loss:
(0, I) for input our generator jointly with the clean examples x to
B
Liotal
Las (0 a adv (1) + + Lac (p (i): (i): adv (1): aug (i))
SAT classification loss
adversarial consistency loss
where B is the batch-size, 3 is the syper-parameet determining the strength of the AC loss denoted by faee and pleenn pydv paug represent the posterior distributions p(y x:lean), p(v a:ady), P.P()) (=
computed using the softmax function on the logits for clean ..d and ==g respectively. Specifically, Lee represents the Jensen -hsennSSannon Divergence (JSD) among the posterior distributions:
Lac (DKI
(prloan clean M) + DKI. || M) + (pi, M))
where M (pelean pady + pang) /3. Consequently, Lac enforces stability and insensitivity across a diverse range of inputs based on the assumption that the classifier should output similar predictions when fed perturbed versions of the same image.
4

---

Under review as a conference paper at ICLR 2021
Algorithm 1 Learning generate noise for multi-attack robustness
input Dataset D, T inner gradient steps, batch size B, perturbation set S.
output Final model paramaters 0
1: for (I....,N) ] do
2: Sample mini- bannhbbtcch of size B.
3: Sample an attack procedure A(x) with its corresponding norm-ball (a(z,)) using Eq. (5).
4: Generate the adversarial examples for A(x) using Eq. (1).
5: Sample (0, I) and generate 2 proj (x + 90(2, x)) using MNG, where B (2, Ba(z,)
is the norm-ball corresponding to the attack procedure sampled Step 3. Update 0 to minimize Eq. (6).
6:
7:
Initialize ((0) =
8:
fort (1..... . do
9: Update g(t) using Eq. (8).
10: end for
Descent a single step to update ((T) ((T+)) by Eq. (9).
12: Update the parameters & of the generator by Eq. (10).
13:
end for
Recently, Rusak et al. (2020) formulated an adversarial noise generator to learn the adversarial noise to improve the robustness on common corruptions. However, our goal is different; the robustness against multiple adversarial attacks is much more challenging task than that against common corruptions. To generate the augmented samples for our purpose, we explicitly perturb the input examples for generalization across multiple perturbations. In particular, MNG meta-learns (Thrun & Pratt, 1998; Finn et al., 2017) the parameters ø of a noise generator Jo to generate an input-dependent noise distribution alleviate the issue of generalization across multiple adversaries. The standard approach to train our adversarial classifier jointly with MNG is to use bi-level optimization (Finn et al., 2017). However, bi-level optimization for adversarial training would be computationally expensive.
To tackle this challenge, we adopt an online approximation (Ren et al., 2018; Jang et al., 2019) to update 0 and é using a single-optimization loop. We alternatively update the parameters 0 of the classifier with the parameters & of MNG. In particular, we first update the parameters using Eq. (6) (step Algorithm 1). Then, given current parameters 0 of our adversarial classifier, we update MNG parameters 6 using the following training scheme:
1. Update model parameters for T steps. First, we update 0 to minimize Les, 25 for T
steps which ensures the learning of the classifier using the knowledge from the generated samples constructed by MNG. It explicitly increases the influence of the ougse-au samples on the classifier in the inner loop. More specifically, for learning rate a, projection operator proj, (()) moves along the following descent direction on a mini-batch of training data:
1 B
((++1) 0(() - a B
VoLots (0(t) 1 (1))) (1))
1=1
(8)
where,
aug = proj (x + 2)(2,x)). B(x,5)
2. Adapt model parameters on single step. Second, perform one-step update to update ((T) to ((T+1) to minimize SAT loss from Eq. (6). This step explicitly models the adaptation of adversarial model parameters the presence of the noise-augmented data using single step of update:
( = TT)) a:
1 B
VoLels (g(T) adv (1))))))))
3. Update generator parameters. In the last step, after receiving feedback from the classifier, measure the SAT loss from Eq. (6) and adapt e to minimize this loss. In particular, & performs the following update step facilitate the classifier parameters 0 in the next step:
5

---

Under review as a conference paper at ICLR 2021
B B
1
V Celss (B(T+1)) 2 (.))()))
Φ p= - a
(10)
Overall, the generator minimizes the loss on the adversarial examples sampled from the perturbation set S. Consequently, á in Eq. (10) is dependent 0(++1) that depends on ((2) (see Eq. (9)), which in turn depends on & Aug (see Eq. (8)) and acts as a path for the flow of gradients. Similarly, the
gradients for @(T+1) are chained through the T steps since ((T+1) is dependent (CT) that depends on ((9) and we use TorchMeta Deleu et al. (2019) for the double backpropagation. We list the
detailed algorithm in Algorithm 1. Formally, the overall objective can be summarized as:
B (B(T+T)
min D B 1=1
Lcla
120
Lad (1) v())
B
subject to 0(T+1) = T) a
1
V Colels (B(T) (i)
B
(11)
((41) = g(t)
1
VoLcls (B()) 11 (1). (1),
B
0. 1
To summarize, MNG-AC consists of perturbation sampling to generate adversarial examples. Then, it perturbs the clean examples in a meta-learning framework to explicitly lower the adversarial classifica- tion loss the sampled perturbation. Lastly, the adversarial classifier utilizes the generated samples, adversarial samples and clean samples to optimize the classification and adversarial consistency loss.
Intuition behind our framework. Unlike existing adversarial defenses that aim for robustness against single perturbation, our proposed approach targets for realistic scenario of robustness against multiple perturbations. Our motivation that meta-learning the noise distribution to minimize the stochastic adversarial classification loss allows to learn the optimal noise to improve multi- perturbation generalization. Based on the assumption that the model should output similar predictions for perturbed versions of the same image, enforce the adversarial consistency loss, which enforces the label consistency across multiple perturbations. We empirically illustrate that our proposed training scheme increases the smoothness of the model (see Figure 3) and pushes the decision boundary (see Figure 4), which confirms our hypothesis for multi cnlti--erturbation generalization.
5 EXPERIMENTS
5.1 EXPERIMENTAL SETUP
Baselines and our model. We compare our method MNG-AC with the standard network (Nat) and state-of-the-and art single-pertubbation basclines including Madry et al. (2017) (Adv,) for foo, l,, and &z norm, Zhang et al. (2019) (TRADES.>). and Carmon et al. (2019) (RST for foo norm. We also consider satte-of-the-asd art multi culli-perturbationn baselines: namely, we consider Adversarial training with the maximum (see Eq. (3)) (Advmax)) average (Advavg) (Tramèr & Boneh, 2019) (see Eq. (4)) over all perturbations, and Multiple steepest descent (MSD) (Maini al., 2020).
Datasets. We evaluate our method on multiple benchmark datasets including CIFAR-10 (Krizhevsky, 2012), SVHN (Netzer et al., 2011 1) on Wide ResNet 28-10 (Zagoruyko & Komodakis, 2016) and Timy-ImageNet ResNet-50 (He et al., 2016) architecture.
Evaluation setup. We have evaluated the proposed defense scheme and baselines against perturba- tions generated by state-of-the-art attack methods. We use the same attack parameters Tramer & Bonch (2019) for training and evaluation. We validate the clean accuracy (Accelesn): the worst (Acetnion) and average (A)) adversarial accuracy across all the perturbation sets for all the models. For lo attacks, we use PGD (Madry et al., 2017), Brendel and Bethge (Brendel et al., 2019),
httpps:/tinn-- weciiiny--mmggneetee magenet www..mtt.hherrkkp.cooom

---

Under review as a conference paper at ICLR 2021
Table I: Comparison of robustness against multiple types of perturbations. All the values are measured by computing mean, and standard deviation across three trials upon randomly chosen seeds, the best and second-best results highlighted in bold and underline respectively. Time denotes the training 80 and a = {0.004, 1.0,
time hours. For CIFAR-10 and SVHN, we use & = 265 255
2000
255
for Eox, f, and l2 attacks respectively. For Tiny- Tiny-ImageNet, we use = { 255 255 255
2000 80
and
{0.004, 1.0, 0.13 for loc é,, and le attacks respectively. We report the worst-case accuracy for all the attacks and defer the breakdown of all attacks to Appendix B.
Model
AcCeleam
80
4
f2
Ade
Acc
Time (h)
Nat (Zagorayko Komodakis, 2016)
94.7=
0.0:
4.4
19.44
0.0
7.94
0.4
Advoc (Madry et at., 2017)
86. 06.8=10 0.1
07
12.8=
64
12.9
42.6
4.5
Adv;
93.3 0.4
0.0
78.1
0.0: 0.0
0.0+ 0.00
25.1 1.6
8.1
CIFAR-10 -FAR--0
Adv2
91.7 0.2
20.74 0.3
27.7= 0.7
76.8 0.4
17.9 0.8
47.61
3.7
TRADES (Zhing 2019)
84.7 0.3
48
17.9= 0.6
03
17.2 0.6
45.4
5.2
RST No (Carmon et. 2019)
88.9 0.2
54.94 1.8
22.0= 0.5
73.6+
21.1 L.0
50.2 0.5
58.8
AdVag (Trmmrr & Bonch, 2019)
87.1 0.2
33.8: 0.7
49.0= 0.3
74.9+
31.0+ 1.4
52.6 0.5
16.9
AdVius (Tmmbr& & Bonch, 2019)
85.4 0.3
39.9 0,9
44.6= 0.2
73.2+ 0.2
35.7 0.3
52.5+ 0.3
16.3
MSD (Maini al., 2020)
82.6 0.0
43.7+ 0.2
41.6= 0.2
70.61
35.84 0.1
52.0# 0.4
16.7
MNG-AC (Ours)
81.5 0.3
42.24 0.9
55.0= 1.2
71.5 0.1
41.6 0.8
56.2 0.2
11.2
Nat (Zagorykko Komodakis,
96.8: 0.1
0.0
4,4+
19.4
0.0
7.9
0.6
Adv (Madry et 2017)
92.8#
46.24
3.0+
59.24
3.3
36.2+ 0.3
6.2
Advi
92.4= 0.9
0.0+
77.9
0.0=
0.0
23,9
11.8
SVHN
Advz
94.9: 0.1
18.7
30.3= 0:3
79.34 0.1
0.7
42.8
6.1
TRADES (Zhang et ol,, 2019)
93.9 0.1
49.9+ 1.7
1.6 0.3
56.0+ 14
1.6
35.8 0.6
7.9
RST (Camon et. 2019)
95.6 0.0
60.9 2.0
0,74 0.6
60.611 0.6
0.7
40.7
112.5
AdVas (Tambr Boneh, 2019)
92.6 0.3
17.4: 2.3
54.2= 2.9
74.74 0.1
16.6 1.3
43.0 1.0
24.1
Advaas (Tnmbr & Bonch, 2019)
88.2 1,3
5.9 1,2
48.3=4.1
31.0d s.0
5.8 1.7
26.7+ 2.5
22.7
MNG-AC (Ours)
93.7 0.1
33.74
47.4=
77.6
30.3:
52.61:
11.9
Nat (He et al., 2016)
62.8 0.4
0.0 0.0
2.7+ 0.3
12.6 088
0.0 0.0
5.1 0.4
0.9
Tiny ImageNet
Adv (Madry et 2017)
54.2 0.1
29.6:t 0.1
31.8=
42.51
19.8 1.1
33.81: 0.1
4.3
Advi
$7.8 0.2
00
39.3=
41.9+
10.1 0.7
30.4
12.9
Adva
59.5 0.1
5.2+
37.2=
44.9
5.2+
29.1
3.7
TRADES (Zhang et al., 2019)
48,2 0.2
28.74 0.9
30.9= 0.2
35.81 0.7
26.1 0.9
32.81: 0.1
5.8
Adv (Trmmr Bonch, 2019)
56.0#: 0.0
23.74 0.2
38.0=
44.6+ 1.8
23.6 0.3
35.4 0.7
26.8
Advau: (Tamir 2019)
53.5
29.8:1
33.4=
42.4:t
29.0::
35.3
20.8
MNG-AC (Ours)
53.1 0.3
27.4:1 0.7
39.6 0.7
44.84
27.4 0.8
37.2 0.6
10.4
Carlini (Carlini & Wagner, 2017), PGD (Madry et al., 2017), Brendel and Bethge (Brendel et al., 2019), and AutoAttack (Croce & Hein, 2020). For & attacks, we SLIDE (Tramer & Boneh, 2019), Salt and pepper (Rauber et al., 2017), and EAD attack (Chen et al., 2018). We provide a detailed description of the experimental setup Appendix A.
and AutoAttack (Croce & Hein, 2020). For E2 attacks,
5.2 COMPARISON OF ROBUSTNESS AGAINST MULTIPLE PERTURBATIONS
Results with CIFAR-10 dataset. Table shows the experimental results for the CIFAR-10 dataset. is evident from the results that MNG-AC achieves relative improvement of ~ 6% and ~ 4% on the Acctnionn and Acg metric
the state-of-the-an methods trained on multiple perturbations. Moreover, MNG-AC achieves ~ 33% reduction in training time compared to the multi-perturbations training baselines. It is also worth mentioning that, MNG-AC also shows improvement over Advmax, which is fundamentally designed to address the worst perturbation.
Results with SVHN dataset. The results for the SVHN dataset are shown in Table We make the following observations from the results: (1) Firstly, MNG-AC significantly outperforms AdVavg Adymax by ~ 14% and ~ 25% on Accunion metric. Furthermore, it achieves an improvement of ~ 7.2% and 26% on Accagg adv metric over Adv Adymax respectively. (2) Secondly, MNG-AC leads to 50% reduction training time compared to the nurti-ppetturbation training baselines.
7

---

Under review as a conference paper at ICLR 2021
Table 2: Ablation study analyzing the significance of SAT, Adversarial Consistency loss (AC) and Meta Noisc
Generator (MNG). The best results highlighted in bold.
SAT
AC
MNG
AcCcloan
t0
11
82
Acc
Acc
Time (h)
CIFARR 10
87.4.1 0.0
34.61 0.7
49.3 1.0
75.5 0.1
33.9. 0.6
53.1 0.1
5.5
/
81.4+ 0.0
40.4= 0.1
53.2+ 0.9
70.2 0.1
40.1
54.6# 0.4
6.8
v
v
81.5+ 0.3
42.2 0.9
55.0+ 1.2
71,5
0.8
56,23 0.2
11.2
SVHN
92.8+ 0.5
23.4 2.4
41.34 4.3
71.0=
22.8+ 1.5
44.9 1.2
7.6
92.1 0.2
32.9 1.8
35,41: 1.5
77.1
28.31 0.1
49.6.: 0.5
9.6
93.7.1 0.1
35.1
47.4
77.6 J.
30.31 1.8
52.6.1 0.5
11.9
Interestingly. MNG-AC achieves significant better performance over & adversarial training with comparable training time which illustrates the utility of our method over standard adversarial training.
Results with Tiny- ImageNe.. We also evaluate our method on Tiny-ImageNet to verify that it performs well on complex datasets. In Table we observe that MNG-AC outperforms the multi- perturbation training baselines and achieves comparable performance to the baselines. Only against for perturbations, we notice that Adv achieves better performance. We believe this is an artefact of the inherent trade-off across multiple perturbations (Tramèr & Boneh, 2019; Schott et al., 2018). Interestingly, MNG-AC even achieves comparable performance to the single perturbation baselines trained on é and & norm. This demonstrates the effectiveness of MNG in preventing overfitting over a single attack, and it's generalization ability to diverse types of attacks.
5.3 ABLATION STUDIES
Component analysis. To further investigate our training scheme, we dissect the effectiveness of various components in Table First, we examine that SAT leads to a ~ 68% and 30% reduction in training time over multiple perturbations basclines and MNG-AC for both the datasets, however, it does not improve the adversarial robustness. Then, we analyze the impact of our meta-noise generator by injecting random noise (0, I) the inputs for the generation of augmented samples. We observe that it significantly improves the performance over the SAT with a marginal increase in the training time. Furthermore, leveraging MNG our combined framework MNG-AC achieves consistent improvements, outperforming all the baselines, demonstrating the efficacy of our meta-learning scheme to defend against multiple perturbations.
Effect of hyperparameters. We further analyze the impact of 3 in our augmentation loss (see Eq. (6)) in Figure We evaluate the worst-attack performance across all l, norm adversarial attacks. Our results show that as the value of / increases the performance on lo and & attacks improves particular, the performance with foe and & attack improve by 4% with an increase in the weight of adversarial consistency loss. However, an increase in 39 also leads to reduction of ~ 3% the robustness against the & attacks, which is in line with the previous works that have showcased an inherent trade-off between various attacks theoretically and empirically (Tramèr & Bonch, 2019; Schott et al., 2018).
significantly.
5.4 FURTHER ANALYSIS OF OUR DEFENSE
Results on unforseen adversaries. We further evaluate our model on various unforeseen perturba- tions (Kang et al., 2019) namely we evaluate on the Elastic, &x -IPEG -JPEG and 22-JPEG attacks. Note that, even though adversarial training methods do not generalize beyond the threat model, we observe that MNG-VS improves the performance on these adversaries. We compare MNG3AT to the baselines trained with multiple perturbations notice that even though, Advmax achieves better performance on JPEJ attacks, it obtains the generaaizss better over both
the SVHN dataset in Table 3. We
minimum robustness across the Acc metric. In contrast, MNG-AC generalizes the baselines for the worst-attack in the set of unforeseen perturbations.
Visualization of loss landscape. As further qualitative analysis of the effect of MNG-AC, we compare the loss surface of various methods against foo, &1, and &g norm attack in Figure 3. We can observe that in most of the instances when trained with single adversary. the adversary can find a direction orthogonal to that explored during training: for example, attack results in non-smooth
8

---

Under review as a conference paper at ICLR 2021
Table 3: Performance of MNG-AC against unforseen adversaries on SVHN dataset.
Ours
Model
Elastic
GPEG
4,JPEG
4-JPEG
Accusion
Advass
77.1
86.12:15
78.1s
79.0= 20
61.51
AdVrus
89.9
87.9
87.0
58.5
MNG-AC
80.8
87.7%
76.61
81.4
64.3:
Eno attack
6 attack
fo attack
Adv. acc. (86)
813
Esp
Effect of
Effect of
Effect of
6.0
880
Figure 2: Ablation study on the impact of Lang on average robustness against ip attacks 100. With an increase B, the robustness against to and l attack increases. However, the robustness of & decreases showing an inherent trade-off across multiple perturbations.
Figure 3: Visualization of the loss landscapes for the &1, l2, and -orm attacks on the SVHN dataset. The rows represent the attacks and columns represent different defenses.
clean
outs
Figure 4: Visualization of decision boundary in the penultimate latent-feature space for AdVavg avg in the left, Adv max the middle, MNG-AC in the right for SVHN dataset on Wide ResNet 28-10 architecture. The shapes represent different classes binary classification task.
loss surface for both Lo and & adversarial training. On the contrary, MNG-AC achieves smoother loss surface across all types of attacks which suggests that the gradients modelled by our model are closer the optimum global landscape. See Appendix for the loss landscape on CIFAR-I0.
Visualization of decision boundary. Finally, we visualize the learned decision boundary on binary- classification task across multiple attacks in Figure We can observe that MNG-AC obtains the least error against all the attacks compared to the baselines trained on multiple perturbations. Furthermore, the consistency regularization embeds multiple perturbations onto the same latent space, which pushes them away from the decision boundary that in turn improves the overall robustness, See Appendix B for visualization of the examples generated by our proposed meta-noise generator.
CONCLUSION
We tackled the problem of robustness against multiple adversarial perturbations. Existing defense methods are tailored to defend against single adversarial perturbation which is an artificial setting to evaluate in real-life scenarios where the adversary will attack the system in any way possible. To this end, we propose novel Meta-Noise Generator (MNG) that learns to stochastically perturb adversarial examples by generating output noise across diverse perturbations. Then we train the model using Adversarial Consistency loss that accounts for label consistency across clean, adversarial, and augmented samples. Additionally, to resolve the problem of computation overhead with conventional adversarial training methods for multiple perturbations, Training (SAT) which samples a perturbation from the distribution of perturbations. We believe that method can be strong guideline when other researchers pursue similar tasks in the future.
introduce Stochastic Adversarial

---

Under review as a conference paper at ICLR 2021
REFERENCES
Dario Amodei, Sundaram Ananthanarayanan, Rishita Anubhai, Jingliang Bai, Eric Battenberg, Carl Case, Jared Casper, Bryan Catanzaro, Qiang Cheng, Guoliang Chen, al. Deep speech End-to-end speech recognition in english and mandarin. In ICML, 2016.
Anish Athalye, Nicholas Carlini, and David A. Wagner. Obfuscated gradients give false sense of security: Circumventing defenses to adversarial examples. In ICML, 2018.
Wieland Brendel, Jonas Rauber, and Matthias Bethge. Decision-based adversarial attacks: Reliable attacks against black-box machine learning models. In ICLR, 2018.
Wieland Brendel, Jonas Rauber, Matthias Kümmerer, Ivan Ustyuzhaninov, and Matthias Bethge. Accurate, reliable and fast robustness evaluation. In NeurlPS, 2019.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp). 2017.
Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, John C Duchi, and Percy Liang. Unlabeled data improves adversarial robustness. In NeurlPS, 2019.
Chenyi Chen, Ari Seff, Alain Komhauser, and Jianxiong Xiao. Deepdriving: Learning affordance for direct perception in autonomous driving. In ICCV, 2015.
Pin-Yu Chen, Yash Sharma, Huan Zhang, Jinfeng Yi, and Cho-Jui Hsieh. Ead: Elastic-net attacks to deep neural networks via adversarial examples. In AAAI, 2018.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter- free attacks. In ICML, 2020.
Tristan Deleu, Tobias Würfl, Mandana Samiei, Joseph Paul Cohen, and Yoshua Bengio. Torchmeta: A meta-learning library for pytorch. arXiv preprint arXiv:1909.06577 2019.
Jacob Devlin, Ming-Wos Weig Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.0480) 2018.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In ICML., 2017.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings deep residual networks. In
Gao Huang, Yu Sun, Zhuang Liu, Daniel Sedra, and Kilian Q Weinberger. Deep networks with stochastic depth. In ECCV,2015
Ajil Jalal, Andrew Ilyas, Constantinos Daskalakis, and Alexandros G Dimakis. The robust manifold defense: Adversarial training using generative models. arXiv preprint arXiv: 2017.
Yunhun Jang, Hankook Lee, Sung Ju Hwang, and Jinwoo Shin. Learning what and where transfer. In ICML, 2019.
Daniel Kang, Yi Sun, Dan Hendrycks, Tom Brown, and Jacob Steinhardt. Testing robustness against unforeseen adversaries. arXiv preprint arXiv: 08016, 2019.
Alex Krizhevsky. Learning multiple layers of features from tiny images. University of Toronto, 05 2012.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial examples the physical world. arXiv preprint arXiv:1607.025:: 2016.
Cassidy Laidlaw and Soheil Feizi. Functional adversarial attacks. In NeurIPS, 2019.
Hae Beom Lee, Taewook Nam. Eunho Yang. and Sung Ju Hwang. Meta dropout: Learning to perturb latent features for generalization. In ICL., 2020.
10

---

Under review as a conference paper at ICLR 2021
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In ICLR, 2017.
Pratyush Maini, Eric Wong, and J Zico Kolter. Adversarial robustness against the union of multiple perturbation models. In ICML, 2020.
Chengzhi Mao, Ziyuan Zhong, Junfeng Yang, Carl Vondrick, and Baishakhi Ray. Metric learning for adversarial robustness. In AAAI, 2019.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. In Workshop on Deep Learning and Unsupervised Feature Learning, NeurIPS, 2011.
Hyconwoo Noh, Tackgeun You, Jonghwan Mun, and Bohyung Han. Regularizing deep neural networks by noise: Its interpretation and optimization. In NeurIPS, 2017.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, higg--peffommance deep learning library. In NeurIPS, 2019.
Jonas Rauber, Wieland Brendel, and Matthias Bethge. Foolbox: python toolbox to benchmark the robustness of machine learning models. In Reliable Machine Learning in the Wild Workshop, ICML, 2017.
Mengye Ren, Wenyuan Zeng. Bin Yang, and Raquel Urtasun. Learning to reweight examples for robust deep learning. In ICML, 2018.
Evgenia Rusak, Lukas Schott, Roland Zimmermann, Julian Bitterwolf, Oliver Bringmann, Matthias Bethge, and Wieland Brendel. A simple way to make neural networks robust against diverse image corruptions. ECCV, 2020.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, al. Imagenet large scale visual recognition challenge. International journal of computer vision, 2015.
Pouya Samangouei, Maya Kabkab, and Rama Chellappa. Defense-GAN: Protecting classifiers against adversarial attacks using generative models. In ICLR, 2018.
Lukas Schott, Jonas Rauber, Matthias Bethge, and Wieland Brendel. Towards the first adversarially robust neural network model on mnist. In ICLR, 2018.
Ali Shafahi, Mahyar Najibi, Mohammad Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry S Davis, Gavin Taylor, and Tom Goldstein. Adversarial training for free! In NeurIPS, 2019.
Dinggang Shen, Guorong Wu, and Heung-Il Suk. Deep learning in medical image analysis. Annual review of biomedical engineering, 2017.
Yang Song, Taesup Kim, Sebastian Nowozin, Stefano Ermon, and Nate Kushman. Pixeldefend: Leveraging generative models to understand and defend against adversarial examples. In ICLR, 2018.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, llya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. JMLR, 2014.
Sebastian Thrun and Lorien Pratt (eds.). Learning to Learn. Kluwer Academic Publishers, 1998.
Florian Tramèr and Dan Boneh. Adversarial training and robustness for multiple perturbations. In NeurIPS, 2019,
Florian Tramer, Nicholas Carlini, Wieland Brendel, and Aleksander Madry. On adaptive attacks to adversarial example defenses. In NeurIPS, 2020.
Jonathan Uesato, Brendan O' DDnoohuue, Aaron van den Oord, and Pushmeet Kohli. Adversarial risk and the dangers of evaluating against weak attacks. arXiv preprint arXiv: 1802.0566,, 2018.
11

---

Under review as a conference paper at ICLR 2021
Eric Wong, Leslie Rice, and J. Zico Kolter. Fast is better than free: Revisiting adversarial training. In ICLR, 2020.
Chang Xiao, Peilin Zhong, and Changxi Zheng. Enhancing adversarial defense by k-winners-take-all. In ICLR, 2020.
Xuwang Yin, Soheil Kolouri, and Gustavo Rohde. adversarial example detection and robust classification. In ICLR, 2020.
Gat: Generative adversarial training for
Sergey Zagonuyko and Nikos Komodakis. Wide residual networks. In British Machine Vision Conference, 2016.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P. Xing, Laurent El Ghaoui, and Michael I. Jordan. Theoretically principled trade-off between robustness and accuracy. In ICMI., 2019.
12

---

Under review as a conference paper at ICLR 2021
A EXPERIMENTAL SETUP
A.1 DATASETS
CIFAR-IO. This dataset (Krizhevsky, 2012) contains 60,000 images with 5,000 images for training and 1,000 images for test for each class. Each image is sized 32 x 32, we use the Wide ResNet 28-10 architecture (Zagoruyko & Komodakis, 2016) as a base network for this dataset.
2. SVHN. This dataset (Netzer et al., 2011) contains 73257 training and 26032 testing images of digits and numbers in natural scene images containing ten-digit classes. Each image is sized 32 x 32, we use the Wide ResNet 28- 10 architecture similar to the CIFAR-10 dataset as the base network.
3. Nit This dataset is a subset of ImageNet (Russakovsky et al., 2015) dataset, consisting of 500, 50, and 50 images for training, validation, and test dataset, respectively. This dataset contains 64 64 size images from 200 classes, we use ResNet50 (He et al., 2016) as a base network for this dataset.
A.2 TRAINING SETUP
We use the SGD optimizer with momentum 0.9 and weight decay 5 10-4 to train all our models with cyclic learning rate with maximum learning rate À that increases linearly from 0 to À over first N/2 epochs and then decreases linearly from N/2 to O in the remainder epochs, as recommended by Wong al. (2020) for fast convergence of adversarial training. We train all the models for 30 epochs on single machine with four GeForce RTX 2080Ti using WideResNet 28- 10 architecture (Zagoruyko & Komodakis, 2016). We use the maximum learning rate of = 0.21 for all our experiments. We use 8 = 16 for all the experiments with our meta noise generator. The generator is formulated as a convolutional network with four 3x3 convolutional layers with LeakyReLU activations and one residual connection from input to output. We use T = 2 for all our experiments and all our algorithms are implemented using Pytorch (Paszke et al., 2019) and TorchMeta (Deleu et al., 2019). We use the weight for the KL divergence (3- 6.0) for TRADES and RST in all our experiments. We replicate all the baselines on SVHN and Tiny ImageNet since most of the baseline methods have reported found that MSD Maini et al. (2020) did We believe that
their results on MNIST and CIFAR-10. Unfortunately,
not converge for larger datasets even after our extensive dyperparameter-seant this is due to the the change in formulation of the inner optimization which leads difficulty in convergence for larger datasets. Since the authors also report their results on CIFAR-10, 10, we do not use as baseline for other datasets.
A.3 EVALUATION SETUP
For foo perturbations, we PGD) (Madry et al., 2017), Brendel and Bethge attack (Brendel et al., 2019), and AutoAttack (Croce & Hein, 2020). For & perturbations, we use CarliniWagner attack (Carlini & Wagner, 2017), PGD (Madry al., 2017), Brendel and Bethge attack (Brendel et al., 2019), and AutoAttack (Croce & Hein, 2020). For & perturbations, we use SLIDE (Tramèr & Boneh, 2019), Salt and pepper (Rauber et al., 2017), and EAD attack (Chen et al., 2018). For CIFAR- 10 and SVHN, we use {
8 2000 80
and a = =0.004, 1.0, 0.1} for fx e,, and & 1 2000 8o } and a f0.004, 1.0, 0.1} } for
255 255
255
attacks respectively. For Emny-ImaeeNer we use
255
255
25
lx, f1, and l attacks respectively. We use 10 steps of PGD attack for fxc lz during training. For l
adversarial training, use 20 steps during training and 100 steps during evaluation. We use the code provided by the authors for evaluation against AutoAttack Croce & Hein (2020) and Foolbox (Rauber al., 2017) library for all the other attacks.
B MORE EXPERIMENTAL RESULTS
Due to the length limit of our paper, we provide breakdown of all the attacks on CIFAR-I0 in Table 4, SVHN on Wide ResNet 28-10 18-10 in Table 5, Tiny-ImageNet on ResNet50 in Table 6. Besides, we analyze the noise learned by our meta-learning framework on multiple datasets and the loss landscape on the CIFAR-10 10 dataset.
"https://ting" bin--iiniiin magenet wwageeettheroouaap.... www.hhapp..com
13

---

Under review as a conference paper at ICLR 2021
Table 4: Summary of adversarial accuracy results for CIFAR-IO on Wide ResNet 28-10 architecture.
od
Adv1
Adv2
Trades
RST
AdVaz
Adymas
MSD
MNG-AC
Clean Accuracy
86.81 0.1 93.3 0.6 91.7. 02 84.71 0.3 88.9 87.11 0.2 85.41 0.3 82.31 0.2
84.91 0.3
PGD-&
46.9
0.40 0.7 23.64 0.2 52.0.: 0.6 $6.9 0.1 35.2 0.8 42.2# 1.1 45.41 0.4
44.5d 1.1
(GD-Foolbox
54.7=
0.33+
35.3+ 044 57.8 62.9
45.0=
50.4+ 04 51.7 088
50.8 088
AutoAttack
44.9
0.0=
20.7= 0.4 48.8 53.9#
33.8= 0.7 39.9 099 42.7
42.8+ 08
Brendel & Bethge
49.9 111
0.0 0.0 26.81 0.3 52.11 0.7 56.51 18 39.61 0.7 45.81 09 48.3 0.4
46.8..99
All Lx attacks
44.9: 0.7 0.0: v.0 20.7 0.3 48.9 0.7 54.9: 33.8 0.7 39.9: 0.9 43.7+ 022
42.2+ 0.9
PGD-61
12.8# 0.6 91.6# 14 27.7= 0.7 17.9 0.6 22.0= 0.5 49.0= 0.3 44.6# 0.2 46.8 1.4 55.0+ 1.2
PGD-Foolbox
35.2 0.7 92.31 13 53.11 0.5 40.3 0.7 44.61 0.3 64.51 0.2 60.7 0.5 60.31 0.4 65.51 n.1
EAD
72.9110 87.1 3.3 75.9= 19 80.2 0.7 84.5# 02 85.7= 02 83.3# 0.5 80.8 0.1 79.3 0.6
SAPA
71.5 0.2 80.2 188 81.9 0.5 71.4 0.7 76.0 0.5 82.7 80.0 0.1 76.9
76.7:
All attacks
12.8 0.6 78.1 18 27.7= 0.7 17.9 0.6 22.0= 0.5 49.0= 0.3 44.6# 0.2 43.7 02
55.0+ 1.2
PGD-&2
78.7=
47.6 84.6= 022 77.0= 09 82.2# 81.5=
79
75.6 0. 04
PGD-Foolbox
74.6=
2.1
79.8 02 06 78.3+
77.6 0. 03
73.4+
Gaussian Noise
85.2
88.5=
90.5= 11 83.2# 87.8=
86.2# 83.3=
70.9
79.3+
AutoAttack
69.91 0.4 0.0. 0.
76.81 044 69.41 0.3 73.71 74.91 04 73.2100 71.91
71.51
Brendel & Bethge
71.8 0.9
0.0 78.1 066 0.1 75.0= 0.3 75.9= 74.1
04
72.3
CWL2
70.5 02 0.1 77.2= 0. 0. 74.2=
0.1 74.6= 12 73.5= 71.1 11
71.0
All l attacks
69.3#
0.0= 76.8= 0.4 69.4 0.3 73.6 74.9 0.4 73.2= 022 70.6 1.1 71.5+ 0.1
upion
12.9 0.5 0.0 0.0
17.9= 088 17.2 0.6 21.1 1.0 31.0= 1.4 35.7 0.3 35.8 0.1
41.6 0.8
Acc avg
42.6# 25.1 1.6 47.6# 0.4 45.4 0.3 50.2+ $2.6 0.5 52.5+ 0.3 52.0+ 0.4
56.2+ 0.2
Visualization of learned noise. To demonstrate the learning ability of our meta-noise generator, we visualize the learned noise by our generator during training. We present representative samples projected various L, norms and datasets in Figure 5 where each sample is projected to their respected norm-ball B(x,E) around x with radius e. From the figure, we can observe that our meta- noise generator incorporates the features by different attacks and learns diverse input-dependent noise distributions across multiple adversarial perturbations by explicitly minimizing the adversarial loss across multiple perturbations during meta-training. Overall, it combines two approaches that complementary to each other and leads to a novel input-dependent learner for generalization across diverse attacks.
Visuaization of loss landscape CIFAR-10. Figure 6 shows the visualization of loss landscape of various methods against foo, E, and l2 norm attack for CIFAR-10 dataset on Wide ResNet 28-10 10 architecture. We vary the input along linear space defined by the norm of the gradient where and y-axes represent the perturbation added in each direction, and the z-axis represents the loss. Similar to the SVHN dataset, we can observe that the loss is highly curved for multiple perturbations in the vicinity of the data point & for the adversarial training trained with a single perturbation, which reflects that the gradient poorly models the global landscape. In contrast, MNG-AC achieves smoother loss surface across all types of bp norm attacks.
14

---

Under review as a conference paper at ICLR 2021
Table 5: Summary of adversarial accuracy results for SVHN dataset on Wide ResNet 28- 10 architec-
ture.
od
Adv1
Adv2
Trades
RST
AdVavz
Advmax
MNG-AC
Clean Accuracy
92.8=
92.4# 1.6
94.9= 0.0 93.9 00 95.6# 0.0 92.6# 0.1 88.2 93.44 0.0
PGD-f
0.1
3.2+ 2.4
29.4 0.1 55.5 1.4 66.9# 088 22.4 3.1 36.6# 2.0
40.5 0.1
PGD-Foolbox
6..7
2.5 1.9
47.6 0.6 66.41 1.1 73.8 0.3 32.5 32
49.9 0.0
57.54 188
AutoAttack
0..2=066
0.0+
18.9 0.5 49.9 61.0= 2.0 17.6 2.6
17.5
33.7+ 18
Brendel & Bethge
51.6:
0.0
22.91
55.8
65.61 1.2 20.21
6.3
40.01
All foc attacks
46.2 0.6 0.0 oo
18.7 0.6 49.9 1.7
60.9 2.0 17.4 2.3
5.94 1.2
33.74 19
PGD-L1
3.1 95.0=
30.5 0.4
1.7 0.7 0.6 55.8
48.4
44.5+
PGD-Foolbox
19.9 088 94.61 0.4 57.51 0.1 15.51 0. 11.31 0. 79.2 344 85.41 3.2
75.2:1 2.8
EAD
6. 2.1 87.8= 19 82.3# 1.2
51.5# 29 60.4 08 84.8 2.4
84.5 388
86.2+ 2.2
SAPA
79.4 0.8 77.3 s2 87.3+ 0.1 73.5 1o 86.2+ 0.5 88.5 0.6 80.9 4.0
89.94 1.6
All & attacks
3.0 77.9= 6.3 30.3= 0.3 1.6 0.3 0.7 54.2
48.3
47.4 2.2
PGD-E,
81.6 0.5
3.9 1.4 87.8= 0.2 83.9 088 85.3 0.2 85.6 066 84.3 1.1
90.4 0.6
PGD-Foolbox
73.2+ 0.2
1,9
82.8# 0.6 75.0= 077 76.0= 0.3 80.6 0.1 60.1
86.1 0.1
Gaussian Noise
92.1
16.5 4.2
94.2+
93.3 94.2 0.6 92.21 02 83.8
93.2.
AutoAttack
59.0.
0.0.
79.31 0.1 56.41 60.7 0.6 75.61 40.0
78.0.1
Brendel & Bethge
68.2 0,5
0.0 0.0
81.0 0.1 64.8 09 68.1 18.1= 0.5 76.4 044 32.7 38
78.44 0.4
CWL2
63.5 088
0.1 0.1
80.1 1.4 61.4= 0.3 63.9 0.2 76.8 0.1 55.3+ 5.2
80.9+ 0.9
All E2 attacks
59.2= 0.7 0.0#
79.3 0.1 56.0# 60.6 0.6 74.7
31.0#
77.64 1.0
Ace union
3.0 0.3
0.0 0.0
16.4 0.7 1.6 0.3
0.7 0.6 16.6# 13 5.8 1.7
30.3+ 1.8
Accavg avg
adl
36.2+ 0.3
23.9= 2.1
42.8
0.2
35.8 0.6
40.7 0.8 43.0
1. 26.7 25
52.6 0.5
15

---

Under review as a conference paper at ICLR 2021
Table 6: Summary of adversarial accuracy results for Imageet on ResNet50 architecture.
Advoc
Adv1
Adv2
Tradesso
AdVarg
Advma
MNG-AC
Clean Accuracy
54.21 0.1 57.81 0.2 59.81 0.1 48.21 0.2 56.01 0.2 53.51 0. 53.1 0.1
PGD-fco
32.11 0.0 11.51 12 17.94
32.2+ 0.4 23.01 0.6 32.0.:
29.3 0.3
PGD-Foolbox
34.6# 0.4 17.2+ 5.2=
34.1# 02 34.0# 022 28.3+
32.3# 0.3
AutoAttack
29.61: 10.11.0. 16.31
28.74 09 23.71 0.2 30.01
27.7a 04
Brendel & Bethge
32.7
14.6# 0.8 20.8 0.6
31.0# 0.9 28.1 0.2 33.2 0.5
31.5= 0.6
Lop attacks
29.6: 0.1 10.3 0.2 28.7 0.9 23.7 0.2 29.8 0.1 27.4 0.7
PGD-&
32.0 1.1 39.31 09 37.21 02 31.11 0. 38.01 0.1 33.61 0.4 39.0 09
PGD-Foolbox
40.0 0.8 44.8= 02 45 37.6
0.9 44.7= 1.5 40.6 0.1 45.0 0.2
EAD
52.3# 1.5 56.31 0.6 $7.3+ 0. 46.7+ 09 54.61 09 51.21 022 52.7 0.3
SAPA
46.5 0.9 52.9 0.7 53.5
40.8 50.3 1.1 46.61
49.3+ 044
All & attacks
31.81 1.0 39.31 1.0 37.21 044 30.91 0.2 38.01 0.2 33.41 0.3 39.6.1 0.7
PGD-6z
48.5 111 49.11 51.81
42.61 0.7 49.91 1.7 47.0.:
49.1 0.4
PGD-Foolbox
45.6 45.24 4.77 41.0=
47.0# 1.3 44.9
47.0= 022
Gaussian Noise
52.5# 1.3 56.1 0.6 03 46.4 09 54.4 0.0
52.1
AutoAttack
42.4# 0.8 41.9# 0.0 4. 38.9=
44.4 13 49
44.6# 0.4
Brendel & Bethge
43.7 0.4 44.4 0.1 46.6+ 11 39.2 07 45.1 1.6 43.6 0.4
45.4= u.1
CWT.2
43.54 1.3 44.8 1.1 0. 39.5
0.4 46.81 19 43.4 0.1
46.0# 0.4
All & attacks
42.5+ 0.6 41.9# 44.9 35.8= 0.7 44.6 0.1 42.4
44.8= 0.1
Acc
union
19.81 11 10.1 0.7 5.2 0.6 26. 09 23.6 0.3 29.0 0.3
27.4 0.8
Acc
33.8 0.1 30.4 0.1 29.1 on 32.8 0.1 35.4+ 0.7 35.3 0.4
37.2 0.6
adv
l
l
t00
Defenses
Ours
CIFARIO 1O
0.30
Dm
au
200
NAAS
ce
6
5
2
landscapes for the norm attacks on the CIFAR-10 10 dataset. The rows represent the attacks and columns represent different defenses. We observe that that MNG-AC obtains smooth loss surface across all norm attacks.
Figure 6: Visualization of E,, fz. and f
Figure 5: Visualization of the generated noise by MNG
along with the perturbed samples on 41 l,, and foe-
attacks for CIFAR-10 10 and SVHN dataset.
16


Reviews:
[{"review_id": "uwcwhu4VVoj", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary\n=======\nThe authors propose a number of techniques to learn models which are adversarially robust to multiple perturbations. These involve a noise generator, a loss to enforce consistency, as well as a stochastic variant of adversarial training. With these changes, they are able to produce improvements to robust accuracy to multiple perturbation types. \n\n\nOverall, I get the idea and the empirical results seem promising. However, the structure and writing of the paper is at times rather confusing, and there are a lot of missing details. If the code were not supplied, it would be difficult in the current state to reproduce the method from the paper. Perhaps due to this, the specifics of the key component, the meta noise generator, are still rather opaque to me. Perhaps the authors can clarify, and I am happy to follow up afterwards. \n\nComments for discussion\n=======================\nThe majority of my confusion lies in section 4, for the specifics of the meta noise generator and parts of the algorithm in general. I am otherwise well acquainted with the relevant literature. \n\n1) Augmented examples (x_aug) are generated by adding noise from the MNG and projecting it onto some ball B. It is not clear to me what ball this is since the authors are considering multiple perturbations. Is it a random type? Or a joint projection? I assume it is at least one of the perturbations being considered, or is that incorrect? \n\n2) Similarly, in the algorithm, the authors generate adversarial examples (x_adv) by sampling a random attack. I could not find what set of attacks were being sampled from, or what the sampling distribution is (I checked the appendix as well). \n\n3) The generator is apparently updated to minimize the classifier loss on the adversarial examples as written in Equation (8). However, the adversarial examples are generated from some unspecified set of attacks, which implies that the set of attacks actually depends on the generator somehow. Is this supposed to be the classifier loss on the augmented samples? If not, then how do the adversarial examples depend on the generator? \n\n4) The consistency loss involves clean, adversarial, and augmented posterior distributions. There are no details on these distributions: are these simply the softmax of the logits? Or is a generative model that outputs a distribution being used? \n\n5) On a more fundamental level, what is the motivation behind training the generator to minimize the classifier loss? Why would we want to do this over random sampling? What's to prevent a degenerate solution of simply learning to produce a zero perturbation (and thus always producing clean examples, which can achieve low loss)? \n\n\nMinor comments\n==============\nI have checked the supplementary material and the authors have included the code for running their experiments. Ideally, this would also include pre-trained model weights. \n\nUpdate\n======\nAfter much effort, I can say that I understand the paper. The edits appear to have incorporated all the identified missing information. I have thus updated my confidence and slightly improved my score, however I am not confident that the current presentation of the approach will be understandable by a reader without contacting the authors, given that the difficulty I had in understanding the paper (and my initial confidence) stemmed primarily from missing information and poor presentation for the approach. Although the results do seem to improve upon past work, its impact will suffer if it is difficult to understand for a non-reviewer reader. I would be more confident if a fresh set of eyes could understand the details of the work without having to go to the authors to clarify so many details. ", "extra_scores": {}, "extra_text": {}}, {"review_id": "uNiWR-V7oXo", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper addresses a timely issue in adversarial robustness - efficient training of robust models against multiple adversarial perturbations. The authors propose a combination of three techniques: stochastic adversarial training (SAT), meta noise generator (MNG), and adversarial consistency (AC) loss for efficient training, and evaluate the robustness using multiple L1, L2, and Linf norm-bounded attacks and three datasets (CIFAR-10, SVHN, and Tiny Imagenet). The results show improved multi-attack robustness over several baselines (including single-attack and multiple-attack models) and reduced training time. Ablation studies are also performed to illustrate the utility of each component of the proposed model. Overall, this paper provides very detailed evaluations involving multiple datasets, attacks, baselines, and robustness metrics. I find the results convincing and important, and also find sufficient novelty in the proposed training method.\n\nThe strengths (S) and weaknesses (W) of this submission are summarized below.\n\nS1. The proposal of MNG and AC is effective and novel.\nS2. The evaluation is thorough and convincing.\nS3. The proposal improves both robustness and training efficiency in most cases.\n\nW1. The adversarial consistency (AC) loss is never defined explicitly. Based on equation (5), it is hard to understand how AC \"represents the Jensen-Shannon Divergence (JSD) among the posterior distributions\" when considering three distributions, P_clean, P_adv, and P_aug. More clarification is needed.\n\nW2. Although the results show improved multi-attack robustness, it will be great if the authors can add more intuition on why the proposed training method leads to performance improvement. Based on the ablation study,  it seems that the role of SAT and MNG is to reduce overfitting in robustness to encourage generalization, rather than optimization over the worst-case scenarios.\n\nW3. The considered multi-attack setting is still limited to different Lp norm perturbation constraints. Although the authors showed improved robustness over unforeseen attacks, the authors should also discuss how the proposed method can generalize to different attacks beyond Lp norms.\n\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "iF-e5SkNBvh", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 1, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "1. Summary\n\nThe authors propose a new method to improve robustness to adversarial examples under various norms (L1, L2 and LInf). Their method combines adversarial training with an adversarial noise generator. They improve upon adversarial training in a multi norm setting by choosing one norm at random for each sample, instead of computing an adversarial for all norms, thus significantly reducing the training time. They additionally improve robustness by regularizing model features between the standard image, the adversarially perturbed image and a perturbation of the image created with an adversarial noise generator.\n\n\n2. Strengths\n+ The method is based on adversarial training. As far as I know and as the authors note this is the only method that reliably leads to more robust models.\n+ The authors attack their models with  a range of attacks that to the best of my knowledge are state-of-the art.\n+ The method apparently works in the multi norm setting.\n\n3. Weaknesses \n- I was missing an intuitive description why the adversarial noise should improve robustness to adversarial attacks. I was only aware of it as a method to improve corruption robustness.\n- I was not always sure if I got everything correctly in sections 4 and 5.3. I think I got it but I sometimes missed a figure. It may e.g. be helpful to include the losses in Figure 1 or make a separate figure. Especially why the MNG was trained the way it is was a bit unclear for me.\n\n\n4. Recommendation\n\nI think this paper is an accept but as I don't work with adversarial examples I am not at all confident in that assessment. From the discussions with people who work on adversarial examples new defenses are usually broken very quickly and there is a number of papers which break numerous defenses. The method is however based on adversarial training which to my knowledge is the only robust method so far and the used attacks seem valid. So I am definitely leaning towards accept but the opinion of a real expert would be highly appreciated as I feel not at all qualified to assess the validity of papers on adversarial examples.\n\n\n5. Questions/Recommendations\n- Is there a difference between the M(eta)NG and the A(dversarial)NG from Rusak et. al. 2020?\n\n\n6. Additional feedback \n- None as the paper is pretty well written.", "extra_scores": {}, "extra_text": {}}, {"review_id": "JN8SQP5aVg", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this paper, the authors propose a novel meta-learning framework that explicitly learns to generate noise to improve model robustness (against multiple types of attacks). The results indicate that the proposed approach improves on the state-of-the-art.\n\nOverall, the paper is well written. However some details are missing and this could make the paper hard to reproduce. The experiments could be expanded.\n\n1) There is a significant amount of work about using generative models to build adversarial examples. The literature review only focuses on classical adversarial robustness and robustness against multiple adversaries. I'd recommend making a review of these approaches, even if they are orthogonal to the one proposed in this paper (e.g., [1,2,3])\n2) In Eq. (6), what is \\mathcal{B}(x, \\epsilon). Since there is multiple threat models, I am assuming that it is selected at random between l_1, l_2 and l_inf (like SAT).\n3) The number of inner steps T seems to be critical (as it will trade-off gradient precision with compute). However, I don't see any study on this in the paper. Also, it is not clear which value was used for the experiments.\n4) Looking at Eq. (7), it seems like backpropagation through the T inner steps is necessary to compute the gradients w.r.t. \\phi. This seems overly expensive and I find surprising that adv_avg and adv_max take so much longer to train.\n5) Concerning Eq. (7), as a curiousity, have authors considered implicit differentiation [4] ?\n6) The experiments are run using 30 epochs which is rather on slim side. E.g., RST_inf should reach about 59% robust accuracy with 200 epochs of training (with 30 epochs it only reaches 55%). I'm curious as to whether the comparison with the proposed approach is unfair (e.g., Adv_inf sees a single adv example per batch, whereas MNG-AC sees 2).\n7) It's not entirely clear to me why beta negatively affects l_2 robustness. I'd assume that if the model was only trained against l_2, then there might be an optimal value for beta that is different that the one from Fig. 2. In general, it would be interesting to see on MNG-AC does if different subsets of threats are used.\n8) The l_2 loss landscapes seem more noisy that what they should be. Also it's unclear why the axes are centered for l_inf and not for l_2 (explain how these are generated).\n9) In Table 5, MNG-AC achieves 35.1% against all l_inf attacks, but only 33.7% against AutoAttack. Am I missing something?\n\nDetails:\n\nA) It would helpful to the reader to have the epsilon values written on top of the different tables. The captions could be expanded to include more details.\nB) Visuaization -> Visualization\n\n[1] https://openreview.net/pdf?id=SJeQEp4YDH: GAT: Generative Adversarial Training for Adversarial Example Detection and Robust Classification\n[2] https://arxiv.org/pdf/1801.02610: Generating Adversarial Examples with Adversarial Networks\n[3] https://arxiv.org/pdf/1710.10766: PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examples\n[4] https://arxiv.org/pdf/1911.02590: Optimizing Millions of Hyperparameters by Implicit Differentiation", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This is a borderline case. The paper seems solid although some of the numbers are likely incorrect because in some results tables in the appendix the error taken over all attacks is higher than for the best individual attack (which should never happen).

The main contribution of this paper is to augment a standard adversarial loss (against attacks from different norms) with a “consistency” term (consistency between clean, adversarial and noise augmented samples). The relatively large jump in robustness compared to existing schemes that do adversarial training against multiple norms is a bit surprising. A possible explanation could be that the additional consistency term smoothes the landscape around the clean samples a little bit, which could help to find better adversarial examples. The latter would be very similar to a paper by Pushmeet and colleagues (https://arxiv.org/pdf/1907.02610.pdf) which is not cited, but definitely should. It might also be worthwhile to compare to this paper. 

Taken together, this work is interesting but not sufficiently convincing yet to belong to the top papers to be selected for publication at ICLR. 

Decision: Reject