Towards Defending Multiple Adversarial Perturbations via Gated Batch Normalization
Authors: Aishan Liu
Shiyu Tang
Xianglong Liu
Xinyun Chen
Lei Huang
Zhuozhuo Tu
Dawn Song
Dacheng Tao
Venue:  (2021)
Keywords: adversarial examples
multiple adversarial peturbation types
adversarial robustness

Abstract: There is now extensive evidence demonstrating that deep neural networks are vulnerable to adversarial examples, motivating the development of defenses against adversarial attacks. However, existing adversarial defenses typically improve model robustness against individual specific perturbation types. Some recent methods improve model robustness against adversarial attacks in multiple $\ell_p$ balls, but their performance against each perturbation type is still far from satisfactory. To better understand this phenomenon, we propose the \emph{multi-domain} hypothesis, stating that different types of adversarial perturbations are drawn from different domains. Guided by the multi-domain hypothesis, we propose~\emph{Gated Batch Normalization (GBN)}, a novel building block for deep neural networks that improves robustness against multiple perturbation types. GBN consists of a gated sub-network and a multi-branch batch normalization (BN) layer, where the gated sub-network separates different perturbation types, and each BN branch is in charge of a single perturbation type and learns domain-specific statistics for input transformation. Then, features from different branches are aligned as domain-invariant representations for the subsequent layers. We perform extensive evaluations of our approach on MNIST, CIFAR-10, and Tiny-ImageNet, and in doing so demonstrate that GBN outperforms previous defense proposals against multiple perturbation types, \ie, $\ell_1$, $\ell_2$, and $\ell_{\infty}$ perturbations, by large margins of 10-20\%.

Under review as a conference paper at ICLR 2021
TOWARDS DEFENDING MULTIPLE ADVERSARIAL PERTURBATIONS VIA GATED BATCH NORMALIZATION
Anonymous authors Paper under double-blind review
ABSTRACT
There is now extensive evidence demonstrating that deep neural networks are vul- nerable to adversarial examples, motivating the development of defenses against adversarial attacks. However, existing adversarial defenses typically improve model robustness against individual specific perturbation types. Some recent methods improve model robustness against adversarial attacks in multiple f, balls, but their performance against each perturbation type is still far from satisfactory. To better understand this phenomenon, we propose the multi-domain hypothesis. stating that different types of adversarial perturbations are drawn from different domains. Guided by the multi-domain hypothesis, we propose Gated Batch Nor- malization (GBN), novel building block for deep neural networks that improves robustness against multiple perturbation types. GBN consists of a gated sub- network and multi-branch batch normalization (BN) layer, where the gated sub- network separates different perturbation types, and each BN branch is in charge of single perturbation type and learns domain-specift statistics for input trans- formation. Then, features from different branches are aligned as domain-invariant representations for the subsequent layers. We perform extensive evaluations of our approach on MNIST, CIFAR-10, 10, and Tiny-ImageNet, and demonstrate that GBN outperforms previous defense proposals against multiple perturbation types, i.e., &,, f2, and tor perturbations, by large margins of 10-20%
INTRODUCTION
Deep neural networks (DNNs) have achieved remarkable performance across wide areas of appli- cations (Krizhevsky et al. 2012 Bahdanau et al. 2014] Hinton et al. 2012), but they are susceptible to adversarial examples (Szegedy et al.] 2013). These elaborately designed perturbations are im- perceptible to humans but can easily lead DNNs to wrong predictions, threatening both digital and
physical deep learning applications (Kurakin et al. 2016 Liu et al.) 2019a).
To improve model robustness against adversarial perturbations, a number of adversarial defense methods have been proposed (Papernot et al.) 2015) Engstrom et al., [2018] Goodfellow et al. 2014). Many of these defense methods are based adversarial training (Goodfellow al. 2014 Madry ct al. 2018), which augment training data with adversarial examples. However, most adversarial defenses are designed to counteract single type of perturbation (e.g., small -noise) (Madry jet al. 2018, Kurakin et al. 2017] Dong et al., (018). These defenses offer no guarantees for other perturbations (e.g., é.. Eg). and sometimes
increase model vulnerability (Kang et al., 2019 Tramer & Bonch 2019). To address this problem, other adversarial training strategies have been proposed with the goal of simultaneously achieving robustness against multiple types of attacks, i.e., foc. f,, and & attacks (Tramer & Boneh 2019 Maini et al., 2020). Although these methods improve overall model robustness against adversarial attacks multiple lp balls, the performance for each individual perturbation type is still far from satisfactory.
In this work, we propose the ial perturbation arise in different domains, and thus have separable characteristics. Training on data from multiple domains can be regarded as solving the invariant risk minimization problem (Ahuja al. 2020), in which an invariant predictor is learnt to achieve the minimum risk for different envi- ronments. For a deep learning model, instance-related knowledge can be stored in the weight matrix
hypothesis, which states that different types of adversar-
Our code will be available upon publication.

---

Under review as a conference paper at ICLR 2021
of each layer, whereas domain-relann knowledge can be represented by the batch normalization (BN) layer statistics (Li et al. 2017).
Inspired by the multi-domain hypothesis, we propose to improve model robustness against multiple perturbation types by separating domain-specific information for different perturbation types, and using BN layer statistics to better align data from the mixture distribution and learn domain-invariant representations for multiple adversarial examples types. In particular, we propose a novel building block for DNNs, referred as Gated Batch Normalization (GBN), which consists of gated sub- network and a multi-branch BN layer. GBN first learns to separate perturbations from different domains on-the-fly fly and then normalizes them by obtaining domain-speciin each BN branch handles single perturbation type (i.e., domain). Then, features computed from different branches are aligned as invariinnn representations that are aggregated as the input to subsequent layers. Extensive experiments on MNIST, CIFAR-10, 10, and Tiny-Imagel demonstrate that our method outperforms previous defense strategies by large margins, Lee, 10-20%.
features. Specifically,
2 BACKGROUND AND RELATED WORK
In this section, we provide a brief overview of existing work on adversarial attacks and defenses, as well as batch normalization techniques.
2.1 ADVERSARIAL ATTACKS AND DEFENSES
Adversarial examples are inputs intentionally designed to mislead DNNs (Szegedy et al. 2013 Goodfellow et al. 2014). Given a DNN fe and an input image x E with the ground truth label y Y, an adversarial example Xadv satisfies
fe (Xadd) # y s.t. ||x - -xado |l S
where || || is a distance metric. Commonly, || is measured by the f--norm (p (11,2,x0))
Various defense approaches have been proposed improve model robustness against adversarial examples (Papernot et al., 2015 Xie et al. 2018 Madry et al.) 2018: Liao et al.) 2018 Cisse et al. 2017], among which adversarial training has been widely studied and demonstrated to be the most effective (Goodfellow et al. 2014 Madry et al. 2018). Specifically, adversarial training minimizes the worst case loss within some perturbation region for classifiers, by augmenting the training set with adversarial examples. However, these defenses only improve model robust- ness for one type of perturbation (e.g., tox) and typically offer no robustness guarantees against other
(x(1), yy0))
attacks (Kang et al. 2019 Tramer Bonch 2019 Schott et al. 2019)
To address this problem, recent works have attempted to improve the robustness against several types of perturbation (Schott al. 2019 Tramèr & Boneh 2019 Maini al. 2020). Schott et al. (2019) proposed Analysis by Synthesis (ABS), which used multiple variational autoencoders to de- fend Lg. l2, and &o adversaries. However, ABS only works on the MNIST dataset. Croce & Hein (2200a) proposed provable adversarial defense against all fp norms for p 21 using regularization term. However, it not applicable to the empirical setting, since it only guarantees robustness for very small perturbations (e.g., 0.1 and 2/255 for & and f.x on CIFAR-10), 10). Tramer & Boneh (2019) tried to defend against multiple perturbation types (L,, fg. and txx) by combining different types of adversarial examples for adversarial training. Specifically, they introduced two training strategies, "MAX" and "AVG", where for each input image, the model is either trained on its strongest ad- versarial example all types of perturbations. More recently, Maini et al. (2020) proposed multi steepest descent (MSD), and showed that a simple modification to standard PGD adversarial train- ing improves robustness to éi, l,, and l.x adversaries. In this work, we follow (Tramèr & Boneh, 2019 Maini et al., 2020) to focus on defense against &, l,, and lxx. adversarial perturbations, which are the most representative and commonly used perturbations. However, we propose completely different perspective and solution to the problem.
2.2 BATCH NORMALIZATION
BN [Ioffe & Szegedy 2015) is typically used to stabilize and accelerate DNN training. Let x e Id denote the input to neural network layer. During training, BN normalizes each neuron/channel
2

---

Under review as a conference paper at ICLR 2021
within m mini-batch data by
X, = (N(x) = = Xj - 14 +Bz. j 1=,,2,...,4 d,
(I)
0o3+5 +
where M4 m E x and - (x)² each neuron, respectively, and is small number to prevent numerical instability. The learnable parameters y and 3 are used to recover the representation capacity. During inference, the population statisties of mean á and variance 6² are used in Eqn. | which usually calculated the running average over different training iterations with update factor a:
p2)² are the mini-batch mean and variance for
ji' = =1 - (1-a) + tapt (0¹)² = (1 (6))()6 a(o"-1)?
(2)
A number of normalization techniques have been proposed to improve BN for style-transfer (Hang & Beloongie 2017) and domain adaption (Li et al., 2017 Chang et 2019 Deecke et al.) 2019),
Compared to studies for domain adaptation, our GBN does not require inpul domain knowles ledglegge during inference. contrast Mode Normalization (Deecke al., 2019), which aims learn mixture distribution for multi-task learning, GBN separates domain- speciifcc statistics for different input types and then aligns the data to build domain-invariant representations. Recently, Xie & Yuille (2020); Xie et al. (2200) applied separate BNs to clean and adversarial examples for statistics estimation. In contrast to the above two studies that improve image recognition or analyze the properties of BN to adversarial training. we aim to defend against multiple adversarial perturbations. Furthermore, GBN can predict the domain for an input sample and then jointly incorporate the outputs of multiple BN branches to provide the final classification result. Conversely, Xie & Yuille (2200; Xie et al. (2020) manually selected the BN branch for each input image, which requires prior knowledge of whether the input is adversarial or clean example (the same domain adaptation).
GATED BATCH NORMALIZATION
In this section, we present our proposed Gated Batch Normalization (GBN) approach, to improve the model robustness against multiple adversarial perturbations. We first illustrate our multi-domain hypothesis, which states that adversarial examples of different perturbation types are drawn from different domains. Motivated by this hypothesis, we demonstrate the architecture of our GBN block, then describe the training and inference procedures.
3.1 MOTIVATION: MULTI-DOMAIN HYPOTHEESI
We assume N adversarial perturbation types each characterized by a set set of perturbations for an input x. Let ID" denote the set of clean examples, and (k* I..., N) denote the of adversarial examples generated by the k-th adversarial perturbation type S*. An adversarial example of the k-th type x generated by the pixel-wise addition of the perturbation 8², i.e., x* X
+ 8*. Our multi-domain hypothesis states that different types of perturbations 4 (for all k 2 0. including the clean examples) are drawn from different domains, inspired by the following observations:
(1) Training on a single perturbation type insufficient for achieving the robustness against other perturbations. Further, training on a mixture of different perturbation types still fail to achieve acceptable performance for each type of perturbation (Tramèr & Boneh 2019 Maini et al. 2020).
(2) Adversarial examples are separable from clean examples (Metzen et al. 2018 Xie & Yuille (2020) also suggested that clean and adversarial examples are drawn from two different domains.
We first empirically investigate the hypothesis for deep neural networks. Specifically, we train separate BNs for different input domains I 2 0). As illustrated Figure I(b) we construct separate mini-batches to estimate BN statistics for different DV i.e., e,, f2, and Exx adversarial images, as well as for clean images. We defer the implementation details to Appendix A Figure I(c) and ((d) show that different & induce significantly different running statistics, according to the running means and variances of different BN branches.
In this work, we consider N- adversarial perturbation types: E1, la and Eoo.
3

---

Under review as a conference paper at ICLR 2021
We further theoretically demonstrate that for linear classifier f(x) sign(w"x + b). different types of adversarial perturbations belong to different domains, with pair-wise Wasserstein distances greater than 0 (c.f. Appendix B).
These theoretical and empirical results support our multi-domain hypothesis, and motivate our de- sign of GBN, as diseussed below.
ReLU
ReLU
BN
BN loan
BN
BN
BN
Conv
Conv
Xekan
X
X
X
Xelean
X
X
X
(a) standard BN
(b) multiple BN branches
Clean
10
Clean
charnel
charnol
(c) running mean
(d) running variance
Figure 1: (a) the standard BN structure; (b) the structure with 4 BN branches, which construct different mini-batches for different & estimate the normalization statistics; (c) and (d): running means and variances of multiple BN branches on 16 randomly sampled channels in VGG-16's conv2.1 layer, showing that different perturbation types induce different normalization statistics. More details of running statistics of different models can be found in Appendix E.
3.2 MODEL ARCHITECTURE WITH GATED BATCH NORMALIZATION (GBN)
Training a model that generalizes well on multiple domains can be regarded as solving an invariant risk minimization problem (Ahuja et al. 2020). Define predictor f X - Y and the risk achieved by f in domain T as (J) E( {(((x). y), where ((-) is the cross entropy loss. We say that data representation æ: X + elicits an invariant predictor w o æ across all domains k E (0, 1.... N] if there is a classifier w: Z Y that achieves the minimum risk for all domains.
To obtain a domain-invariant representation π,, we use the BN layer statistics to align data from the mixture distribution. Specifically, GBN uses BN layer with multiple branches v (BNN (-), O, 0,...,N N} during training, where each branch BN* (.) is exclusively in charge of domain D The aligned data then aggregated the input subsequent layers to train classifier which achieves the minimum risk across different domains (i.e., model robust to different perturbations).
One remaining problem is that the model does not know the input domain during inference. To calculate the normalized output in this case, GBN utilizes a gated sub-network Φx to predict the domain for each layer input x; we will illustrate how to train ox(x) in Section 3.3 Given the output
4

---

Under review as a conference paper at ICLR 2021
of sub-network g &x(x)= E RN+1, calculate the normalized output in soft-gated way:
N
& = GBN(x)
g
(3)
A==
where ga represents the confidence of the k-th domain for x, and x* is the normalized output of (.), which uses the population statistics of the domain D perform standardization.
We provide overview of the inference procedure in Figure 2(b) and the details are presented in Algorithm In Appendix E.6 we discussed an alternative approach that takes the top-1 prediction of g (the hard label), and it achieved comparable performance.
Our GBN aims to disentangle domain-invariant features and domain -spcciic features: (1) the distri- butions in different domains are aligned by their normalized outputs (all are standardized distribu- tions), which ensures that the following linear layers learn domain-invarian representations; and (2) the domain- domain-specific features for each domain D are obtained by the population statistics (A² )*) of its corresponding BN branch (.). We show empirically that (it", 6*) of BN)( learns the domain-specific representation well, shown in Figure T
Algorithm Inference of Gated Batch Normalization (GBN)
Input: Layer input x Output:
Normalized output by GBN
1: Compute the output of gated sub-network g &x
2: for & in N+1 domains do
Let x go through 3N*() (.) and obtain the normalized output x based on Equ.
4: end for
5:
Compute GBN output based on Eqn.
RelU
malial
Dreetos
Inanche
BN
BN
BN
BN
BN
BN
gated sub-network ftr
goted sub network
for
gated sub network
GBN
GBN
GBN
Conv
Conv
X
X
(a) Training of GBN
(b) Inference
Figure 2: The training and inference procedures of GBN. During training, samples from different domains (denoted by different colors) are fed into corresponding BN branches update the running statistics. During inference, given mini-batch data, the gated sub-network first predicts the domain of input x and then jointly normalizes it using multiple BN branches.
3.3 TRAINING
We provide overview of the training procedure in Figure 2(a) and the details are in Algorithm 2 Specifically, for each mini-batch 8 consisting of clean samples, we use PGD attack (Madry et al. 2018) to generate batches of adversarial examples 1*th (k E N1, for each domain D To capture the fomain--peciffe statistics for different perturbation types, given mini-batch data 8 from domain D ensure that 8 goes through its corresponding BN branch BN (+), and Eqn.
5

---

Under review as a conference paper at ICLR 2021
to compute the normalized output B* The population statistics fA* _*) of (-) are updated In other words, we disentangle the mixture distribution for normalization and apply separate BN branches different perturbation types for statistics estimation.
based on Eqn. 2
To train the gated sub-network g, we provide the supervision of the input domain for each training sample. Specifically, we introduce the domain prediction loss Lpr:
CDP
((Do(x).
k= (x,A)EE
Finally, we optimize the parameters θ of the entire neural network (e.g., the weight matrices of the convolutional layers, except for the gated sub-network) using the classification loss Lets:
Lets Σ Σ R=U (X
(5)
Computational cost. With respect to the parameters, we have N+1 pairs of {āz, ô} to estimate compared one in standard BN. With respect to the time consumption, in the adversarial learning literature, the most consuming procedure in ad adversarial training is generating adversarial exam- ples. Although we introduce more parameters, our time cost is close other adversarial defensive baselines (see Appendix E.2 for more details).
Algorithm Training of Gated Batch Normalization (GBN) for Each Iteration.
Input: Network f with GBN
Output: Robust model parameters and
Given mini-batch data B generate corresponding adversarial example batch
B
(k
{1,.., NY) with specific perturbation type by PGD attacks
2: for N+1 domains do
3: Let 8 go through (-) at each layer obtain normalized outputs 8 based on Eqn.
4: Update the population statistics (jih, 6* of BN* (-) at each layer based on Eqn.
5: end for
6: Update parameters of the gated sub-network at each layer based on Eqn.
7: Update parameters θ of the whole network based Eqn.
4 EXPERIMENTS
We evaluate the effectiveness of our GBN block to simultaneously achieve robustness against é,, f2, and foc perturbations, which are the most representative and commonly used adversarial pertur- bation. We conduct experiments on the image classification benchmarks MNIST (LeCun 1998). CIFAR-10 (Krizhevsky Hinton 2009), and Tiny-ImageNet (Wu et al. 2017).
4.1 EXPERIMENT SETUP
Architecture and by hyperparameters. We use LeNet architecture (LeCun al.) 1998) for MNIST;
ResNet-20 (He et al. 2016). VGG-16 (Simonyan & Zisserman) 2015), and WRN-28 10 (Zagoruyko Komodakis (016) for CIFAR-10; and ResNet-34 (He et al. (2016) for -iny-Im For fair comparisons, keep the architecture and main
the same for GBN and other baselines. A detailed description of the hyper-parametes can found in Appendix D
Adversarial attacks. To evaluate the model robustness, we follow existing guidelines (Schott al. 2019 Tramer& Boneh 2019: Maini et al. 2020) and incorporate multiple adversarial attacks for different perturbation types. For MNIST, the magnitude of perturbation for &, ε2, and foo is 10, 2, 0.3. For CIFAR-10 10 and Tiny- ImageNet, the magnitude of perturbation for l,, l2, and lxc 12, 0.5, 0.03. For & attacks, adopt PGD attack (Madry et al. 2018). and Brendel &

---

Under review as a conference paper at ICLR 2021
Bethge attack (BBA) (Brendel et al.
2019); for &y attacks, we use PGD attack, C&W attack (Carlini
& Wagner 2017). Gaussian noise attack (Rauber et al. 2017). and boundary attack (BA) (Brendel
et al.
2018) ; For Lo attacks, we use PGD attack, Fast Gradient Sign Method (FGSM) (Goodfellow
(et al. 2014), SPSA (Uesato et al. 2018), Nattack (Li et al., 2019), Momentum Iterative Method
(MI-FGSM) (Dong et al. 2018), C&W attack, and the recently proposed AutoAttack (Croce &
Hein (2020b) (a stronger ensemble of parameter-free attacks, including APGD, FAB (Croce & Hein
2020c), and Square Attack (Andriushchenko et al., 2020). We adopt FoolBox (Rauber et al. 2017)
as the implementation for the attacks. Note that, to prove that no obfuscated gradient has been introduced, we adopt both the white-box and the black-box or gradient-free adversarial attacks. A
more complete details of all attacks including hyper-parameters can be found in Appendix D.I
Adversarial defenses. We compare with works that achieve robustness against multiple perturba- tion types. All of them consider the union of f1. f2 for adversaries ( We compare with ABS (Schott et al., 2019). MAX, AVG (Tramèr & Boneh| 2019), and MSD) (Maini et al., 2020). For complete- ness, we also compare with TRADES (Zhang et al. 2019) and PGD adversarial training (Madry) ct al. 2018) with single perturbation type (i.e. E,, E2. and &ec. respectively as P1. Pz, and Pac)
Normalization techniques. We further compare our methods to some normalization techniques (i.e., MN (Deecke et al. 2019) and MBN (Xie & Yuille 2020).. MN extends BN to more than single mean and variance, detects the modes of the data, and then normalize samples that share common features. For MBN, since manually selecting the BN branch is infeasible in the adversarial setting, add 2-way gated sub-network each MBN block.
4.2 ADVERSARIAL ROBUSTNESS FOR MULTIPLE PERTURBATIONS
In this section, we evaluate model robustness against multiple perturbation types (i.e., l1, fg., and oo) in both the white-box and black-box -bxx setting. We use the worst case top-1 classification accuracy, where we report the worst result among various attacks for each perturbation type (the higher the better). We also report all attacks which represents the worst-case performance computed over all attacks for each example (Maini et al. 2020). Specifically, given an image, if generate an adversarial attack that leads to wrong predictions using any attacks, we mark it as a failure.
For all attacks except FGSM and APGDDE (the default setting in AutoAttack), we made 5 random restarts for each of the results. As for models using our GBN, we trained each model 3 times, and the results are similar, showing that training with GBN is stable.
The results on MNIST using LeNet, CIFAR-10 using ResNet-20, and Tiny-ImageNet using ResNet- 34 are shown in Table [ We present the detailed breakdown results for individual attacks and more experimental results using VGG-16 and WRN-28-10 on CIFAR-10 in Appendix E.3 We observe that: (1) on multiple perturbation types (i.e., E,. E2, s,)), our GBN consistently outperforms the baselines by large margins of over 10% on all three datasets; (2) due to the trade-off between adversarial robustness and standard accuracy (Tsipras et al. 2019), our clean accuracy is lower than the vanilla model. However, GBN maintains a comparatively high clean accuracy compared to other adversarial defenses; and (3) GBN is easy to train on different datasets, while the other methods (e.g., MSD) are difficult to converge on large datasets such as Tiny-ImageNet.
We also evaluate our model on PGD attacks using different iteration steps (i.e., 50, 100, 200, and 1000) in terms of Loo norm. As shown in Appendix E.4. our GBN still outperforms other methods and remains a high level of robustness.
Comparison with other normalization methods. As shown in Table GBN outperforms MN and MBN by large margins (up to 60%). Here, we provide more comparisons with MN and MBN. Apart from the target task, our GBN differs them on the general ideas: GBN first aligns the distribu- tion among different perturbations by using one BN branch to handle one perturbation type (learn domain-specific features), and then the aligned distribution contributes to the subsequent modules or layers (e.g., convolutional or fully connected layer) for learning domain-invariant representations. GBN ensure that the normalized output for each perturbation type has a Gaussian distribution, and these Gaussian distributions among different perturbations are well aligned (e.g., one clean example and its corresponding adversarial examples are more likely to be close in the space of normalized output. See Appendix E.9 for the visualization of features after normalization blocks using t-SNE
Schott et al.
2019 consider the perturbations, which is subsumed within the hall of the same radius.
7

---

Under review as a conference paper at ICLR 2021
Laurens & Hinton (008)). By doing this, the other modules easier to learn the native rep- resentation of the data (domain-invariant representations). However, MN and MBN cannot align the distributions. MN aims to preserve the diversity among different distributions, by learning the mixture of Gaussian, not one single Gaussian. As for the modified MBN, the data from different perturbations are mixed in the same BN. The results also confirm multi-domain hypothesis.
Different variants of white-box attacks. Here, we examine the performance of GBN in the more rigorous white-box attack scenario. In this case, conduct two experiments follows: (1) tacking the GBN layers, i.e. generate adversarial examples to fool the classification of the gated sub-networks; (2) the adversary manually selects the BN branch to generate adversarial examples. Note that these attacks are specially designed for our defense aiming to provide further analysis and understanding. As shown in Table 1 and 17 in Appendix E.8 GBN remains the same level of robustness against these attacks. We conjecture the reasons are as follows: (1) our GBN does not rely on obfuscated gradients (Athalye al., 2018) to provide model robustness; (2) different GBN layers use different features to predict the domains, and it is difficult for an adversary to generate perturbations to fool all GBN layers within model under the magnitude constraint (see Table [6 for results of prediction accuracy of gated sub-networks different layers). We defer more discussions to Appendix E.
[5]
Table 1: Model robustness different datasets (%). We also provide the Standard Deviation for All attacks of each method on each dataset.
LeNet on MNIST
Vanilla
TRADES
AVG
MAX
ABS
MSD
MN
MBN
GBN(aurs)
f2 attacks
1.4
12.0
60.2
59.0
83.0
71.4
18.9
65.1
93.2
to attacks
77.0
45.8
58.8
17.0
38.9
19.2
0.0
9.8
29.7
31.2
17.0
38.9
19.2
68.9
attacks
+0.06
=0.12
+0022
=0.21
+0.28
+0024
+0004
+0.36
+0.30
(b) ResNet-20 CIFAR-10
Vanilla
TRADES
P.
P2
Px
AVG
MAX
MSD
MN
MBN
GBN(ours)
attacks
16.1
22.3
34.2
17.9
44.9
43.7
39.8
44.9
f2 attacks
0.0
59.5
1.1
51.0
50.3
59.1
56.0
58.9
30.0
20.8
68.9
20 attacks
42.1
All attacks
0.0
16.0
0.0
19.0
17.8
28.2
24.9
37.9
13.0
20.7
48.7
100.33
10.32
10000
1.042
10.38
-0.51
1:0.45
1:0.58
1:0.52
1:0.61
1:0.6
Clean
89.7
86.9
84.3
87.9
84.2
80.6
77.0
79.1
82.3
79.4
80.7
(c) ResNet- 34 Tiny- Imageet
Vanilla
AVG
MAX
MSD
MN
MBN
GBN(ours
attacks
5.8
27.3
17.0
7.6
8.6
36.6
44.2
22 attacks
10.3
29.1
24.2
11.0
17.9
31.1
42.1
2o
0.0
4.5
2.7
0.0
4.6
2.2
5.0
6.7
20.0
38.9
attacks
10.06
#:065
=0.58
#:069
0066
1:0.52
=0.62
Cleam
54.2
41.6
36.3
28.8
46.7
45.7
43.4
4.3 ABLATION STUDY
Attacks outside the perturbation model. To examine whether GBN could generalize perturba- tion types that the model was not trained on, we trained variant of GBN models that only includes BN branches. Specifically, these GBN models are trained on two L, perturbation types only, but evaluated on all the three É, perturbations, including the held-out perturbation that is not trained. Unsurprisingly, compared to the full GBN model with 4 BN branches, the robustness of the held-out perturbation type decreases some degree. However, the robustness is still significantly better than the vanilla model, and sometimes even outperforms other baseline defenses that are trained on all perturbation types (c.f. Appendix E.7
8

---

Under review as a conference paper at ICLR 2021
Adding GBN into different layers. Our GBN block can be inserted any layer in deep network. In this part, we study GBN by single-layer and layer- group studies. We first add GBN to different single layers: shown in Figure 3(a] the standard performance and adversarial robustness decrease as we add GBN into the deeper layers; we then add GBN to layer groups (i.e. top-n layers): according the results in Figure 3(b the adversarial robustness improves (but the clean accuracy remains comparatively stable) as more layers are involved. In summary, adding GBN into shallow layers achieves better performance with respect both clean and adversarial accuracy. The reasons for this might be two-fold: (1) as shown in (Liu et al. (019b), shallow layers are more critical to model robustness than deeper layers; (2) the layer depth increases, the features from different domains are highly entangled, making it harder for models to separate them (c.f. Appendix [E.10))
Predictions the gated sub-network Moreover, we provide the prediction results of the gated sub-network in GBN, Le., the classification accuracy for the domain of different input samples. As shown in Table TT] for MNIST, the gated sub-network of LeNet at different layers achieves high prediction accuracy for the input domains (i.e., clean, L,, l2. and fox adversarial examples). However, on CIFAR-10, the prediction accuracy of the gated sub-network drops with the increasing of the layer depth. We can conclude that shallow layers more critical to model robustness the prediction accuracy at the shallow layers is higher than that of the deeper layers. The reason might be that shallow layers contain limited information about the image statistics and can be learned via a few of samples (Asano et al.) 2020). in which the features are more easy to separate. However, features the first layer is still unexplored, which we leave
the reason for the indistinguishabiiiy for future works. To further improve model robustness against multiple perturbations, we suggest to improve the prediction accuracy of the gated sub-network.
Claszificatton Accuracy %
P.D
Claszification Accuracy %
PGD-4.
PSD-i
PGD-6
Clean
POD-4
19
Layers
Top-m Layers
(a) single layer
(b) top-m layers
Figure 3: (a) demonstrates the results of adding GBN different single layers. (b) shows the results of adding GBN to top-m layers. All the experiments are conducted using a VGG-16 on CIFAR-10.
CONCLUSIONS
Most adversarial defenses are typically tailored to single perturbation type (e.g., small -noie,, but offer no guarantees against other attacks. To better understand this phenomenon, we explored the multi-domain hypothesis that different types of adversarial perturbations are drawn from different domains. Guided by this hypothesis, Normalization (GBN), which consists of a gated network and a multi-branches BN layer. The gated sub-network separates different perturbation types, and each BN branch is in charge of single perturbation type and learns the domain specific statistics for input transformation. Then, features from different branches are aligned as domain-invariant representations for the subsequent layers. Extensive experiments on MNIST, CIFAR-10, and Immg-Im demonstrate that our method outperforms previous proposals against multiple perturbation types by large margins of 10-20%.
propose novel building block for DNNs, Gated Batch
9

---

Under review as a conference paper at ICLR 2021
REFERENCES
Kartik Abuja, Karthikeyan Shanmugam, Kush Varshney, and Amit Dhurandhar. Invariant risk min- imization games. In International Conference on Machine Learning. 2020.
Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein. Square at- tack: a query-efficient black-box adversarial attack via random search. In European Conference Computer Vision, 2020.
Yuki M Asano, Christian Rupprecht, and Andrea Vedaldi. A critical analysis of self-supervisiio, or what can learn from single image. In International Conference on Learning Representations, 2020.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give false sense of security: Circumventing defenses to adversarial examples. International Conference on Machine Learning, 2018.
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.04755 2014.
Wieland Brendel, Jonas Rauber, and Matthias Bethge. Decision-based adversarial attacks: Reliable attacks against black-box machine learning models. In International Conference on Learning Representations, 2018
Wieland Brendel, Jonas Rauber, Matthias Kümmerer, Ivan Ustyuzhaninov, and Matthias Bethge. Accurate, reliable and fast robustness evaluation. Advances in Neural Information Processing Systems, 2019.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In IEEE Symposium on Security and Privacy, 2017.
Woong-Gi Gi Chang, Tackgeun You, Seonguk Sco, Suha Kwak, and Bohyung Han. Domain-specific batch normalization for unsupervised domain adaptation. In IEEE Conference Computer Vi- sion and Pattern Recognition, 2019.
Moustapha Cisse, Piotr Bojanowski, Edouard Grave, Yann Dauphin, and Nicolas Usunier. Parseval networks: Improving robustness to adversarial examples. In International Conference on Machine Learning, 2017.
Francesco Croce and Matthias Hein. Provable robustness against all adversarial lp-perturbations for p 2 1. International Conference on Learning Representations, 2020a.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter- free attacks. In International Conference on Machine Learning. 2020b.
Francesco Croce and Matthias Hein. Minimally distorted adversarial examples with fast adaptive boundary attack. In International Conference on Machine Learning, 2020c.
Lucas Deecke, lain Murray, and Hakan Bilen. Mode normalization. In International Conference on Learning Representations, 2019.
Yinpeng Dong, Fangzhou Liao, Tianyu Pang. and Hang Su. Boosting adversarial attacks with mo- mentum. In IEEE Conference on Computer Vision and Pattern Recognition, 2018.
Logan Engstrom, Andrew Ilyas, and Anish Athalye. Evaluating and understanding the robustness of adversarial logit pairing. arXiv preprint 1 2018.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples (2014). arXiv preprint arXiv:1412.6572.. 2014.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In IEEE Conference Computer Vision and Pattern Recognition, 2016.
10

---

Under review as a conference paper at ICLR 2021
Geoffrey Hinton, Li Deng, Dong Yu, George E. Dahl, Abdelrahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, and Tara N. Sainath. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal Processing Magazine, 2012.
Xun Huang and Serge Belongie. Arbitrary style transfer in real-time with adaptive instance normal- ization. In IEEE International Conference on Computer Vision, 2017.
Sergey loffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International Conference on Muchine Leurning, 2015.
Daniel Kang, Yi Sun, Dan Hendrycks, Tom Brown, and Jacob Steinhardt. Testing robustness against unforeseen adversaries. arXiv preprint arXiv:1908.080 08016, 2019.
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Tech- nical report, Citeseer, 2009.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. Imagenet classification with deep convo- lutional neural networks, In International Conference Neural Information Processing Systems, 2012.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial examples in the physical world. arXiv preprint arXiv: loo..0250.05 2016.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial machine learning at scale. In International Conference on Learning Representations, 2017.
Van Der Maaten Laurens and Geoffrey Hinton. Visualizing data using t-sne. Journal of Machine Learning Research, 2008
Yann LeCun. The mnist database of handwritten digits. http://yann. lecun. com/exdb/mmist// 1998.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEFE, 1998.
Yandong Li, Lijun Li, Liqiang Wang, Tong Zhang, and Boqing Gong, Nattack: Learning the dis- tributions of adversarial examples for an improved black-box attack on deep neural networks. In International Conference Machine Learning, 2019.
Yanghao Li, Naiyan Wang, Jianping Shi, Jiaying Liu, and Xiaodi Hou. Revisiting batch normaliza- tion for practical domain adaptation. In International Conference on Learning Representations, 2017.
Fangzhou Liao, Ming Liang, Yinpeng Dong, Tianyu Pang, Xiaolin Hu, and Jun Zhu. Defense against adversarial attacks using high-level representation guided denoiser. In IEEE Conference on Computer Vision and Pattern Recognition, 2018
Aishan Liu, Xianglong Liu, Jiaxin Fan. Yuqing Ma, Anlan Zhang, Huiyuan Xie, and Dacheng Tao. Perceptual-sensitive gan for generating adversarial patches. 33rd AAAI Conference on Artificial Intelligence, 2019a.
Aishan Liu, Xianglong Liu, Chongzhi Zhang, Hang Yu, Qiang Liu, and Junfeng He. Training robust 2019b. Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018
deep neural networks via adversarial noise propagation. arXiv preprint
Pratyush Maini, Eric Wong, and Zico J. Kolter. adversarial robustness against the union of multiple perturbation model. International Conference on Machine Learning, 2020.
Jan Hendrik Metzen, Volker Fischer, and Bastian Bischoff. On detecting adversarial perturbations. In International Conference on Learning Representations, 2018.
11

---

Under review as a conference paper at ICLR 2021
Nicolas Papernot, Patrick Medaniel, Xi Wu, Somesh Jha, and Ananthram Swami. Distilla- tion as a defense to adversarial perturbations against deep neural networks. arXiv preprint arXiv:1511.04500, 2015.
Jonas Rauber, Wieland Brendel, and Matthias Bethge, Foolbox: A python toolbox benchmark the robustness of machine learning models. 2017.
Lukas Schott, Jonas Rauber, Matthias Bethge, and Wieland Brendel. Towards the first adversarially robust neural network model on mnist. In International Conference on Learning Representations, 2019.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. International Conference on Learning Representations, 2015.
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 2013.
Florian Tramèr and Dan Boneh. Adversarial training and robustness for multiple perturbations. In Advances in Neural Information Processing Systems, 2019.
Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robustness may be at odds with accuracy. In International Conference on Learning Representa- tions, 2019.
Jonathan Uesato, Brendan O'Donoghue, Aäron van den Oord, and Pushmeet Kohli. Adversarial risk and the dangers of evaluating against weak attacks. In International Conference on Machine Learning. 2018.
Jiayu Wu, Qixiang Zhang, and Guoxi Xu. Tiny imagenet challenge. 2017.
Cihang Xie and Alan Yuille. Intriguing properties of adversarial training at scale. In International Conference Learning Representations, 2020.
Cihang Xie, Jianyu Wang, Zhishuai Zhang, Zhou Ren, and Alan Yuille. Mitigating adversarial effects through randomization. In International Conference on Learning Representations, 2018.
Cihang Xie, Mingxing Tan, Boqing Gong, Jiang Wang, Alan L Yuille, and Quoe V Le. Adversar- ial examples improve image recognition. In IEEE Conference on Computer Vision and Pattern Recognition, 2020.
Dong Yin, Gontijo Raphael Lopes, Jonathon Shlens, D Ekin Cubuk, and Justin Gilmer. A fourier perspective on model robustness in computer vision. In Advances Neural Information Process- ing Systems, 2019.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. The British Machine Vision Conference, 2016.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. 2019.
12

---

Under review as a conference paper at ICLR 2021
A EXPERIMENTAL SETTING FOR SEPARATING BNS
We provide the experimental setting for separating BNs study in Section 3.1 Specifically, we train a DNN on CIFAR-I0 with each BN layer composed of 4 branches (i.e., we replace each BN with BN branches). During training, we construct different mini-batch for clean, ε,, l2, and &oo adversarial images to estimate the normalization statistics of each branch of BN; during inference, given mini-batch of data from a specific domain, we manually activate the corresponding BN branch and deactivate the others. We update the running statistics of each BN branch separately but optimize the model parameters using the sum of the four losses.
B PROOF IN SECTION 3.1
Proof. We first calculate the optimal perturbation & in the Ip ball of radius c. For simplicity, we assume that the classification loss is given by (x) -x)) where h is non-decreasing function. Note that this is a broad family of losses including the hinge loss and logistic loss. The objective of the adversary is to find a perturbation d² which maximizes the loss. For the linear classifier, 8 can be explicitly calculated follows
8
arg max S1
= arg 1ol1 M-y/(x
arg max &| yw"
arg máx =
e0 ywllo
where 1/p + 1/g = 1 and the last equality uses the property that 0||x||, = arg | 8f Notice that the optimal perturbation 52 is independent of the inputs corresponding to particular la- bel. Therefore, the new distribution Dy of the adversarial examples generated from the perturbation type S can be written as DD + c0 - -ywllp where ID represents the clean example domain.
For p = or = co, is easy to verify that 0| - ywll, = (0,0, 0, ... ... where i = arg maxy |uyl Ø|| ywll, (sign(-yvn respectively.
sign(-ywg), ywu),, ... sign(-ywa)
For 2, Thus, we have the Wasserstain distance between ( and D as
simply obtain that 0|| ywll2 = (-yw;/llulla
ywa/llul22)
W2(DS D&*) WD y D call ywlli) =
where the last equality follows from the translation invariance of Wasserstein distance. Similarly, the Wasserstein distance between D and 1D4 is
(2 1233) = ev/2 20-22pe
ww ////////l...
Finally, the Wasserstein distance between D and - can be calculated
W2(DD DE=),,D=>>>>)) = / / + 2| w | w w/
C GBN IMPLEMENATION DETAILS
Network architecture. We first illustrate the architecture of the gated sub-network g in GBN. As shown in Figure 4 we devise two types of gated sub-network, namely Conv gate and FC gate.
The notation Conv2d(d, s, k) refers to a convolutional layer with & filters whose size are convolved with stride The notation ReLU(-) denotes the rectified linear unit used as the activation function in the network. FC(m) denotes a fully connected layer with output size m.
Conv gate is convolutional neural network consisting of two convolutional layers, two ReLU layers and one fully-cone colne-coneected layer. c denotes the input channel of this gate. In the default setting, we stride for the first convolutional layer and for the second. For the filter sizes in both layers,
13

---

Under review as a conference paper at ICLR 2021
padding is set to and I, respectively. We set the fully-connected layer output size m=4 to match the four data distributions used in our experiment.
FC gate is simple folne-connected neural network that includes two fully-connected layers and one ReLU layer. We use m=512 and m=4 for the two fully-connected layers.
Conv2d(3, 1, c)
ReLU() Conv2d(3, 2, 2*(c ReLU()
FC(512)
ReLU()
FC(4)
FC(4)
Softmax()
Softmax()
(a) Conv gate
(b) FC gate
Figure The architecture of the gated sub-network used this paper.
To train models containing GBN, we add the GBN block into all layers within model. Specifically, we set the g in the first GBN as the Conv gate and use the FC gates for other GBN blocks further capture domain-specific information and improve model robustness. We use the default values for and a in PyTorch.
Discussion. To empirically prove the effectiveness of the above strategy, we conduct additional experiments using the Conv gate and FC gate for all GBN blocks. In other words, we train a VGG- 16 model with all GBN blocks using the Conv gule denoted "Conv..i" " and train another model with all GBN blocks using the FC gate denoted "FCau". As shown in Table 2 our strategy (denoted "Conv+FC") achieves the greatest robustness. Thus, use Conv gate for all the GBN blocks in the single layer study, and use Conv gate for the first GBN and FC gates for the other GBN blocks in the layer group study (Section
We conjecture that there are two reasons for this: (1) the rumning statistics between different domains in the first layer are almost indistinguishable (c.f. Appendix E.I0). Thus, solely using the FC layer (FC gate) fails to extract sufficient features from the first layer to perform correct classification; (2) using conv layers for all GBN blocks may suffer from over-fitting problem, since only adding GBN into the first layer achieves considerable robustness (as shown in Figure (a)). We will further address it in the future studies.
Table 2: Model robustness of VGG-16 on CIFAR-10 (the higher the better).
Conv
FCal
Conv+FC
PGD-41
30.9%
35.5%
58.5%
PGD--2
29.9%
34.6%
69.5%
PGD-
20.6%
32.6%
60.1%
Clean accuracy
41.6%
39.6%
80.7%
D EXPERIMENTAL SETTINGS
Dataset details. MNIST is dataset containing 10 classes of handwritten digits of size 28 x 28 with 60,000 training examples and 10,000 test instances. CIFAR-10 consists of 60.000 natural scene color images in 10 classes of size 32 x 32. Emng-emaeenet has 200 classes of size 64 x 64, and each class contains 500 training images, 50 validation images, and 50 test images.
Implementation details. Here, we provide the details of the experimental settings for GBN and other baselines on different datasets. All the models for each method on MNIST, CIFAR-10, 10, and Imag-em are trained for 40, 40, and 20 epochs, respectively. We set the mini-batch size=64, use the SGD optimizer with weight decay 0.0005 for Tiny- ImagNNe and use no weight decay for MNIST and CIFAR-10. We set the learning rate 0.1 for MNIST and 0.01 for CIFAR-10 10 and
14

---

Under review as a conference paper at ICLR 2021
Imag-mmaeNNet. For these baselines, use the published implementations for ABS MAX/AVC M and TRADES
D.1 ADVERSARIAL. ATTACKS
In this paper, adopt both the white-box and black-box adversarial attacks. In the white-box setting, adversaries have the complete knowledge of the target model and can fully access the model; while in the black-box setting, adversaries have limited knowledge of the target classifier (e.g., its architecture) but can not access the model weights.
For l attacks, use PGD attack (Madry et... et al. 2019).
2018). and Brendel & Bethge attack (BBA) Brendel
For l2 attacks, Gaussian noise attack (Rauher et al. 2017). and boundary attack (BA) (Brendel et al. 2018).
PGD attack (Madry et al. 2018). C&W attack (Carlini & Wagner 2017
For for attacks, though fo PGD adversary is quite effective, for completeness, we additionally use attacks including, Fast Gradient Sign Method (FGSM) (Goodfellow et al., (2014), SPSA (Uesato al. 2018), Nattack (Li et al., 2019), Momentum Iterative Method (MI-FGSM) (Dong al. 2018). and AutoAttack (Croce & Hein (0200)) (a stronger ensemble of parameter-free free attacks, including APGD, FAB (Croce Hein 2020c), and Square Attack (Andriushchenko et al. 20200)
Among the attacks mentioned above, BA, SPSA, Nattack, and Square Attack are black-box attacks. We then provide the
of each attack.
PGD-L1. On MNIST, we set the perturbation magnitude ==10, iteration number k=50, and step size a=//10. On CIFAR-10 and ImageNn, we set the perturbation magnitude c=12, iteration number k=50, and step size a=0.55.
PGD-L2. On MNIST, we set the perturbation magnitude c=2.0, iteration number k=100, and step size 0=1.1. On CIFAR-10 and Iiny-ImageNet, we set the perturbation magnitude c=0.5, iteration number k=50, and step size a=//10.
PGD- On MNIST, we set the perturbation magnitude c=..3, iteration number k=50, and step size a=0.01. On CI and Imay- we set the perturbation magnitude e=0.03, iteration number k=40, and step size a==/10.
PGD-1000-4.. On CIFAR-10, we set the perturbation magnitude c=0.03, iteration number k=1000, and step size o=1/10.
BBA. On MNIST, CIFAR-10 and Tiny-ImageNet, we set the perturbation magnitude e=10, 12, 12 in terms of é norm, respectively. For all datasets, we set the number of optimization steps as 1000, learning rate as 0.001, momentum as 0.8, and the binary search steps as 10.
C&W-E2. On MNIST, 10 and Tiny-Image we set the perturbation magnitude c=2.0, 0.5, 0.5 in terms of le norm, respectively. For all datasets, we set the number of optimization steps as 10000, each step size 0.01, and the confidence required for an example to be marked as adversarial as O.
C&W- On CIFAR- 10, set the perturbation magnitude c=0.03 in terms of loc norm. We set the number of optimization steps as 10000, each step size as 0.01, and the confidence required for an example to be marked as adversarial
Gaussian noise. On MNIST, CIFAR-10 and Tmay-ImageNet, we set the perturbation magnitude <2.00, 0.5, 0.5 in terms of lg norm, respectively.
BA. On MNIST, CIFAR-10 and Tiny-ImageNet, we set the perturbation magnitude c=2.0, 0.5, 0.5 in terms of la norm, respectively. For all datasets, set the maximum number of steps as 25000, initial step size for the orthogonal step as 0.01, and initial step size for the step towards the target as 0.01.
15

---

Under review as a conference paper at ICLR 2021
FGSM. On MNIST, we set the perturbation magnitude c=0.3 in terms of foe norm. On CIFAR-10 and line-lmageNet, we set the perturbation magnitude c=0.03 in terms of &o norm.
MI-FGSM. On MNIST, we set the perturbation magnitude c=0.3, the decay factor p=1 in terms of lo norm, and set the step number k=10. On CIFAR- and Tiny-ImageNet, we set the perturbation magnitude c=0.03, the decay factor μ=1 in terms of &oo norm, and set the step number k=10. The step size a=//k.
SPSA. We set the maximum iteration 100, the batch size 8192, and the learning rate 0.01. The magnitude of perturbation is 0.3 in terms of (o norm on MNIST, and 0.03 in terms of &n norm on CIFAR-10 and Imag-em.
NATTACK.
We set 7=600 as the maximum number of optimization iterations, b=300 for the sample size, variance of the isotropic Gaussian o²=0.01, and learning rate as 0.008. The magnitude of perturbation is 0.3 in terms of &o norm on MNIST, and 0.03 in terms of &o norm on CIFAR-10 and Tiny-ImageNet.
AttkA AutoAttack selects the following variants of adversarial attacks: APGD without random restarts, APGDDLR. the targeted version of FAB as FAB", and Square Attack with one run of 5000 queries. We use 100 iterations for each run of the white-box attacks. For APGD, we set the momentum coefficient a=0.75, p=0.75, initial step size =2e) where is the perturbation magnitude in terms of the lo norm. For FAB, we keep the standard hyper-parameters based on the implementation of AdverTorch. For Square Attack, we set the initial value for the size of the squares p= 0.8.
D.2 ADVERSARIAL DEFENSES
ABS. ABS uses multiple variational autoencoders to construct a complex generative architecture to defend against adversarial examples in the MNIST dataset. As for the limited code released by the authors, directly use the results reported in Schoot et al. (2019). They set the perturbation magnitude c=12, 1.5, 0.3 for fo, &,, and foo attack, respectively. ABS used an fn perturbation model of higher radius and evaluated against lu attacks. So the reported number is near estimate of the l adversarial accuracy.
AVG. For each batch of clean data (size=64), we generate corresponding adversarial examples (61, f2, and loo) using PGD attack. We train the model using a combination of clean, f,, E2, and loc adversarial examples simultaneooully The syper-parameters of PGD adversaries can be found in Appendix D.I (PGD-L1, PGD-22, and PGD-f..).
MAX. For each batch of clean data (size=64), we generate the strongest adversarial examples (one of the 4, f,, and lx attack) using PGD attack. We train the model using a combination of clean examples and the strongest attack. The hyper-parameters of PGD adversaries can be found in Ap- pendix D.I (PGD-L1, PGD-E,, and PGD-
MSD. MSD creates single adversarial perturbation by simultaneously maximizing the worst-case loss over all perturbation models at each projected steepest descent step. For MNIST, we set the perturbation magnitude c=10, 2.0, 0.3 for f1, f2, and foo attack, respectively, and iteration number k=100. For CIFAR-10 and Timy-ImageNet, we set the perturbation magnitude e=12, 0.5, 0.03 for l,, f2, and loo attack respectively, and iteration number k=50.
TRADES. TRADES is an adversarial defense method trading adversarial robustness off against accuracy which won 1st place in the NeurIPS 2018 Adversarial Vision Challenge. We set 1/A=3.0 and set other parametrssss the default values following the original paper.
Pt. We adversarially train the model with 50% clean examples and 50% adversarial examples generated by & PGD attack for each mini-batch data. For PGD attack, on CIFAR-10, we set the perturbation magnitude c=12, iteration number k=50, and step size a=0.05.
Pz. We adversarially train the model with 50% clean examples and 50% adversarial examples generated by & PGD attack for each mini-batch data. For PGD attack, on CIFAR-10, 10, we set the perturbation magnitude e0..5 iteration number k500, and step size o=e/10.
16

---

Under review as a conference paper at ICLR 2021
Poo. We adversarially train the model with 50% clean examples and 50% adversarial examples generated by fo PGD attack for each mini-batch data. For PGD attack, on CIFAR-I0, we set the perturbation magnitude c=0.03, iteration number k=40, and step size a=e/10.
D.3 NORMALIZATION TECHNIQUES
MN. We set the number of modes in MN to 2, which achieves the best performance according the original paper (Deekke et al. 2019). During training, we feed the model with a mixture of clean, é1.
l2. and &o adversarial examples using the same setting AVG.
MBN. The original MBN (Xie et al. 2020 Xie & Yuille 2020) manually selects the BN branches for clean and adversarial examples during inference, which is infeasible in adversarial defense setting (the model is unaware of the type of inputs). Thus, we add the 2-way gated sub-network in MBN to predict the input domain label; we then keep the following 2 BN branches the same. During training, we compel the clean examples to go through the first BN branch and the adversarial examples (i.e., l, f2, and (a) the second BN branch. The adversarial examples are generated via PGD using the same setting as AVG.
E MORE EXPERIMENTAL RESULTS
In this section, we provide further experimental results.
E.1 MULTIPLE PERTURBATIONS FROM THE FOURIER PERSPECTIVE
100
100
100
100
3
80
40
40
40
20
20
Figure 5: Model sensitivity additive noise aligned with different Fourier basis vectors on CIFAR- 10. From left to right: vanilla, E fitrrined, l2-trained, and fx -trained models. The numbers indicate model error rates.
We study the differences between different perturbation types from the Fourier perspective Yin jtt al. 2019). We evaluate four different ResNet-20 models: a vanilla model and 3 adversarially. trained models using only L1, l2, and foo adversarial examples, respectively. We visualize the Fourier heatmap following (Yin et al.] 2019) and shift the low frequency components to the cen- ter of the spectrum. Error rates averaged over 1000 randomly sampled images from the test on CIFAR-10. The redder the zone, the higher the error rate of the model to perturbations of spe- cific frequency. As shown in Figure models trained using different l, perturbations demonstrate different model weaknesses in the Fourier heatmap (i.e., different hot zones). For example, PAT more susceptible to high-frequency perturbations while less susceptible some low-frequency perturbations (the smaller blue zone in the center), while -PAT is comparatively more robust to frequeny perturbations (light yellow zones). Therefore, different perturbation types have different frequency properties, and models trained different perturbations are sensitive to noise from different frequencies. This observation further demonstrates that different perturbation types may arise from different domains (our multi-doman hypothesis).
E.2 COMPUTATIONAL COST
Here, provide the time consumption of AVG, MAX, MSD, MN, MBN, and GBN. All the experiments are conducted on NVIDIA Tesla V100 GPU cluster. We compute the overall training time using ResNet-20 CIFAR-10 for 40 epochs. We compute the inference time using ResNet-20
17

---

Under review as a conference paper at ICLR 2021
CIFAR-10 for 10000 images. As seen in Table our method achieves comparable results to other baselines, demonstrating its applicability practice.
Table 3: Runtime analysis of different methods on CIFAR-10 (the lower the better).
AVG
MAX
MSD
MN
MBN
GBN(ours)
Training speed
(seconds/ epoch)
1837.9
1113.2
689.8
19329.1
2737.9
2802.6
Training duration (hours)
20.4
12.4
7.7
214.8
30.4
31.1
Inference speed
(seconds/ 10000 images)
0.60
0.58
0.61
4.8
1.9
2.0
E.3 ADVERSARIAL ROBUSTNESS AGAINST MULTIPLE PERTURBATIONS
In this part, we provide the breakdown for each individual attack on MNIST, CIFAR-10, 10, and Tiny- ImageNet in Tabel and 6 We also report the results for individual attacks of AutoAttack in Further, we provide the breadkdown for each individual attack on CIFAR-10 10 using VGG-16 16 and WideResNet-28-111 and Table
9
According to the results, our GBN outperforms other methods for almost all attacks by large mar- gins. However, is reasonable to notice that our GBN shows slightly weaker comparable perfor- mance on some individual attacks compared to defenses trained for the specific perturbation types. For example, Pi outperforms GBN for PGD-é, on CIFAR-IO and TRADES shows better perfor- mance for some fx.. attacks on MNIST.
Table 4: Model robustness of LeNet on MNIST over each individual attack (the higher the better).
Vanilla
TRADES
AVG
MAX
ABS
MSD
MN
MBN
GBN
f attacks
PGD-é BBA
74.4% 6.55
80.8% 10.2%
77.9% 31.2%
77.3%
77.0%
80.4%
79.6% 64.5%
86.8%
33.1%
42.6%
25.25
79.6%
PGD-4z
24.7%
90.1%
73.5%
74.6%
69.5%
77.2%
78.5%
97.8 %
Eg atlacks
CRW-éa
2.0%
42.5%
68.1%)
67.6%%
72.44%
20.0%
67.3%%
98.1 %
Gaussian Noise
99.2%
98.5%
98.7%:
98.25
98. 98.0%
97.3%
97.6%%
98.6%
99.2
BA
10.5%
12.1%
60.9%
59.2%
83.0%
78.7%
24.2%
90.2%
97.7 %
PGD-f.c
64.2%
96.1%%
72.6%
74.7%
51..6%
78.9%
80.3%
96.3 %
FGSM
48.7%
96.9%%
87.5%)
86.3%
34.0%
69.7%%
97.5%
75.44%
89.5%
MI-GGSM
36.2%
96.8%
82.11%
81.5%
17.0%
60.9%
90.7%
40.7%
85.2%
oo
SPSA
37.6%
93.3%
64.2%
60.5%
71.4%
77. 8%
77.8%
98.2 %
NATTACK
91.7%
79.55
85.2%
90.2%
89.3%
97.7% %
AutlAAtack
0.0%
77.1% 1%
46.1%
60.15
39.0%
0.0%%
20.0%
71.7%
All attacks
0.0%
9.8%
29.7%
31.2%
17.0%
38.9%
0.0%
19.2%
68.9%
accuracy
98.1%
In summary, our proposed GBN trains robust models in terms of multiple perturbation types (i.e.,
e,, 82, loo) and outperforms other methods by large margins.
E.4 ATTACKS WITH DIFFERENT ITERATION STEPS
Here, we further provide the experimental results of PGD adversarial attacks using different iteration step numbers (i.e.. 50, 100. 200, 1000) in terms of Eo nomm CIFAR-10. For PGD attacks, set the perturbation magnitude c=0.03, step size a=e/10, and different iteration steps k. As shown in "able 1O GBN outperforms other methods on PGD attacks using different iteration steps.
18

---

Under review as a conference paper at ICLR 2021
Table 5: Model robustness of ResNet-20 on CIFAR-10 over each individual attack (the higher the better).
Vanilla
Ps
f attacks
PGD- BBA
31.7%
25.7%
70.3%
39.8%
18.7%
47.4%
38.1%
49.9%
44.1%
47.7%
58.5%
0.0%
16.2%
22.4%
34.2%
18.0%
45.0%
33.4%
43.8%
40.0%
45.8%
69.1 %
PGD-22
32.1%
19.8%
8%
49.4%
60.5%
69.5%
C&W-(2
0.0%
59.5%
4.8%
51.2%
50.4%%
59.7%
56.0%
62.5%
30.1%
20.8%
74.9%
Eg
Gaussian Noise
81.2%
79.8%
68.9%
84.8%
81.6%
76.4%
62.9%
69.1%
60.2%
49.5%
19.1
0.3%.
60.2%
1.2%
62.3%
58.4%.
62.0%
57.8%
59.9%
39.1%
47.8%
73.2%
PGD-1000-
0.0 %
48.1%
18.4%
44.7%
40.4%
37.66%
42. 6%.6
32.8%
FGSM
9.2%
55.2%
30.4%
46.7%
54.3%
39.9%
42.7%
46.4%
40.1%
62.4%
56.0%
3o attacks
MI FGSM
C&W-F...
0.0%
2.0%
53.7%
53.9%
0.1%
0.9%
40.9%
30.2%
51.3%
50.8%
47.8%
37.8%%
44.3%
31.6%%
49.3%
45.3%%
42.1%
31.2%
51.1%
600.1%
72.9%
67.2%%
SPSA
0.4%
50.2%
0.6%
19.4%
45.8%
33.2%
30.4%
44. 9%
38.4%
68.2%
69.8
19.7%
AutoAttack
0.0%
45.1%
0.0%
20.25
42.3%
29.3%
25,4%
38. 16
13.2%
40.8%
50.99% %
All attacks
0.0%
16.0%
0.0%
19.0%
17.8%
28.2%
24.9%
37.9%
13.0%
20.7%
48.7%
accuracy
87.9%
82.3%
79.4%
80.7%
Table 6: Model robustness of ResNet-34 on Tiny-ImageNet over each individual attack (the higher the better).
Vanilla
AVG
MAX
MSD
MN
MBN
GBN
attacks
PGD-E, BBA
10.0%
28.1%
17.1%
9.0%.
12.4%
45.2% 37.0%
55.7%
5.8%
27.5%
26.2%
7.6%
8.8%
45.2%
PGD
12.3%
30.5%
24.4%
14.2%
18.1%
31.7%
$3.9
C&W-
10.9%
29.
11.2%
19.2%
31.4%
Eg altacks
Gaussian Noise
54.2%
41.6%
36.3%
26.7%
40.2%
46.2%
42.8%
BA
27.1%
35.2%
24.9%
13.7%
30.2%
38.4%
44.1
PGD-
7.1%
6.9%
4.7%
7.6%%
19.8%
40.2%
$00.2%
FGSM
3,4%
11.1%
8.9%
22.4%
41.4%
500.3%
MI-FGSM
1.4%
9.4%
6.0%
8.0%
15.1%%
36.3%
53.7%
attacks
SPSA
0.3%
7.1%
6.5%
10.0%
89
39.2%
$2.0
NATTACK
0.8%
8.2%
7.35%
12.3%
29.9%
AutoAitack
0.0%
4.6%
2.9%
5.1%
6.8%
20.1%
39.8%
All attacks
0.0%
4.6%
2.2%
5.0%
6.7%
20.0%
38.9%
ascuracy
46.7%
E.5 PREDICTION RESULTS OF THE GATED SUB-NETWORD
In this part, we show the prediction results of the gated sub-network in GBN, i.e., the classification accuracy for the domain of different input samples. The results LeNet on MNIST and ResNet-20 on CIFAR-10 10 are presented in Table IT For MNIST, the gated sub-network of LeNet at different layers achieves high prediction accuracy for the input domains (i.e., clean, L,, l2, and foc adversarial examples). However, on CIFAR-10, the prediction accuracy of the gated sub-network drops with the increasing of the layer depth. From the results, we can also confirm our findings in Section 4.3 (1) shallow layers are more critical to model robustness; and (2) the layer depth increases, the features from different domains are highly entangled and mixed, making it harder for models to separate them. To further improve model robustness against multiple perturbations, suggest to improve the prediction accuracy of the gated sub-network. We will further address the future studies.
19

---

Under review as a conference paper at ICLR 2021
Table 7: Model robustness of VGG-16 CIFAR-10 over each individual attack (the higher the better).
Vanilla
Ps
f attacks
PGD- BBA
8%
27.9% 18.2%
72.2%
42.1%
19.8%.
48.8%
39.6%
51.2%
46.0%
49.5%
59.9%
0.0%
24.8%
36.1%
18.7%
45.9%
35.5%
44.9%
41.1%
46.8%
69.5%
PGD-22
32.9%
21.7%
64. 64.9%
70.7% %
C&W-f2
0.0%
61.1%
6.8%
52.1%
52.5%
61.9%
58.0%
64.2%
31.4%
21.9%
75.2%
Eg
Gaussian Noise
82.0%
79.9%
70.1%
86.0%
82.8%
74.9%
62.9%
69.9%
61.8%
51.2%
79.918
0.9%
61.8%
2.3%
64,4%i
60.2%:
64.1%%
59.7%
61.4%
40.2%
49.3%
74.8%
RGD-1000-f
0.0% %
49.9%
20.2%
41.0%
45.9%
42.2%
39,8%
42.5%
33.1%
56.4%
58.8%
FGSM
9.1%
56.2%
32.1%
47.7%
55.8%
41.0%
44.6%
447.3%
41.8%
63.77
57.4%
MI FGSM
0.3%
54.2%
1.2%
31.8%
51.9%
39.3%
33.4%
47.0%
32.7%
60.99%
73.8%
3o attacks
SPSA
0.6%
52.2%
0.9%%
20.4%
46.8%
34.9%
32.1%
46.
39.9%
69.8%
700.8
NATTACK
2.4%
50.4%
2.8%
19.7%
47.1%
34.3%
30.9%
45. 1%6
20.
45.4%
64.2 %
27.4%
All attacks
0.0%
17.9%
0.0%
19.5%
18.5%:
30.2%
27.0%
38.9%
14.1%
21.5%
50.6%
Clean uccuracy
90.9%
87.1%
84.9%
88.1%
85.0%
82.1%
79.2%
78.4%
80.2%
84.3%
84.1%
Table 8: Model robustness of WideResNet-28-10 on CIFAR-10 over each individual attack (the higher the better).
Vanilla
P3g
PGD-é1 BBA
24.7%
27.2%
71.4%
40.9%
19.5%
53.3%
40.0%
53.1%
46.0%
48.8%
62, 1%
f attacks
0.0%
16.8%
23.1%
35.6%
49.0%
36.1%
43.2%
42.0%
47.6%
71.8%
PGD-
27.1%
60.8%
21.8%
63.7%
61.2%
65.3%
62.6%
63.1%
51.1%
70.9%
C&W-
0.0%
60.5%
5.2%
53.0%
51.9%
62.7%
58.8%
64.1%
33.4%
24.8%
76.4
Eg
Gaussian Noise
91.2%
80.2%
69.9%%
85.9%
82.6%
78.8%
70.3%
77.8%
63.9%
54.2%
77.8%8
0.5%
60.7%
1.0%
61.4%
58.1%.
67.0%
61.4%
65.4%
42.0%
50. 1%
75.6%
PGD-1000-C.c
0.4
50.2%
18.9%
40.1%
46.5%
42.1%
40.6%
47..5%
35.1%
56.9%
60..4%
FGSM
8.7%
58.2%
32.4%
47.4%)
52.65%
39.95%
44.4%%
52.22%
42.0%
65.2%
58..1%
Fou attacks
MI FGSM
0.0%
54.2%
0.6%
30.6%
48.3%
37.8%%
36.0%
50.3%
35.1%
61.9%
SPSA
0.2%
50.1%
0.7%
19.6%
42.3%
33.2%
34.8%
.7.2%
39.9%
69.0%
70.8%
NATTACK
1.9%
50.6%
2.1%
19.7%
47.4%
34.3%
36.1%%
46. 1%
43.0%
47.9%
66.0%
52.4%
All attacks
0.0%
16.5%
0.0%
19.0%
18.8%
31.3%
30.9%
40.0%
15.6%
24.5%
51.5%
Clean accuracy
93.2%
87.6%
83.4%
88.5%
85.1%
83.29
79.15%
80.3%
83.6%
84.1%
83.9%
E.6 ALTERNATIVE PREDICTION APPROACH OF THE GATED SUB-NETWORK
In the main body of our paper, we calculate the normalized output using the gated sub-network in a soft-gated way (denoted "soft") based on Eqn. 3 In this section, we also try taking the top-1 pre- diction of (the hard label) to normalize the output an alternative approach (denoted "hard").. As shown in Table [2] the two approaches achieve comparable performance in terms of both robustness and standard accuracy.
E.7 ATTACKS OUTSIDE THE PERTURBATION MODEL.
In this section, we present some additional experiments exploring our model's performance on at- tacks which lying outside the perturbation model.
We conduct two experiments: (1) adversarially train a GBN model on & and fo perturbations, and evaluate it lg adversaries (denoted GBN ); and (2) adversarially train GBN model on l2 and &x perturbations, and evaluate on E adversaries (denoted GBN +f.).. We use PGD-É,, PGD- and PGD-Lac attacks in Appendix D.1 The results shown in Table 13.
20

---

Under review as a conference paper at ICLR 2021
Table 9: Robustness evaluation by AutoAttack (the higher the better). We report the clean accuracy, the robust accuracy of the individual attacks as well as the combined one of AutoAttack (denoted AA) using LeNet, ResNet-20, and ResNet-34.
Vanilla
TRADES
P.
P2
Pos
AVG
MAX
MSD
MN
MAN
GBN
MNIST &.
APGDEE
74.7%:
APGDDLR
0.0%
82.1%
41.2%
55.7%
33.9%
0.0%
52.2%
88.4%
FAB"
0.2%
85.0%
52.1%
60.1%
37.3%
0.0%
80.7%
96.15
0.0%
AA
0.0%
77.1%
46.1% 1
60. 1%
39.0%
0.0%
20.0%
71.7%
CIEAR-10-
0.0%
49.1%
0..0%
20.6%
44.9%
30.8%
29.9%
44.3%
20.8%
53.5%
60.8%
APGD DLR
0.0%
50.2%
0.0%
18.8%
43.2%
28.1%
25.7%
39.1%%
29.9%
62.4%
58.9%.
FAB"
0.0%
49.8%
0.8%
19.6%
41.2%
29.6%
26.3%
39.3%
30.1%
40.9%
70.2%
Square
0.1%
47.4%
4.6%
28.45
39.0%%
40.5%
31.6%%
46.1%
17.8%)
14.5%
60.3%
AA
0.0%
45.1%
0.0%
20.2%
42.3%
29.3%
25.4%
38.1%
13.2%
40.8%
50.9%.
Tiny-ImageNel
A
0.0%
5.7%
3.7%
6.3%
8.7%
39.3%
44.3%
4.2%
10.45
FAB"
0.2%
5.2%
2.7%
4.3%
9.1%
37.3%
50.0%%
Square
0.7%
10.1 1%
4.0%
5,91%
6.4%
29.3%
39 39.4%
AA
0.0%
4.6%
2.9%
5.1%
6.8%
20.1% 1%
39.8%
Table 10: Model performance of ResNet-20 on CIFAR-10 on different PGD-k attacks (k denotes the iteration steps).
Vanilla
TRADES
AVG
MAX
MSD
MN
MBN
GBN(ours)
PGD-50
2.4%
49.8%
42.1%
39.5%
44.3%
34.6%
58.6%
60.0%
PGD-100
0.0%
49.5%
41.7%
39.2%
44.1%
34.1%
58.2%
59.6%
PGDD2000
0.0%
49.3%
41.4%
38.7%
43.5%
33.6%
57.7%
59.0%
PGD-1000
0.0%
48.1%
40.4%
37.6%
42.6%
32.8%
55.4%
58.1%
We empirically observe that training on 2 perturbation types actually improves overall robustness against all three perturbation adversarics. Unsurprisingly, the robustness of GBN, for lz ad- versarial examples and the robustness of GBN for & adversarial examples decrease some degree, but the models still perform better than vanilla, and sometimes even outperform baseline methods.
In addition, we further evaluate model robustness on attacks outside the perturbation model using different prediction approaches of the gated sub-network (i.e., soft label and hard label). As shown in Table [4] we achieve the similar results using the soft or hard label.
E.8 DIFFERENT VARIANTS OF WHITE-BOX ATTACKS
Here, we examine the performance of GBN more rigorous white-box attack scenarios. In this case, the adversary not only knows every detail of the GBN architecture, but also can manually select the BN branch to generate adversarial attacks.
We first generate adversarial attacks by fooling all the GBN layers in the model. Specifically, we generate adversarial attacks using PGD-(, PGD-(2, and PGD- attacks in Dopendix[D.I]pp mak- ing the gated sub-network to wrong predictions of the input domain. We force the generated per- turbations to fool all the GBN layers in a ResNet-20 model at the same time. In particular, we first generate adversarial examples for different perturbation types using ResNet-20 model; based these examples for each domain, we then perturb them to fool the GBN layers. As shown in Table GBN still achieves high robustness against such adaptive attack. To further investigate this 15
21

---

Under review as a conference paper at ICLR 2021
Table 11: Gated sub-network prediction accuracy inputs from different domains different datasets (the higher the better). We demonstrate the prediction accuracy of gated sub-network at each layer from top bottom. The adversarial examples are generated by PGD-L1, PGD-12, and PGD- attacks in Appendix D.1
(a) LeNet on MNIST
Clean
8
to0
gate
96.7%
78.2%
92.3%
98.6%
gatez
94.2%
74.5%
89.9%
98.8%
(b)
ResNet-20 on CIFAR-I0
Clean
f2
8o0
gate1 79.7%
85.2%
88.7%
84.8%
gate2
77.2%
84.5%
85.9%
83.7%
gatez
77.4%
84.2%
82.3%
83.9%
gate4
75.2%
83.5%
75.2%
82.1%
gate;
74.7%
82.2%
73.4%
81.6%
gates
73.2%
82.5%
69.1%
79.8%
gate,
71.7%
81.2%
67.9%
78.3%
gates
65.2%
74.5%
67.2%
75.7%
gate,
64.7%
74.2%
66.8%
75.5%
gate 64.2%
73.6%
65.7%
74.8%
gate 64.7%
73.2%
64.6%
74.5%
gate12 64.2%
72.9%
64.4%
73.9%
gate13 63.9%
71.2%
64.1%
73.7%
gate 63.0%
69.5%
64.0%
72.3%
gate
64.1%
69.1%
64.2%
71.6%
gate1e
64.2%
68.7%
63.9%
70.9%
gate17 62.5%
68.2%
63.9%
70.6%
gate1s 63.2%
68.5%
63.6%
69.5%
gate19
62.7%
67.9%
63.4%
69.4%
Table 12: Model robustness of ResNet-20 CIFAR-10 using different prediction approaches of the gated sub-network (the higher the better). We use PGD-E,, PGD-12, and PGD-L attacks in Appendix D.I
vanilla
hard
soft
PGD-4
31.4%
58.2%
58.5%
PGD-&
32.1%
69.6%
69.5%
PGD-Ex
3.2%
59.9%
60.1% 1%
Clean accuracy
89.7%
80.4%
80.7%
Table 13: Attacks outside the perturbation model of ResNet-20 CIFAR-10 (the higher the better).
Vanilla
AVG
MAX
MSD
GBN
GBN,, +1 GBN
PGD-( 31.7%
47.4%
38.1%
49.9%
58.5%
57.4%
40.9%
PGD-(2
32.1%
59.2%
58.0%
62.8%
69.5%
41.8%
65.4%
PGD-Ex
3.2%
42.1%
39.7%
44.5%
60.1%
57.2%
56.8%
Clean accuracy
89.7%
80.6%
77.0%
79.1%
80.7%
81.3%
81.1%
22

---

Under review as a conference paper at ICLR 2021
Table 14: Attacks outside the perturbation model of ResNet-22 CIFAR-10 using different pre- diction approaches of the gated sub-network ("h" denotes the hard label and "s" denotes the soft label).
GBN (s)
GB (s)
GBN, +e (h)
GBN (h)
PGD-&1 PGD-&z
57.4%
40.9%
57.9%
41.5%
41.8%
65.4%
40.2%
67.9%
PGD-f
57.2%
56.8%
58.2%
56.1% %
Clean accuracy
81.3%
81.1%
81.2%
80.9%
phenomena, also provide the prediction accuracy of the gated sub-network in GBN different layers given adversarial examples above. As can be found in Table it hard to fool all GBN lay- within model. In other words, though some GBN layers fooled misclassify the domains of the samples, some other GBN layers could still keep comparatively high prediction accuracy.
We then generate adversarial examples by manually selecting the BN branches, Specifically, we generate 4,, f2, and &oo adversarial examples through BN", BN BN and BN in GBN, respec- tively. The that their statistics are estimated by clean, l1. l2, and foc adversarial examples during training, re- spectively. presented in Table 17 the model achieves good robustness on different datasets in this more rigorous white-box attack scenario. In particular, the worst-case accuracies among all branches are largely comparable to the results presented in Table [ suggesting that the robustness of GBN does not rely obfuscated gradients (Athalye al. 2018).
for PGD-&1, PGD-(2, and PGD-& attacks in Appendix D.1 Note
Table 15: White-box attacks on CIFAR-10 using ResNet-20 by fooling all the GBN layers.
BNO
BN
BN2
BN3
73.2%
71.4%
72.4%
71.8%
E.9 VISUALIZATION OF FEATURES
Here, we further provide the visualization of features before and after the normalization blocks of each method using t-SNE. Specifically, given 100 CIFAR-I0 clean samples from specific class (e.g., "dog"") we first generate corresponding f, lz, and l. adversarial examples; we then visualize the features before and after the normalization blocks (i.e. the BN block for AVG and the GBN block for our method). Note that we use the hyper-pan aneer-parmmeeers of PGD-L, PGD-L2, and PGD- attacks in Appendix D.I As can be seen in Figure the features of different domains are aligned domain- invariant representations after normalized by GBN. Specifically, after the normalization of GBN, one clean example (the red dot) and its corresponding adversarial examples (the corresponding blue, green, and yellow dots) more likely to be close in the space of normalized output (Figure (6()). However, after the normalization of BN (Figure 7 the distance between the clean example and its corresponding adversarial examples are still far, and cannot figure our any tendency between Figure 7(a) and Figure 7(b)
E. MORE DETAILS OF THE RUNNING STATISTICS OF DIFFERENT MODELS
model with the multiple BN branch structure and inspect the running statistics of each BN at different layers, as shown in Figure and Figure Surprisingly, the running statistics between different domains in the first layer are almost indistinguishable. How- according to the ablation study in adding GBN into shallow layers is the most robust than adding GBN to other layers. We may understand it from three following viewpoints: (1) gated net-network prediction accuracy. We provided the prediction accuracy of the gated sub-network inputs from different domains in Table IT According to the experimental results, the gated sub- network achieves the highest prediction accuracy in the first layer, though the running statistics in
We train VGG-16 and WideResNet-228
23

---

Under review as a conference paper at ICLR 2021
Table 16: Gated sub-network prediction accuracy adversarial examples generated fool the GBN layers. We demonstrate the prediction accuracy of gated sub-network at each layer from top bottom.
Clean
E0o
gate1 69.7%
48.2%
54.7%
70,2%
gatez
70.2%
50.3%
58.2%
73.3%
gates
70.4%
51.4%
57.3%
72.1%
gate4
71.1%
50.1%
56.4%
74.0%
gates
68.2%
50.9%
53.1%
73.6%
gates
67.4%
49.0%
50.2%
73.5%
gate7
65.1%
49.2%
51.0%
72.4%
gates
60.3%
48.3%
49.2%
70.5%
gateg
60.8%
46.1%
49.3%
69.8%
gate1o 60.5%
44.0%
48.2%
69.1%
gate11 62.2%
42.1%
46.2%
69.4%
gate 60.2%
41.1%
46.3%
67.3%
gate13 61.9%
40.3%
45.1%
66.8%
gate14
58.1%
40.4%
44.2%
66.9%
gate15
59.2%
40.2%
44.4%
65.4%
gate16 59.3%
41.3%
41.5%
63.2%
gate17 54.4%
41.4%
42.9%
60.1%
gate18 54.1%
40.4%
41.4%
60.5%
gate 54.2%
40.0%
40.3%
60.0%
Table White-box attacks on MNIST and CIFAR-10 by manually selecting BN branches gen- erate adversarial examples (BN", BN BN2, and BN denotes the BN branch for clean, L1, l2, and loo adversarial examples in GBN, respectively).
LeNet MNIST
BN
BN
BN2
BN3
PGD-4
86.8%
88.5%
86.9%
97.1%
PGD-(2
97.8%
98.9%
97.8%
98.7%
PGD-fo
97.1%
98.6%
96.3%
95.4%
Clean accuracy
98.7%
98.7%
98.7%
98.7%
(b) ResNet-20 on CIFAR-10
BN
BN3
BN2
BN
PGD-E,
58.5%
56.4%
59.9%
60.3%
PGD-&2
69.7%
69.4%
68.2%
69.5%
PGD-fco
60.3%
61.1%
61.3%
59.2%
Clean accuracy
80.7%
80.7%
80.7%
80.7%
the first layer are almost indistinguishabl.. (2) the convolutional layers in the gated sub-networt may still be able to distinguish the differences between these features (see discussions Appendix (). (3) adding GBN to the first layer shows better performance than solely adding GBN to deeper layers might because those shallow layers are more critical model robustness Liu al. (2019h). We will study it in future work.
24

---

Under review as a conference paper at ICLR 2021
Dean
before GBN block
(b) after GBN block
Figure 6: We use the GBN block at layer conv3_1 of a VGG-16 model. (a) demonstrates the visualization of features before the GBN block; (b) shows the visualization of features after the GBN block.
Gean
before BN block
(b) after BN block
Figure 7: We use the BN block at layer conv3 1 of a VGG-16 model trained using AVG. (a) demon- strates the visualization of features before the BN block; (b) shows the visualization of features after the BN block.
25

---

Under review as a conference paper at ICLR 2021
Clean
-20
12
chamnal
charnal
(a) running of layer convl.
(h) running variance of layer cone
30
channel
shannel
(c) running mean of layer comv/2_
(d) running variance of layer conv2_2
2.5
vari
1.0
0.5
(c) running mean of layer conu3.1 1
(f) running variance of layer corve3 .1
3.12
5.53
1.54
channal
(g) running mean of layer core4_1
(h) running variance of layer corve__1
Figure 8: Running statisties (running mean and running variance) of each BN in the multiple BN branches at different layers of VGG-16 model CIFAR-10.
26

---

Under review as a conference paper at ICLR 2021
Clean
chamnal
charnal
(a) running of block llayerI.bnl
(b) running variance of layer block I. .layerl.bm
-39
2
channel
channel
(c) running mean of layer block .layerl.b22
(d) running variance of layer block1 I. .laeer..b22
(c) running mean of layer block2.layerr.bbr
(f) running variance of layer block2.layere Iny
f
channel
3 running mean of layer Jlack3.layrr...m lay.rI..blll 1bnl
(h) running variance of layer block3.
Figure 9: Running statistics (running mean and running variance) of each BN in the multiple BN branches at different layers of WideResNet-28- model on CIFAR-10.
27


Reviews:
[{"review_id": "tT3Vj-Ps4Tt", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Existing defenses are usually robust against certain types of attacks. This paper proposes to build a defense against different types of norm-bounded attacks via gated batch normalization. This work treats different types of adversarial attacks as data from different domains and proposes to use gated batch normalization to improve the robustness across different domains. The idea is interesting. The result shows the effectiveness of the defense. \n\nWeakness and Concerns:\n\n1. Adaptive attack for the GBN layer is missing. For example, one possible attack is to attack the GBN layer directly. Say the L_inf attack's aim is to fool the GBN layer, such that the gate output is misled from dealing with the L_inf but the L_0, the defense should fail. The reviewer worries that the robustness is not measured correctly, maybe rely on the obfuscated gradient [5].\n\n2. No comparison to the STOA method. For example, the L_inf on LeNet MNIST does not compare to the TRADES [2] paper. On CIFAR-10, the author should also provide results on the WRN model [3] given the rich literature using it. Also, the standard CW and PGD attack for L_inf attack is not provided here. The paper should also compare to [4], as it also builds robustness against different kinds of norm bounded attacks. \n\n3. Can the author list exactly how many steps they used for each attack evaluation? Does the paper use the best attack algorithm for each norm bounded attack? For example, it is known that CW/PGD is effective for L_inf attack. \n\n4. Theory only considers the linear case.\n\n5. Can the author compare to baseline model TRADES [2]? Since it can also be used to align the features from different types of attacks. Without such comparison, it is not clear how much gain the proposed method achieves compared with the established benchmark.\n\n6. What is the difference between this method and [1]?\n\n\n\n[1] Xie et. al. Adversarial Examples Improve Image Recognition.\n[2] Zhang et. al. Theoretically Principled Trade-off between Robustness and Accuracy.\n[3] Madry et. al. Towards Deep Learning Models Resistant to Adversarial Attacks.\n[4] Schott et. al. Towards the first adversarially robust neural network model on MNIST.\n[5] Athalye et. al. Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples.\n\nThe rating will be updated based on the rebuttal.", "extra_scores": {}, "extra_text": {}}, {"review_id": "_FkLFcty8L", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper introduces an algorithm for defending against multiple adversarial attacks (L1, L2, L-inf) by learning separate batch-norm statistics for each attack type. At inference time, the batch-normalized outputs corresponding to the different attack types are averaged according to the probability of attack types as given by a separate prediction network (one per batch-norm layer). The algorithm is evaluated on a range of datasets (MNIST, CIFAR-10, Tiny-ImageNet) and is shown to outperform recent competing approaches on a range of adversarial attacks.\n\nStrengths:\n- Empirical evaluation covers a range of datasets and attacks, including the most recent strong attacks\n- Performance improvement is significant over competing methods\n\nWeaknesses:\n- Experiment results seem to be from only a single run\n- Proposed method only applies to architectures that use batch-norm, unlike competing methods\n- Inconsistencies in the motivation vs empirical analyses, and experimental setups:\n  * Plots in Fig 1 to support the multi-domain hypothesis are on VGG16, but this is not the architecture used in experiments (ResNets were used instead)\n  * Main experimental results are on ResNets, but ablation studies are on VGG16\n  * Ablation study shows that adding GBN to the first layer results in the best performance (Fig 3a) but batch-norm statistics in this layer are almost indistinguishable between the different attack types (Fig 6a,b). \n\nOverall this paper introduces a new approach to defending against multiple adversarial attacks based on batch-norm statistics, with strong improvements over competing methods. Where it falls slightly short is in the inconsistencies mentioned above between motivation, empirical analyses to support the motivation and experimental setups; I will gladly raise my score accordingly if these are fixed.\n\nOther questions and comments:\n\n- What do the batch-norm stats look like on the ResNet architectures used? What is the effect of network architecture on the method?\n- How many runs were the reported results dervied from?\n- What is the performance of the best performing model (e.g. using adversarial training) under an individual attack? The proposed method almost seems like training separate models for each attack type - how does it perform against such a baseline where you have a classifier to pick the relevant model?\n- Why is the algorithm not trained using soft-gating, which will better reflect the actual prediction made at inference time?\n- It would be useful to show evidence of \"domain invariant representations\" learnt, for instance, using tSNE plots of the representations.\n- Clarify the meaning of \"All attacks\" in the results tables\n- I'm not sure if the theorem provided in the paper is helpful - the loss function -yf(x) is not commonly used.\n\n*** Post response comments ***\nI thank the authors for the clarifications and additional data provided in the revision; most of my concerns have been addressed. I have to say it is a bit worrying that the indistinguishable statistics in the first layer also occurs for the WideResNet models, which needs more explanation and exploration since it is a key motivation of the method. Also, the point raised by R1 on the evaluation is somewhat concerning. As such, I think this paper is borderline even though I am raising my rating to 6.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "Vi3v___WWM", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "### Summary\n\nThe authors propose to improve the robustness against adversarial attacks in deep networks via the use of a customized normalization strategy. Their central methodological suggestion is called gated batch normalization (GBN), and involves a (soft, but some other variants are briefly explored as well) gating mechanism that maps to distinct normalization branches. Their idea is evaluated on a set of benchmarks and robustness is measured threefold: in terms of $L_1$, $L_2$ and $L_\\infty$ perturbations.\n\n### Strengths\n\nThe authors formulate an intellectually compelling underlying hypothesis, namely that adversarial perturbations can be viewed as coming from distinct domains, special in character since they may be positioned very closely to the original domain of natural images of the unperturbed distribution. This is an interesting viewpoint (although I am not sure why we would want to call these entities domains, a \"mode\" or \"manifold\" would seem more appropriate), and this aspect is in my opinion the strong point of the paper!\n\nThe experimental section evaluates GBN against competitor approaches, either directly purposed for adversarial robustness (e.g. MSD) and conceptually similar normalization techniques (MN, MBN). CGN achieves convincing performance, and the underlying idea of separating BN statistics seems to improve adversarial robustness in a substantial way.\n\n### Weaknesses\n\nWhile the methodology of several gated BN units is well motivated, the technical novelty is somewhat limited. In particular, the method proposed here is very similar to the work of Deecke et al. (ICLR, 2019), with its main differentiation being that the gating mechanism is computed in a round-robin fashion (i.e. parameters are fixed when updating the gates, and vice versa) and with ground-truth information passed to the normalization units.\n\nThat being said, introducing supervision (with respect to the initially applied perturbation) seems to be crucial to be effective when defending against adversarial examples, and increases performance against a range of competing approaches. Because of this, I would argue that the limited novelty in terms of the normalization is offset somewhat by the idea to try this out in an adversarial setting – and getting it to work there.\n\n### Suggestions\n\nThere are a few suggestions that I would have liked to see included in the manuscript to increase its strength:\n* further study around what constitutes a good gating mechanism, in particular (i) what is the performance for a Gumbel-based approach; (ii) since the gates are computed offline, is there some sort of non-local parameter-sharing that (e.g. via a regularizer) benefits performance? This would also help in increasing the novelty of this work respective to the various competitive normalization approaches introduced in earlier publications.\n* Additional analysis around Figure 1 (c) and (d). Why was this particular layer chosen? Does this trend hold in other layers as well? What about deep lower versus higher layers. There are a number of interesting questions to be explored around this theme, for example is this trend a function of the semanticity? For some background reading on the depth of networks and their semantic function, I recommend looking into Asano et al. (ICLR, 2020).\n\n### Minor suggestions\n\nGood job on Figure 1 (a) and (b), this helped tremendously with quickly understanding the proposed mechanism! However, Figure 2 could use some improvement (in fact, maybe this can be removed in favor of additional discussions as outlined above).\n\nThere seems to be some issue with the citations, e.g. Kurakin et al. (ICLR, 2017) and Deecke et al. (ICLR, 2019) are cited with first & last names in the wrong order.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper first examines a multi-domain separation phenomenon, where different types of adversarial noise lead to different running statistics, and then introduces Gated Batch Normalization (GBN), a building block for deep neural networks that improves robustness against multiple perturbation types. GBN consists of a gated subnetwork and a multi-branch BN layer, and each BN branch handles a single type of perturbation. Results were reported on MNIST, CIFAR-10, and Tiny-ImageNet. Though the idea and methodology are valid and of interest, some concerns regarding the experimental section remain after rebuttal discussions.
Decision: Reject