Adversarial Masking: Towards Understanding Robustness Trade-off for Generalization
Authors: Minhao Cheng
Zhe Gan
Yu Cheng
Shuohang Wang
Cho-Jui Hsieh
Jingjing Liu
Venue:  (2021)
Keywords: Adversarial Machine Learning
Adversarial Robustness
Adversarial Training
Generalization

Abstract: Adversarial training is a commonly used technique to improve model robustness against adversarial examples. Despite its success as a defense mechanism, adversarial training often fails to generalize well to unperturbed test data. While previous work assumes it is caused by the discrepancy between robust and non-robust features, in this paper, we introduce \emph{Adversarial Masking}, a new hypothesis that this trade-off is caused by different feature maskings applied. Specifically, the rescaling operation in the batch normalization layer, when combined together with ReLU activation, serves as a feature masking layer to select different features for model training. By carefully manipulating different maskings, a well-balanced trade-off can be achieved between model performance on unperturbed and perturbed data. Built upon this hypothesis, we further propose Robust Masking (RobMask),  which constructs unique masking for every specific attack perturbation by learning a set of primary adversarial feature maskings. By incorporating different feature maps after the masking, we can distill better features to help model generalization. Sufficiently, adversarial training can be treated as an effective regularizer to achieve better generalization. Experiments on multiple benchmarks demonstrate that RobMask achieves significant improvement on clean test accuracy compared to strong state-of-the-art baselines.

Under review as a conference paper at ICLR 2021
ADVERSARIAL MASKING: TOWARDS UNDERSTAND- ING ROBUSTNESS OFF
OFF FOR GENERALIZATION
Anonymous authors Paper under double-blind review
ABSTRACT
Adversarial training is a commonly used technique to improve model robustness against adversarial examples. Despite its success as a defense mechanism, ad- versarial training often fails to generalize well to unperturbed test data. While previous work assumes is caused by the discrepancy between robust and non- robust features, in this paper, we introduce Adversarial Masking. new hypothesis that this trade-off is caused by different feature maskings applied. Specifically, the rescaling operation in the batch normalization layer, when combined together with ReLU activation, serves as a feature masking layer to select different features for model training. By carefully manipulating different maskings, well bellnbalanced trade-off can be achieved between model performance on unperturbed and per- turbed data. Built upon this hypothesis, we further propose Robust Masking (Rob- Mask), which constructs unique masking for every specific attack perturbation by learning set of primary adversarial feature maskings. By incorporating dif- ferent feature maps after the masking, we can distill better features to help model generalization. Sufficiently, adversarial training can be treated as an effective reg- ularizer to achieve better generalization. Experiments multiple benchmarks demonstrate that RobMask achieves significant improvement on clean test accu- racy compared to strong nate-of-the-art baselines.
INTRODUCTION
Deep neural networks have achieved unprecedented success over variety of tasks and across dif- ferent domains. However, studies have shown that neural networks are inherently vulnerable to ad- versarial examples (Biggio et al., 2013; Szegedy et al. (0044). To enhance model robustness against adversarial examples, adversarial training (Goodfellow et al. 2015 Madry et al., 2018) has become one of the most effective and widely applied defense methods, which employs specific attacking algorithms to generate adversarial examples during training in order to learn robust models.
Albeit effective in countering adversarial examples, adversarial training often suffers from inferior performance on clean data (Zhang et al.] [011 Balaji et al.) (019). This observation has led prior work to extrapolate that trade-off between robustness and accuracy may be inevitable, particularly for image classification tasks (Zhang et al. 2019 Tsipras et al.) 2019). However, Yang et al. (2020) recently suggests that is possible to learn classifiers both robust and highly accurate on real im- age data. The current state of adversarial training methods falls short of this prediction, and the discrepancy remains poorly understood.
In this paper, we conduct an in-depth study on understanding the trade-off between robustness and clean accuracy adversarial training, and introduce Adversarial Masking, new hypothesis stating that a widely used technique, batch normalization (BN), has a significant impact on the trade-off between robustness and natural accuracy. Specifically, we break down BN into normalization and rescaling operations, and find that the rescaling operation has a significant impact on the robustness trade-off while normalization only has marginal influence. Built upon this observation, we hypoth- esize that adversarial masking (i.e., the combination of the rescaling operation and the follow-up ReLU activation fucntion) acts as a feature masking layer that can magnify or block feature maps to influence the performance of robust clean generalization. In this hypothesis, different rescaling parameters in BN contribute to different adversarial maskings learned through training. By using simple linear combination of two adversarial maskings, rather than using robust features learned

---

Under review as a conference paper at ICLR 2021
by adversarial training (Madry al. 2018 lyass al. 2019 Zhang et al. 2019), show that well-balanced trade-off can be readily achieved.
Based on the Adversarial Masking hypothesis, we further propose RobMask (Robust Masking), new training scheme that learns an adaptive feature masking for different perturbation strengths. We the learned adaptive feature masking to incorporate different features so that we could improve model generalization with a better robustness trade-off. Specifically, each perturbation strength is encoded low-dimensional vector, and we take this vector input learnable linear projec- tion layer together with ReLU activation, to obtain the adversarial masking for the corresponding perturbation strength. Therefore, for different perturbation strengths, we learn different maskings accordingly. By doing so, rather than hurting the performance on clean test data, we use ad adversarial examples as powerful regularization to boost model generalization. Experiments on multiple bench- marks demonstrate that RobMask achieves not only significantly better natural accuracy, but also better trade-off between robustness and generalization.
We conduct detailed analysis to demonstrate that the rescaling operation batch normalization has a significant impact on the trade-off between robustness and natural accuracy. (fi) We introduce Adversarial Masking, a new hypothesis to explain that this trade-off is caused by different feature maskings applied, and different combinations of maskings lead to different trade-offs. (tii) We propose RobMask, new training scheme to learn an adaptive masking for different perturbation strengths, in order to utilize adversarial examples to boost generalization clean data. RobMask also achieves better trade-off between robust and natural accuracy.
Our contributions are summarized as follows. (i)
2 PRELIMINARY AND RELATED WORK
Adversarial Training Since the discovery of the vulnerability of deep neural networks, diverse approaches have been proposed to enhance model adversarial robustness. A natural idea is to itera- tively generate adversarial examples, add them back to the training data, and then retrain the model. For example, Goodfellow et al. (2015) uses adversarial examples generated by FGSM to augment the data, and Kurakin et al. (2017) proposes to use a multi-step FGSM to further improve performance. Madry al.J (2018) shows that adversarial training can be formulated problem, and proposes PGD attack (similar to multi-step FGSM) to find adversarial examples for each batch. Specifically, for a K-class classification problem, let us denote D {(x,, 31) ]=)== as the set of training samples with æ; Rd. 34 f1..... K). where K is the number of classes.
min-max optimization
Considering a classification model fa(æ)) : RRd + A parameterized by 0, where A represents K -ddimensional simplex, adversarial training can be formulated as:
min Σ max (..,.
{(f(2)))
where (x, c) denotes the lp norm ball centered at a, with radius c, and ((..)) )) is the cross-entropy loss. The inner maximization problem aims to find adversarial version of given data point æ; that yields the highest loss. In general, (,,,,, <) can be defined based on the threat model, but the to ball is the most popular choice among recent work (Madry al. 2018 Zhang al. 2019 which is also adopted in this paper.
For deep neural networks, the inner maximization does not have closed-form solution. Thus, ad- versarial training typically uses a gradient-based iterative solver to approximately solve the inner problem. The most commonly used choice is multi-step PGD (Madry et al. 2018) and C&W at- tack (Carlini & Wagner (017). Since then, most defense algorithms (Zhang et al.) 2019; Balajii
al. 2019 Wang et al. 2019 Ding et al., 2018 Cheng et al. 2020) are based on a similar min-max framework.
Trade-off between Robustness and Accuracy While effective in improving model robustness, adversarial training is known to bear performance drop on clean test data. Tsipras et al. (2019) provides a theoretical example of data distribution where any classifier with high test accuracy must also have low adversarial accuracy under foo perturbations. They claim that high performance both accuracy and robustness may be unattainable due to their inherently opposing goals. Zhang et al. (019) decomposes the robust error the sum of natural (classification) error and boundary
2

---

Under review as a conference paper at ICLR 2021
error, and provides a differentiable upper-bound using the theory of ceffiiaation-aaibbateee loss, based on which they further propose TRADES to achieve different trade-offs by tuning the regular- ization term.
Most recently, Xie et al. (2020) proposes AdvProp to assign another batch normalization for gener- ating adversarial examples, and shows improved performance on clean data in image classification tasks. There are also parallel studies on applying adversarial training to improve clean data perfor-
mance in language understanding and asiionanddlanguage tasks (Zhu al. 2019 Jiang al. 2019
Liu et al. 2020; Gan et al. 2020).
Batch Normalization Batch Normalization is a widely adopted technique that enables faster and more stable training of deep neural networks, below, we provide a brief overview batch nor- malization, which paves the way to introduce our method. Specifically, batch normalization (loffe & Szegedy 2015) is proposed to reduce the internal co-variate shift to ease neural network train- ing. Considering convolutional neural network, can define the input and output 10,c,a.y and Ob, respectively. The dimensions correspond to examples with a batch b, channel c, and two spatial dimensions I. y- A neural network applies the same normalization for all activations in given channel:
Ob. t I..C.. - He + Vb, C,
(2)
Vo?+ +
where fec = [88] Eo...y Ab..aar denotes the mean for channel c, and or denotes the corresponding standard deviation. y and 3 are two learnable parameters for the channel-wise wise affine transformation, i.e, rescaling operations. is small number control numerical stability.
3 ADVERSARIAL MASKING
3.1 BATCH NORMALIZATION ACTS AS ADVERSARIAL MASKING
Ilyas et al. 2019) disentangles adversarial examples as a natural consequence of non-robust fea- tures. Specifically, they construct robust features from an adversarial trained "robust model" di- rectly. Therefore, a common belief is that adversarial robustness comes from feature representations learned through adversarial training (Ilyas et al.) [019 Santurkar et al.) 2019). An interesting ques- tion we would like to ask is: can we learn robust features from vanilla standard-trained model, or, can we obtain non-robust features from an adversarial trained "robust model"?
To answer this question, we design the following experiments. We first train ResNet-18 model with stan- dard and adversarial training, then finetune the networks by allowing only batch normalization (BN) to be changed while freezing other param- eters. Specifically, we finetune BN in standard trained model using adver- sarial training, and finetune BN in an adversarial trained model with stan- dard training, respectively. Results are summarized in Table Given a standard trained model, by only per- forming adversarial finetuning of the BN layers, the resulting model can already achieve a reason- ably good robust accuracy 26.51% (the 1st block). Similarly, given adversarial trained model, by only performing standard finetuning of the BN layers, the clean accuracy of the model increases significantly from 78.47% to 86.96% (the 2nd block).
Method
Clean Acc.
Robust Acc.
Standard Training
91.97% 53.96%
0.0% 26.51 %
Adv. Finetuning of BN
Adv. Training Standard Finetuning
78.47%
48.67%
86.96%
5.83%
Table 1:
Clean and robust accuracy of ResNet-18 models
trained under different settings on CIFAR-10. All robust accuracy results are obtained using the 8/255 foo ball. (BN: Batch Normalization)
This experiment demonstrates that we can control the trade-off between clean and robust errors by only tuning the BN layer, so here comes natural question: which part in BN has contributed to this performance trade-off? To investigate this, we take a step further to check the difference of every parameter used in BN. In particular, check the first BN layer after the first convolution layer.
3

---

Under review as a conference paper at ICLR 2021
byyer C-nunning mes'
bayer Ouruniing ver
D-wwighh
lnyer C-blass
0.4
Ady
0.05
514 EN Innetone
0.4
Adi Tha
Std GN Reets A0
0.2
D:04
1:0
092-
11
6M
20
(a) Running mcan jz
(b) Running variance d
(c) Rescaling
(d) Rescaling S
Figure 1: Batch statisties in the first batch normalization (BN) layer of an adversarial trained R8 model on CIFARIO, with and without further standard fine-tuning of BN (orange and blue lines, respectively). The running mean f and variance a, as well as the rescaling shift param- eter / are almost the same (overlapped in the figure), while the rescaling weight y has significant difference, which has a notable contribution to the clean and robustness trade-off.
Model
p
1.0
0.8
0.6
0.4
0.2
0.0
Adv. Training with and w/o Std. Finetuning of BN
Clean Acc.
78.47%
81.16%
82.8%
84.66%
86.06%
86.96%
Robust Ace.
48.67%
44.88%
37.2%
24.79%
12.85%
5.83%
Sid. Training with and w/o Adv. Finetuning of BN
Clean Acc.
91.97%
88.66%
74.87%
58.76%
53.89%
53.96%
Robust Ace. 0.0%
0.0%
0.12%
5.73%
19.93%
26.51%
Table 2:
The clean and robust accuracy using different combination coefficient p on CIFAR-10 with ResNet- 18. The 1t block uses adversarial trained models with and without further standard fine- tuning of batch normalization (BN). The 2nd block uses standard trained models with and without further adversarial finetuning of BN. All robust accuracies are obtained using 8/255 foo ball.
As well known, BN uses a running average of the mean and variance during testing. Figure Ta and Tb illustrate the difference the running mean μ and running variance a. The batch statistics with and without further standard finetuning of BN (under the setting in the 2nd block of Table D are nearly identical across all the dimensions. and B. We can clearly see that the fine-tuned rescaling parameter y is completely different from the original one, with unchanged, indicating that y has a significant impact on the clean and robust trade-off while still performing similar normalization on both sides.
the learned rescaling parameters
On the other hand, Rectified Linear Unit (ReLU) (Agarap) 2018) is the most commonly used acti- vation function in deep neural networks. The function returns Ó if it receives any negative input, and for any positive value x it returns that value back (i.e., fax) = max(0, During the rescaling operation, y would magnify or shrink the magnitude of feature maps. Together with B, after ReLU activation, features that become negative will be blocked as O. By combining the rescaling operation with the following ReLU activation function, the resulting layer can be viewed to magnify or block the features maps from the convolution layer (see Figure 2(a) a) for illustration). To further validate this, plot the feature maps after ReLU activation functions in Figure 4 in the Appendix. Some feature maps are blocked after finetuning BN (completely black when all pixels are set to 0) as well as some are magnified significantly. We term the above observation as Adversarial Masking, and hypothesize that this leads to the trade-off between robustness and natural accuracy.
masking layer
3.2 CONTROLLING ROBUSTNESS TRADE-OFF VIA ADVERSARIAL MASKING
The above analysis suggests that, rather than feature representations, rescaling in the BN layer to- gether with ReLU activation function serves as masking layer for selecting different feature combi- nations that can achieve different performance trade-off's between clean and perturbed test sets. From this hypothesis, a different combination of BN together with ReLU can be regarded as a different adversarial masking. With such a masking, we can readily achieve series of trade-offss, without the need of training the model from scratch again, which is the case for conventional adversarial training. In the following experiment, maskings to achieve this trade-off, and empirically, we observe that this simple design is sufficient. Specifically, denote (r, B) and (2') as two learned adversarial maskings (i.e., the learned rescal-
use simple linear combination of two learned adversarial
4

---

Under review as a conference paper at ICLR 2021
Myy Wy 8
u.
Conv
BN
Affine
RetU
Affine
RelU
x
Adversarial Masking
RobMask
(a) Adversarial Masking.
(b) The proposed RobMask method.
Figure 2: Illustration of (a) Adversarial Masking hypothesis, and (b) RobMask method for im- proving the generalization performance. Instead of just using single masking for both clean and adversarial examples, we use the linear combination of primary rescaling parameters {1)))_1
and {w/ 1 to incorporate different perturbation strength
ing parameters in the BN layer), and we p (1-hh' and p3 (1- We then use the new adversarial masking (f, () for evaluation. Table shows that different clean and robust accuracies can be readily achieved by selecting different values.
Previous work (Zhag et al. 2019) uses regularization hyperparameter À to balance between clean error and robust error. By tunning x, they could achieve different robustness trade-offs. However, takes enormous time and effort to retrain the model from the scratch. Instead, this finding inspires us that we can just store one model and employ a series of learned adversarial maskings to control the robustness trade-off real time.
IMPROVING MODEL. GENERALIZATION VIA ROBMASK
As mentioned in Section 3.2 different trade-offs can achieved by linearly combining two pre- trained batch normalization layers. However, this may not be ideal due to several deficiencies. First, the "clean" mask learned by fine-tuning is not distilled and may partially override the mask learned from adversarial examples, leading strength tends to have different masking, if we only utilize one perturbation strength, we lose all the other maskings generated by the perturbation strength in-between. Third, it requires careful selection of what the maximum perturbation strength is. In the extreme case, if a sample is perturbed to the decision boundary, the learned adversarial mask might be completely meaningless. Or, if the chosen perturbation strength is too small, there will not be enough regularization for improving generalization. To address these issues, we propose RobMask (Robust Masking), a new framework that aims to actively learn the adversarial masking to boost generalization performance.
sub-optimal solution. Second, since every perturbation
Specifically, we propose to incorporate different perturbation strengths for model training, instead of just one. Note that could treat 0 for the unperturbed data. Then, straightforward way to just learn set of ys. 8, independently for every perturbation strength 6G. However, due to the limited number of sampled perturbation strengths, each % could have poor generalization due to over-fitting. At the same time, loses the correlation between all the generated maskings. Instead, need to jointly learn the corresponding maskings simultaneously.
To this end, we assume that every rescaling parameter "i can be well-approximated by a linear combination of A basic rescaling parameters {w,}f= where k small number. By encoding perturbation strength e; into a k-dimensional vector u; we can linearly combine wy using u; to obtain rescaling parameter for strength 6; 7 Σÿ=1 W.. Take k 2 as example:
we can encode un = [1.0 0.0]T for e = and Ummar = [0.0 1.0]T for E; = Emazz respectively. Naturally, the intermediate perturbation strength 6; = P: Emaz be encoded u, (1
5

---

Under review as a conference paper at ICLR 2021
Algorithm The proposed RobMask method for improving model generalization.
Input: Training dataset (æ,, M perturbation upper bound Cmax- for epoch= 1, ....... N do for i 1, ...... B do
Sample a random number p from 0 to 1
e, k p- Emar obtain the current perturbation strength in this mini- u; f Encode(E;) # encoder the perturbation strength as vector
ò; +
for = -....,,m do
+ 8; + a sign( (Vel[o((a; + 8.),y:) PGD adversarial attack d: max(min(( ( <i), -ci)
end for
N 1j=1 Wi Br 4 1== u. # the rescaling parameters in BN
W W 711 (w((fol;;
()),;)) update W and W
W' + W' -th Vwa(ff
- -o,,-11,1 7,, 3.).
B t Tl Vel(fo(x; + 6,,%
n. 3.).)))) update neural network parameters 0
end for
end for
return 0 W. W
Pa) Po) + p; Pillemar' Therefore, instead of learning 7i separately for every e,, we learn a low-rank matrix W = W1, [1 W Wg to incorporate different perturbation strengths and learn series of maskings. Similarly, we learn another matrix W for B4.
During training, in every iteration, with a randomly selected perturbation strength = P; Emaas we first generate mini-batch of adversarial examples by conducting PGD attacks. Then, we learn the rescaling parameter of BN by using low-rank linear layer (W and W') and encoded attack strength u, Finally, we minimize the total loss using stochastic gradient descent (SGD) to update model parameters. Detailed algorithm is summarized in Algorithm
Connection with AdyProp Xie et al. (2020) hypothesizes that the performance degradation on unperturbed test dataset is mainly caused by the distribution mismatch between adversarial examples and clean images. They propose AdvProp to assign an auxiliary batch normalization for adversarial examples, and show that adversarial examples can be useful to achieve better performance test data. However, as shown in Figure I the running mean and variance are kept the same after fine-tuning. We argue that the improved model generalization is realized by different adversarial mask learned by auxiliary batch normalization in the AdvProp procedure.
clean
Although we utilize adversarial training to boost generalization well, our approach has clear differences. First, in AdvProp, there is no connection between the traditional and auxiliary batch normalization (BN). The traditional BN only obtains inputs from clean data, and the auxiliary BN only obtains inputs from adversarial examples. This type of disentanglement would completely separate different masks, which violates the reality that some masks can useful for both clean and robust performance. Second, AdvProp has to designate a perturbation strength that should not be too large or too small, which difficult tune in practice. Also, the proposed RobMask method more general than Advprop, and AdvProp can be considered as special case of RobMask when we set the linear layer rank & to and freeze p = in the whole training process.
5 EXPERIMENTS
In this section, we conduct experiments to show that RobMask can successfully improve general- ization performance. We also provide additional robustness evaluation for completeness.
5.1 EXPERIMENTAL SETUP
Datasets and Model Architectures We use two popular datasets CIFAR-10 and CIFAR- 100 (Krizhevsky al. 2009) for experiments. For model architectures, we the popular

---

Under review as a conference paper at ICLR 2021
ResNet (He et al., 2016a) family including Preact ResNet (He et al. 2016b), ResNeXt (Xie al. 2016) and the recent well-performed DenseNet (Huang et al. 2016).
Baselines We compare RobMask with two basclines: (i) AdvProp: Dual batch normalization (iee at 2020). where different batch normalizations are used for clean and adversarial examples during training; and (ii) BN: Standard training with normal batch normalization enabled. Note that as our main goal is to improve the generalization performance instead of robust test accuracy, we do not compare against standard adversarial training methods, as they are reported largely decrease generalization performance (Madry et al. 2018 Zhang el al. 2019 Balaji et al. 2019).
Implementation Details For CIFAR-10 10 and CIFAR-100, we the number of iterations in adver- sarial attack to 7 for all the methods during training. All PGD attacks are non-targeted attacks with random initialization. We set the PGD attack strength 8/255 with cross-entropy (CE) loss and the step-size to c/5. All models are trained using SGD with momentum 0.9, weight decay 5 x 10-4. We use cosine learning rate scheduling with initial learning rate 71 = 722 = a 0.1. To have a fair comparison, for RobMask, we set k 2 and Émaz 8/255 in all our experiments, which has the same number of model parameters and same regularization strength as AdvProp. All our experiments are implemented in Pytorch. Code will be released upon acceptance.
5.2 EXPERIMENTAL RESULTS
Generalization Table 3 summarize the results of all the evaluated methods on CIFAR-I0 and CIFAR-100. Across all the tested model architectures, RobMask shows significant improvement over both normal batch normalization (BN) and AdvProp. Specifically, as shown in Table for 100- epochs training CIFAR-10, RobMask achieves around 1.5% test accuracy improvement over BN and 0.8% over AdvProp, respectively. Similar improvements can also be observed on the CIFAR- 100 dataset. Further, when comparing results between Table 3
we observe that RobMask also leads to faster convergence: 20-epochs training using RobMask leads to results that are comparable to 100-epochs training using BN.
Additionally, we add the large-scale dataset ImageNet into the comparison. Table 4 summarize the results with R on ImageNet datasets. It could be clearly seen that while Advprop has very limited improvement on ResNet- 18, 8, RobMask has around 0.4 percent improvement, which further shows RobMask's 's effectiveness on the generalization.
Robustness Evaluation In addition to improved generalization performance, our method can also achieve a better robust and natural accuracy trade-off over adversarial training. For CIFAR-10 and CIFAR-100, we evaluate all the methods under the white-box = 8/255 los -nom bounded non- targeted PGD attack. Specifically, we use 100-step PGD with step size e/5) that is equipped with random start. Moreover, to further verify the robust accuracy achieved, we use the Autoattack(Crovg & Hein 2020) to evaluate the performance. Note that, when c = 0, robust accuracy is reduced to the test accuracy of unperturbed (natural) test samples, i.e, clean accuracy. Results are summarized in 5nd RobMask clearly outperforms other methods among from 0 to 6/255. However, RobMask performs slightly worse e 8/255. It is because both AdyProp and Adversarial training models are trained with adversarial examples generated with e = 8/255, while our methods use random perturbation where emaz = 8/255. That is, we use a weaker perturbation strength compared to both AdvProp and Adv train.
To achieve better result on e = 8/255, we relax the max epsilon constraint from 8/255 to 10/255. From Table o we could see that the clean performance drop slightly with a increasing robust accu- racy on e 6/255 so that we now could achieve better robust accuracy on e = 8/255. Even with the slight degrading performance on clean accuracy, RobMask achieves a better adversarial robustness trade-off over other methods.
Importance of low-rank matrix: We conduct an ablation study on DenseNet-121 121 over CIFAR- 10 to investigate the importance of using a low-rank matrix to incorporate multiple perturbation strengths. Here, we extend AdvProp to use randomly selected strength e; Pi -maa to generate adversarial examples, and then feed into auxiliary batch normalization. Advprop can also be gener- alized using multiple auxiliary BNs when given multiple perturbation strengths. In the experiments, instead of an auxiliary batch normalization for adversarial examples generated by e = 8/255, we also give another batch normalization for e = 4/255 adversarial examples. Table 7 shows Adyprop
7

---

Under review as a conference paper at ICLR 2021
CIFAR-10
CIFAR-100
Model
#Epoccs
BN
AdvProp
RobMask
BN
AdvProp
RobMask
20
92.59%
93.45%
94.64 %
75.88%
76.15%
77.61 %
ResNet-18
100
94.87%
95.3%
96.10% %
77.7%
76.82%
78.77% %
20
93.26%
94.61%
95.30% %
74.79%
73.61%
75.11% %
DenseNet-121
100
94.71%
94.61%
96.47% %
77.63%
76.88%
80.18 %
20
91.93%
92.55%
94.04% %
70.79%
72.71%
73.59 %
Preact-18
100
94.37%
95.33%
95.97 %
76.14%
76.87%
78.19%
20
93.12%
93.37%
95.03%
74.26%
72.18%
74.65%
ResNeXt-29
100
95.15%
95.29%
96.06% %
78.60%
76.35%
79.54
Table 3: Comparison on CIFAR-10/10 over ResNet-18, DenseNet- 121, Preact-18, and ResNeXt- 29. Models are trained for 20 and 100 epochs using normal Batch Normalization (BN), AdvProp and our RobMask. RobMask shows a significant performance improvement on all model architectures. Also, RobMask trained with 20 epochs achieves comparable performance 100-epoch training using BN and AdyProp.
ImageNet
Model
BN
AdvProp
RobMask
ResNet-18
69.76%
69.79%
70.14%
Table 4: Comparison on ImageNet over ResNet-18 18. Models are trained for 105 epochs using normal Batch Normalization (BN), AdvProp and our RobMask. RobMask shows significant performance improvement on all model architectures.
2/255
4/255
6/255
8/255
Adv. Training 78.86%
72.61%
65.27%
57.15%
47.97%
AdvProp 85.98%
78.34%
69.24%
57.61%
46.04%
RobMask 89.99 %
82.91 %
72.23%
58.63%
44.50%
Table 5: Robust accuracy under different levels of PGD fx attacks on CIFAR-10 with ResNet-18 architecture. RobMask clearly outperforms AdvProp and standard adversarial training in all the test perturbation strengths except = 8/255 on which AdvProp and standard adversarial training are trained.
2/255
4/255
6/255
8/255
Adv. Training 78.86%
70.99%
62.94%
53.83%
44.66%
AdvProp 85.98%
77.73%
67.57%
55.36%
43.13%
RobMask 89.99
81.87%
70.99
55.9%
41.7%
RobMask 10/255
86.5%
78.47%
68.08%
56.89 %
44.69 %
Table 6: Robust accuracy under different levels of Loo Autoattacks on CIFAR-I0 with ResNet-18 architecture. RobMask clearly outperforms AdvProp and standard adversarial training in all the test perturbation strengths except 8/255 on which AdvProp and standard adversarial training are trained.
has degenerated performance when using random perturbation strength. When adding more iliary batch normalization, the performance improves slightly, also observed in Xie et al. 22020) However, RobMask still significantly outperforms AdvProp variations.
Training curve: In Figure we plot the training curves for both RobMask and Advprop for CIFAR- 10 datasets DenseNet- 121. RobMask clearly outperforms Adyprop during the whole training process. Also, RobMask will not introduce additional training overhead cost than AdyProp. Since both RobMask and AdvProp use two forward and one backward pass, they have almost identical training time per epoch. For example, in DenseNet-121. it takes around 870 seconds for Advprop and around 890 seconds for RobMask. i.e. our model did take much longer to train.
8

---

Under review as a conference paper at ICLR 2021
Method
Clean Acc.
AdvProp
94.61% %
with random perturbation strength
94.50%
with auxiliary BN
95.10
RobMask
96.47%
Table Comparison on CIFAR-10 over DenseNet-12) 121 on AdvProp and its extensions. Models are trained for 100 epochs.
CONCLUSIONS
In this paper, analyze the impact of batch normalization (BN) on adversarial robustness, and show that the rescaling operations in BN has strong impact on the clean and robust- ness trade-off. We then formalize the rescal- ing operations together with ReLU activations as an adversarial mask, and show that sim- ple linear combination of two adversarial mask-
Training curve
100
Accuracy
ings can be utilized directly to achieve different performance trade-offs. Inspired by these find- ings, we propose RobMask, an active adversar- ial mask learning method that designed to achieve better generalization performance. The success of RobMask indicates that adversarial training can serve as a strong regularizer, in- stead of a performance killer, for training better models.
RobMask Acc
60
Advprop Test Acc
RobMask Train Acc
40
Advprop Train Acc
20
40
60
80
100
Fnorhs
Figure 3: Training curve on DenseNet- 121

---

Under review as a conference paper at ICLR 2021
REFERENCES
Abien Fred Agarap. Deep learning using rectified arXiv: urXiv:/803.08.0 0803.08375, 2018.
linear
units
(relu).
arXiv preprint
Yogesh Balaji, Tom Goldstein, and Judy Hoffman. Instance adaptive adversarial training: Improved accuracy tradcoffs in neural nets. arXiv preprint arXiv:1910.08051, 2019.
Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Smniić, Pavel Laskov, Gior- gio Giacinto, and Fabio Roli. Evasion attacks against machine learning test time. In Joint European conference on machine learning and knowledge discovery databases, pp. 387-402, 2013.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In IEEE Symposium Security and Privacy, pp. 39-57, 2017.
Minhao Cheng, Qi Lei, Pin-Yu Chen, Inderjit Dhillon, and Cho-Jui Hsieh. Cat: Customized adver- sarial training for improved robustness. arXiv preprint arXiv:2002.06789 2020.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter- free attacks. arXiv preprint arXiv:2003. 000600000000,,,,... 2020.
Gavin Weiguang Ding, Yash Sharma, Kry Yik Chau Lui, and Ruitong Huang. Max-margin adversar- ial (mma) training: Direct input space margin maximization through adversarial training. arXiv preprint 1XXiv:81120026377 2018.
Zhe Gan, Yen-Chun Chen, Linjie Li, Chen Zhu, Yu Cheng, and Jingjing Liu. Large-scale adversarial training for vision-and aanon-agd-l representation learning. arXiv preprint (00i:::200606/992 2020.
lan J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. International Conference Learning Representations, 2015.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016a.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In European conference on computer vision, pp. 630-645. Springer, 2016b.
Gao Huang, Zhuang Liu, and Kilian Q Weinberger. Densely connected convolutional networks. corr abs/1608.06992 (2016). arXiv preprint arXiv: 1608: 06999, 2016.
Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Adversarial examples not bugs, they features. In Advances in Neural Information Processing Systems, pp. 136. 2019.
Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.0316 2015.
Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, and Tuo Zhao. Smart: Robust and efficient fine-tuning for pre-trained natural language models through principled regu- larized optimization. arXiv preprint arXiv: 1199111.3377 2019.
Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. 10 (canadian institute for advanced re- search). 2009.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial machine learning at scale. Inter- national Conference on Learning Representations, 2017.
Xiaodong Liu, Hao Cheng, Pengcheng He, Weizhu Chen, Yu Wang. Hoifung Poon, and Jianfeng Gao. Adversarial training for large neural language models. arXiv preprint arXiv:2004.00 08999, 2020.
10

---

Under review as a conference paper at ICLR 2021
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. To- wards deep learning models resistant to adversarial attacks. International Conference on Learning Representations, 2018.
Shibani Santurkar, Andrew Ilyas, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Image synthesis with a single (robust) classifier. In Advances in Neural Information Processing Systems, pp. 1262-1273, 2019.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. International Conference Learning Representations, 2014.
Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry- Robustness may be odds with accuracy. In International Conference Learning Representa- lions, 2019.
Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improving adversarial robustness requires revisiting misclassified examples. In International Conference on Learning Representations, 2019.
Cihang Xie, Mingxing Tan, Boqing Gong. Jiang Wang, Alan L Yuille. and Quoc V Le. Adversarial examples improve image recognition. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 819-828, 2020.
Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. Aggregated residual trans- formations for deep neural networks. 2016. arXiv preprint arXiv:1611.05431 2016.
Yao- Yuan Yang, Cyrus Rashtchian, Hongyang Zhang, Ruslan Salakhutdinov, and Kamalika Chaud- huri. A closer look at accuracy vs. robustness. arXiv preprint urXiv:2003.0246/ 2020.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. International Conference Machine Learning. 2019.
Chen Zhu, Yu Cheng, Zhe Gan, Siqi Sun, Tom Goldstein, and Jingjing Liu. Freelb: Enhanced adversarial training for natural language understanding. ICLR, 2019.

---

Under review as a conference paper at ICLR 2021
A APPENDIX
A.1
MULTIPLE RUNS
We run RobMask for three times with different random seed and present the mean in Table report the more detailed result in Table 8
We
Model
#Epochs
CIFAR-I0
CIFAR- 100
20
94.5410.22
77.41 0.20%
ResNet-18
100
95.9010.224
8.59:10.18%
20
95.10:1 0.20%
75,94 0.17%
DenseNet-121
100
96.29 0.18%
79.99 0.18%
20
94.924 0.12%
7 0.16%
Preact-18
100
95.83-+ 0.14%
78.01
20
94.83+ 0.21%
74.43 0.22%
Res.Nett229
100
96.84 0.22%
79.31 0.23%
Table 8: RobMask results on CIFAR-10/100 over ResNet-18, DenseNet-121, Preact-18, 18, and ResNeXt-29. Models are trained for and 100 epochs.
A.2 COMPARISON ON BATCH NORMALIZATIONS
In this section, we extend our experiment in Figure 1 to show the batch statistics the deeper layers across the neural networks. It clearly shows that the rescaling weight has more effect than other parameters the batch normalization. To be noted, since the deep layer's 's mean and variance would be affected by the shallow layers' rescaling weight parameter, the result the deeper layer couldn't disentangle the effect between normalization and rescaling because it is mixed.
Mean
Variance
Weight
Bias
Layer 0
1.0
1.0
0.7620
I.0
Layer 1
0.9842
0.9718
0.7883
1.0
Layer 2
0,9530
0.9199
0.7544
1.0
Layer 3
0.9743
0.9691
0.8594
1.0
Layer 4
0,8894
0.9340
0,8126
1.0
Layer 5
0.9555
0.9516
0.8813
1.0
Layer 6
0.9853
0.9452
0.7141
1.0
Layer 7
0.9554
0.9169
0.8609
1.0
Layer 8
0.9903
0.9646
0.8961
1.0
Layer 9
0.9635
0.9755
0.8046
1.0
Layer 10
0.9823
0,9522
0.9396
1.0
Layer 11
0,9823
0,9769
0.7906
1.0
Layer 12
0.9753
0.9593
0.7839
1.0
Layer 13
0.9914
0.9874
0.8891
1.0
Layer 14
0.9699
0.9898
0.6593
1.0
Layer 15
0,9902
0.9870
0.8605
1.0
Layer 16
0.9603
0.9889
0.7423
1.0
Layer 17
0.9736
0.9742
0.6809
1.0
Layer 18
0.9772
0.9838
0.9573
1.0
Table 9: Cosine similarity on under every batch normalization layer under standard fine-tune train- ing on adversarial trained model
A.3 ADVERSARIAL MASKING VISULIAZATION
12

---

Under review as a conference paper at ICLR 2021
(a) Adversarial training with further stan- dard finetuning of BN
(b) Adversarial training
Figure 4: Illustration of the Adversarial Masking effect. We mark several feature maps (red and green boxes) are blocked out or magnified when comparing (a) and (b), which can be viewed selection mask on "non-robust" and "robust" features.
13


Reviews:
[{"review_id": "lCBthU0XoNo", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\nThis paper follows the direction of previous work AdvProp (Xie et al. (2020)) and aims to use adversarial training as a regularizer to improve the network generalization on clean data. The authors analyze that the different rescaling operation in the batch normalization layer along with ReLU acts as feature masking/selection layer, which can control the trade-off between adversarial robustness and clean data performance. Unlike AdvProp, which uses different batch normalization layer for clean images and adversarial images at a specific perturbation strength, here the authors propose a technique called RobMask that adapts the rescaling parameters of batch normalization based on the perturbation strength during training. The authors show that such adapting technique is more effective than using different batch normalization layers (as in AdvProp) for each perturbation strength and thus improves the clean data performance on CIFAR10/100.\n\nStrengths: \n+ Motivation and analysis is clear.\n+ Discussed its differences to previous works.\n+ Proposed technique is simple, easy to adapt in the existing setup of adversarial training and addresses the limitations of the previous work AdvProp.\n+ Evaluation is carried out on CIFAR10/100 across different network architectures: ResNet-18, DenseNet-121, Preact ResNet18, ResNeXt-29.\n+ Results on CIFAR-10/100 using four different deep network architectures suggest that this work improves clean data performance than the baselines: standard network training and AdvProp.\n\nWeaknesses:\n-\tThe authors claim about well-balanced robustness trade-off using their method and also claim that their major objective is only to improve network generalization on clean data. There is a little ambiguity regarding the major contribution of this paper. The authors can make this point more clear. \n-\tIsn’t the hypothesis that is stated as “new” in this work already discussed in AdvProp i.e. using different batch normalization for clean and adversarial images improves network generalization, which in turn draw the conclusion that rescaling operation of batch norm could control the robustness and generalization trade-off. Why this hypothesis considered as “new” then ?\n-\tThe two learned adversarial maskings discussed in section 3.2, it is not clear how they are generated. \n-\tResults demonstrate that the proposed approach improves generalization but the performance gain is minimal (only 1%-2%) and not so significant compared to the baselines.\nMinor point:\n-\tI understand that the major objective of this work is to improve performance on clean images but not the adversarial robustness. The results demonstrate higher robustness against PGD based adversarial attacks with perturbation strength lower than 8/255 is interesting but not of practical importance since the method requires perturbation strength as an additional input and very specific to PGD based attack. I wouldn’t consider this as major weakness since it is not the primary objective of this work.\n\nFinal thoughts:\nThe proposed method is clearly motivated. Although the performance gains on network generalization are minimal compared to the baselines, this work cleverly addressed the limitations of previous work and extend it with simple modifications. I tend to accept this paper. However, I suggest the authors to also consider the evaluations carried out in AdvProp (Xie et al. (2020)) to improve the significance of their work.\n\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "cm3qgaVHmpX", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\nThe paper investigates the role of Batch Normalization (BN) in the generalization of deep networks, and its impact in the trade-off between clean and robust accuracy in adversarially trained networks. The authors demonstrate that the rescaling operations in BN when considered in conjunction with the ReLU activation, serve as a feature masking operation. Based on these observations, the authors propose RobMask, which uses a linear combination of such rescaling operations to achieve improved generalization in deep networks. \n\n\nPros: \n1) Interesting observation, and highlights the needs to re-examine techniques from the standard training paradigm that are directly used in adversarial training of deep networks \n2) Easy to integrate with any network architecture that already uses Batch Normalization\n3) Demonstrates enhanced standard performance over different network architectures and datasets\n4) Achieves improved trade-off between clean and robust accuracy for adversarially trained networks on smaller constraint sets (L-infinity eps = 2/255 to 6/255)\n\nCons: \n1) The paper lacks novelty from the standpoint that very similar observations were made by Xie et al. [1]. \n2) Further, the primary results are demonstrated for the case of k=2, using a linear combination of Batch Normalization parameters obtained for normal and adversarial training, which represents only a minor change from the algorithm proposed in [1]. \n3) While the method itself is not difficult to understand, it is still unclear why it helps strike a better balance for the accuracy-robustness trade-off in adversarially trained networks. Could the authors provide an intuitive or theoretical explanation for the same?\n4) The adversarial training of larger networks such as DenseNet-121 can be highly computationally intensive, requiring almost an additional order of magnitude in training time. Thus the performance comparison made in Table-3 at the 20th epoch for models trained using normal training and RobMask is unfair, since RobMask uses 7-step adversarial training. Perhaps a better metric to consider would be the standard performance obtained after a fixed training time.\n5) The clean accuracy shown in Table-1 and Table-3 for the adversarially trained ResNet-18 model on CIFAR-10 is quite low (78%), compared to standard values reported in [2,3], which is often around 82%. \n6) To quote from the last para of Section4: “Also, the proposed RobMask method is more general than Advprop, and AdvProp can be considered as one special case of RobMask when we set the linear layer rank k to 2 and freeze p = 1 in the whole training process.” Thus, could the authors provide additional results for k=3 or k=4? This is quite important to set apart the proposed method from AdvProp, which is highly similar, as final results are only reported with k=2 in Table-3.\n7) In test time, it is not immediately clear what choice of $u_i$ should be used, since it determines the effective batch normalisation parameters that are used. Could the authors clarify the exact BN parameters used in final evaluation on clean and adversarial samples?\n8) When k is set to a value larger than 2, it is not immediately clear what the different $w_k$’s would represent, since the 2-dimensional $u_i$’s already encode information about different $\\ell_\\infty$ constraints. Could the authors clarify this? Further, it would be beneficial to the reader to include an example with k=3, along similar lines to what is presented in Section 4. \n9) Evaluation on PGD-100 step attack alone is not sufficient, as with stronger attacks such as MultiTargeted attack [4] and AutoAttack [5], the difference in adversarial accuracies might be much lower for eps=2/255 to 6/255, as reported in Table-3.\n10) Since the authors say that the proposed methods does not achieve better adversarial accuracy for eps=8/255 in Table-4 due to the sampling of p, could the authors show results where the maximum constraint is set to 10/255 to show an improvement for the standard evaluation setting of 8/255? This would greatly help in the comparison of different methods, particularly the clean accuracy of different models.\n\n[1] Xie, C., Tan, M., Gong, B., Wang, J., Yuille, A. L., & Le, Q. V. (2020). Adversarial examples improve image recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 819-828).\n[2] Wong et al. Fast is Better than Free: Revisiting Adversarial Training, ICLR 2020\n[3] Rice et al., Overfitting in adversarially robust deep learning, ICML 2020, https://arxiv.org/abs/2002.11569 \n[4] Gowal et al., An Alternative Surrogate Loss for PGD-based Adversarial Testing, https://arxiv.org/pdf/1910.09338.pdf\n[5] Croce et al., Reliable Evaluation of Adversarial Robustness with an Ensemble of Diverse Parameter-free Attacks, ICML 2020\n\nIn summary, although the observations presented in this work are interesting, further analysis and thorough evaluations are required to justify the claims made in this paper.\n\nExpecting the authors to address the following during rebuttal period: \n \n- Please address and clarify the cons as listed above.\n- Could the authors please provide additional details on how the finetuning is performed in Section 3.1?\n- In Figure1, could the authors clarify the quantity along the x-axis? If it represents iterations, or epochs?\n- Could the authors clarify how the plot for Adv-Train is obtained (in relation to the Std-BN finetune plot)? How were the updates performed on the Adv-Train model to obtain this plot?\n- Given that the parameters of the first BN layer are presented in Figure1, since the convolutional layer that occurs before this BN layer is frozen, it is expected that Plots 1(a) and 1(b) are identical for the two cases, and does not offer additional insight. \n- Could the authors comment on the behaviour of BN parameters in deeper layers of the network when the same fine-tuning experiment is performed?\n- In Figure2, we observe that very few feature maps are indeed changed. Could the authors comment on this? Also, could the authors clarify which layer was used to obtain these figures?\n- In Algorithm 1, could the authors clarify if the number p is sampled uniformly from the [0,1] range?\n- Further, it is not clear which set of BN parameters is used in the crafting of the PGD attack in the proposed algorithm. This small detail is likely to cause dramatic changes in the final outcome of training. As currently presented, it appears as though the BN parameters corresponding to standard training are used in this step.\n- Could the authors show an ablation experiment where the BN parameters corresponding to RobMask are used for attack generation to understand this better?\n- Could the authors clarify if the network parameters $\\theta$ are also adversarially updated (along with W and W’) using the loss on the perturbed sample $x+\\delta$ as presently shown for all experiments? If so, could the authors clarify why the accuracy for eps=0 in Table4 differs from that shown in Table3 (5-6% difference)?\n- Also, it is not completely clear from the algorithm if BN parameters of all layers in the network are updated, or if it is restricted to some specific layer.\n- Could the authors share details used for adversarial training (optimizer, learning rate schedule, number of epochs, validation split, use of early stopping)? These factors play a crucial role in the final robust accuracy achieved and in the trade-off with clean accuracy as well, as often the model obtained at the last epoch of training achieves lower adversarial accuracy compared to intermediate epochs.\n\n########################## Update after rebuttal ##########################\n\nI thank the authors for their detailed response; several concerns have been addressed in the rebuttal. I would encourage the authors to use commonly used practices to improve the robust performance of models in Tables 5 and 6. The use of early-stopping [3] can significantly boost the robust performance, and produce models with better clean accuracy than is presently reported for Adv. Training. I would like to update the score to 5 based on the author's response. Aside from the robust evaluation, the improvement in clean accuracy is of a relatively smaller magnitude given the disproportionate increase in training requirements. Thus, I have not further increased the score.\n\n[3] Rice et al., Overfitting in adversarially robust deep learning, ICML 2020, https://arxiv.org/abs/2002.11569 \n", "extra_scores": {}, "extra_text": {}}, {"review_id": "KuEnOsFHYdP", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper observes that the rescaling operation in the batch normalization layer and the ReLU activation learn to select different features for standard and adversarial trainings. The authors call this effect \"Adversarial Masking.\" Based on this observation, the authors then propose Rob-Mask that achieves good standard and adversarial accuracies at the same time.\n\nThe trade-off between standard and adversarial accuracies are an important problem in adversarial machine learning. The findings in this paper are really interesting and deserve further investigation from the research community.\nMy biggest concern is that, since it is an empirical paper without rigorous guarantees, I would like to see the experiments on more datasets. The experiments in this paper are only on CIFAR-10 and CIFAR-100, which are similar datasets. Does the finding still hold in other datasets like ImageNet?\nAlso, there are many models that do not use Batch Normalization, but the standard accuracies also drop when doing adversarial training. The proposed hypothesis cannot explain that. One ablation test that I would like to see is whether fine-tunning a single convolutional layer will have similar effect as fine-tunning the BN layer done in this paper.", "extra_scores": {}, "extra_text": {}}, {"review_id": "BWhOnU82YNU", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "## Overview \n\nThe paper focuses on the generalization issue with adversarial training that various work has recently demonstrated. The paper studies the role of batch normalization (BN) in adversarial robustness and generalizability. The authors single out the rescaling operator in BN to significantly impact the clean and robustness trade-off in CNNs.  They then introduce Robust Masking (Rob-Mask), which is shares similarities to the CVPR2020 paper by Xie et al. (2020). Xie et al. use an auxiliary BN in adversarial training, which uses different batch normalization parameters for adversarial samples, to improve the generalizability of CNNs. Still, the authors clearly state the differences between Rob-Mask and AdvProp. \n\n## Contributions\n\nThe contributions of the paper are as follows:\n\n1. Showing the effect of BN (and, more specifically, the scale parameter of BN together with ReLU) as adversarial masking.\n\n   a. Authors show that *adversarial fine-tuning* of only the BN parameters of a *vanilla-trained* network provides some adversarial robustness, although at the trade-off losing test accuracy. \n\n   b. Authors show that *standard fine-tuning* of only the BN parameters of an *adversarially-trained* network increases the network's generalizability, although at the trade-off losing robustness. \n2. Showing that interpolating between the BN parameters in Contribution 1 provides a smooth trade-off between generalizability and robustness.\n\n3. Devising an approach for utilizing different perturbation strengths for model training. The authors build on their Contribution 2 and propose $k$ basic (or better to say principle) rescaling parameters, the linear combination of which leads to a rescaling parameter. \n\n4. Providing a short yet informative, ablation study to show the effectiveness of Contribution \n5. Showing experimental benefits over AdvProp on CIFAR10 and CIFAR100 datasets.\n\nContribution 3 turns AdvProp into a particular case of RobMask.  In fact, Xie et al. (2020) mention in their paper that \"a more general usage of multiple BNs will be further explored in future works,\" which seems to be the inspiration behind this paper.  \n\n## Weaknesses\n\n1. The main limiting factor for the impact of this paper is the experiments. The paper only reports performance on CIFAR10 and CIFAR100. Given that the paper can be considered an extension/improvement over AdvProp, it is desirable to have similar largescale experiments in Xie et al. (2020) on ImageNet and its variations. A head-to-head comparison with the experiments in Xie et al. (2020) would provide a clearer picture to show the proposed method's power.\n2. Regarding the practicality of the approach, I am missing a computational analysis of the approach to compare it against BN and AdvProp, e.g., it would be great if the authors provided a head-to-head comparison of training curves. Does your method take much longer to train?\n3. How many times did you run each experiment? What are the standard deviations in Table 3 (and other tables)? Providing this information, at least in the supplementary materials, could clarify your results' statistical significance.\n\n## Questions and comments for the authors\n\n1. The notation $\\gamma_i$ is used both for BN's scaling parameter and for the learning rate, which turns the equations hard to follow.\n2. On the bottom of page 7, you wrote: \"It is because both AdvProp and Adversarial training models are trained with adversarial examples generated with \u000f$\\epsilon= 8/255$, while our methods use a random perturbation where \u000f$\\epsilon_{max}=8/255$.\"  The term \"random perturbation\" is misleading here, as I believe you are also using PGD attack, but the adversarial perturbation's strength is randomized. Is that correct?\n3. Please refer to Weaknesses 2.\n4. I don't find Figure 2 informative at all. I suggest that the authors remove the figure and use the space to address the raised concerns.\n\n## Evaluation logic\n\nI find the paper an interesting extension of the CVPR2020 paper by Xie et al. However, the paper's experimental section does not provide enough information to the reader to see the concrete benefit of the proposed method in training a large scale CNNs. I think the paper could significantly benefit from a more extensive experimental setting. Given the limited novelty and lack of concrete evidence of practicality, I score the paper as a 5. \n\n## Post rebuttal evaluation\n\nI thank the authors for providing answers to the raised questions and providing further experiments. Regarding Figure 3, I suggest that the authors provide accuracy as a function of wallclock instead of epochs currently reported in the paper. As a result of the authors' responses, I increase my score to 6. ", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviews were a bit mixed, with some concerns on the incremental nature of this work, which the AC concurs (after independently going through both the submission and Xie et al 2020). In a nutshell, the main contribution on the authors' side appears to be a simple linear interpolation of two masks so that it is possible to leverage attacks with varying strengths. Other claimed contributions are not substantiated. In particular: 

(a) Fig 1 and its conclusion are a bit disturbing. It is suggested that the authors back up their claim with more empirical and theoretical evidence. For example, why can one conclude from the same mean and variance that there is no distribution mismatch between clean and adversarial examples (as claimed in Xie et al)? If the two sources have similar distribution, why is there a sharp difference in gamma for the two? When one claims different results from previous work, due diligence is required. For instance, did the authors reproduce the mean and variance on the same architecture and dataset of Xie et al? How about other BN layers (in addition to the first one)? How to explain the difference in gamma? The fact that you are using different masks for different epsilon is an indication that their distributions are probably different. The authors mentioned the joint effect between gamma and relu activation, which could be potentially insightful. However, this is a bit speculative in its current presentation. How about an ablation study with leaky relu or tanh/sigmoid? Without these careful comparisons, these claimed contributions are not appropriate to publish in their current form.

(b) As the reviewers pointed out, why BN, after all for models that do not use BN they still suffer from adversarial examples? Even when we restrict to models that use BN, why replicating BN for different sources helps generalization? Here, an excellent experiment point is to compare to fine-tuning or replicating other layers in the network. During rebuttal, the authors only tried to fine-tune ONE convolution layer and quickly concluded its ineffectiveness. Note that in contrast the authors fine-tuned ALL BN layers. How about all convolution layers, some pooling layers, the last softmax layer? These experiments could help us understand if there is some magic in BN. Or maybe it is just more convenient to fine-tune BN because of its small number of parameters? In any case these experiments would largely strengthen the findings of this work.

(c) As pointed out by the reviewers, Xie et al hinted at the advantage of using multiple BN masks and the authors proposed to linearly interpolate the masks. In the ablation study, what if we increase the number of BN masks in Xie et al, say we discretize p into 11 values p = {0, 0.1, ..., 0.9, 1} and have 1 mask for each value? Here an interesting experiment is to compare K = 11 (basically AdaProp) with smaller K (such as 2 or 5). The authors seemed to suggest that a larger K does not seem to help, which would be clarified through the preceding experiment (and perhaps more). Note that using a uniformly random p is equivalent as adversarial training with a weaker (and varied) attack, and the better tradeoffs shown in the experimental section (e.g. Table 3) are perhaps expected.

(d) As pointed out by the reviewers, a head to head comparison against AdaProp (preferably with more masks) is desirable. The authors mentioned some difficulty in conducting this experiment fully. If it is only the software side, maybe check the sources here: 
https://paperswithcode.com/paper/adversarial-examples-improve-image

(e) Finally, a minor point: Algorithm 1 with k=2 and p=1 does not reduce to AdaProp as one will only train on adversarial examples and ignore all clean samples? 

In the end this submission appears to be a bit incremental. However, the authors are strongly suggested to follow the reviewers' comments to further polish their work and address the concerns above. With proper revision this work can eventually become a solid contribution on top of AdaProp.
Decision: Reject