Encoding Robustness to Image Style via Adversarial Feature Perturbations
Authors: Manli Shu
Zuxuan Wu
Micah Goldblum
Tom Goldstein
Venue: NeurIPS 2021 Poster (2021)
Keywords: adversarial training
domain generalization
out-of-distribution

Abstract: Adversarial training is the industry standard for producing models that are robust to small adversarial perturbations.  However, machine learning practitioners need models that are robust to other kinds of changes that occur naturally, such as changes in the style or illumination of input images. Such changes in input distribution have been effectively modeled as shifts in the mean and variance of deep image features. We adapt adversarial training by directly perturbing feature statistics, rather than image pixels, to produce models that are robust to various unseen distributional shifts. We explore the relationship between these perturbations and distributional shifts by visualizing adversarial features. Our proposed method, Adversarial Batch Normalization (AdvBN), is a single network layer that generates worst-case feature perturbations during training. By fine-tuning neural networks on adversarial feature distributions, we observe improved robustness of networks to various unseen distributional shifts, including style variations and image corruptions. In addition, we show that our proposed adversarial feature perturbation can be complementary to existing image space data augmentation methods, leading to improved performance. The source code and pre-trained models are released at \url{https://github.com/azshue/AdvBN}.

Encoding Robustness to Image Style via Adversarial Feature Perturbations
Manli Shu'
Zuxuan Wu²
Micah Goldblum
Tom Goldstein'
University of Maryland, College Park, USA
Fudan University, Shanghai, China New York University, New York, USA
manlis@cs. umd. edu, zxwu@fudan. cn, goldblum@nyu.edi edu, tomg@cs und edu
Abstract
Adversarial training is the industry standard for producing models that are ro- bust small adversarial perturbations. However, machine learning practitioners need models that are robust to other kinds of changes that occur naturally, such changes the style illumination input images. Such changes in input distribution have been effectively modeled as shifts in the mean and variance of deep image features. We adapt adversarial training by directly perturbing feature statistics, rather than image pixels, to produce models that are robust to various un- seen distributional shifts. We explore the relationship between these perturbations and distributional shifts by visualizing adversarial features. Our proposed method, Adversarial Batch Normalization (AdvBN), is single network layer that generates worst-case feature perturbations during training. By fine-tuning neural networks on adversarial feature distributions, we observe improved robustness of networks various unseen distributional shifts, including style variations and image corrup- tions. addition, we show that our proposed adversarial feature perturbation can be complementary to existing image space data augmentation methods, leading improved performance. The source code and pre-trained models are released at AdvBN.
https: //g::://iiii /gggttu.b......
Introduction
(8) ImageNet
(b) LmaatNa--Instarram
(c) ImaggeNot-C
(4) ImageNet- Sketch
(c) Stylized ImageNet
(() ImageNeet-AdBBN
99 0% goldffnch
80 8% goldfinch
57.4% geldfinch
$1 1144% black grouse
16.2% gong
10.3%
hon-ef-hh-erodc
0.05% bubuul
7.4% sulpbur buttertly
11.8% beambling
6.7% joy
8,8% belete
5.1% Iszznn hound
0.02%
house Ernch
0.5% humminghee
888% puilotine
6.2% coucal
4.5% fox squime
4,0% Samingo
Figure Images from ImageNet variants along with classification scores by a pre-trained
ResNet-50. The image of column (a) is from ImageNet validation set. Dataset of column (d) collected independently of the ImageNet dataset. Dataset of Column (f) generated by our Adversarial Batch Normalization module. Details on how generate column (f) be found in Section 3.
Robust optimization for neural networks has been a major focus of recent research. A mainstream approach reducing the brittleness classifiers adversarial training, which solves a min-max
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

optimization problem in which an adversary makes perturbations to images to degrade network performance, while the network adapts its parameters to resist degradation [9, 18, 25]. The result is a hardened network that is no longer brittle to small perturbations to input pixels. While adversarial training makes networks robust to adversarial perturbations, does not address other forms of brittleness that plague vision systems. For example, shifts in image style, lighting, color mapping, and domain shifts can still severely degrade the performance of neural networks [12].
We propose adapting adversarial training to make neural networks robust to changes in image style and appearance, rather than small perturbations at the pixel level. We formulate a min-mar main-max game in which an adversary chooses adversarial feature statistics, and network parameters are then updated to resist these changes in feature space that correspond to appearance differences of input images. This game is played until the network is robust to a variety of changes in image space including texture, color; brightness, etc.
The idea of adversarial feature statistics is inspired by the observation that the mean and variance of feature maps encode style information, and thus enable the transfer of style information from source image to a target image through normalization [15, 40J. Unlike standard approaches that rely on feature statistics from auxiliary images define an image style, we use adversarial optimization of feature statistics to prepare classifiers for the worst-case style that they might encounter.
We propose training with Adversarial Batch Normalization (AdvBN) layers. Before each gradient update, the AdvBN layer performs an adversarial feature shift by rernormalg with the most damaging mean and variance. By using this layer in robust optimization framework, we create networks that are resistant to various domain shifts representable by shifts in feature statistics. An advantage of this method is that it does not require additional auxiliary data from new domains. We show that robust training with AdvBN layer hardens classifiers against changes in image appearance and style [6, 45], as well as common image corruptions [12]. Besides classification, the effectiveness of AdvBN is also shown in the task of semantic segmentation, where it improves cross-domain generalization.
Related work
Adversarial training. Adversarial training and its variants [7, 25, 32, 49] have been widely studied for producing models that are robust to adversarial examples [26, 36] through solving min-max optimization problems. Besides defending against adversarial attacks, recent work has shown that adversarial training can be effectively applied many other tasks [8. 17, 30. 53, 55]. Adversarial data augmentation [34, 42, 54] proposes to generate worst-case unseen domains using data augmentation and an adversarial loss, thus improving the generalization of neural networks. Another work [46] interprets the original formula of adversarial training as type of data augmentation and reveals the distributional discrepancy between the feature representations of adversarial examples and clean data. The combination of adversarial training and feature statistics has been studied in previous work [24, 27], where has been used to defend against adversarial attacks [27], or to generate feature distributions [24]. Our method differs from previous work in both the target of perturbation and the objective function. We craft adversarial feature distributions by directly perturbing the mean and variance of feature maps instead of through variational auto-encoder [24], and our objective does not include regularization terms.
Feature Perturbation. Feature perturbation has been an effective approach to generate novel data distributions from given source domain [14, 19, 22, 39]. Feature perturbations be applied in different ways, such as adding spatial noise to the feature maps [37] or transforming feature maps using classical signal processing [22]. We will specifically discuss feature perturbation through re-normalization. While feature normalization [16, 40] is first proposed to accelerate neural network training [1], the mean and variance of deep feature representations have been shown effectively capture image style information, and style transfer can be realized through feature re- re-normalization [15]. The idea of feature re-normalization has also been adopted to help neural networks adapt from the source domain to a target domain, using feature statistics obtained from the latter [21, 31]. Recent work [20] also proposes use ne-oomaalzaaionn for feature interpolation to improve the generalization capability of neural networks. Instead of re- re-normalizing features with statistics of other samples or from other domains, we simulate the worst-case scenario, encouraging models to be less sensitive to style information and thus to generalize better to images of varying appearances.

---

Robustness to distribution shifts. The definition of "distribution shifts" varies from one topic to another. For example, distribution shifts in the meta-learning literature[19, 37] often refer to the discrepancy in discriminatory feature distributions of novel tasks from different domains. Another definition is the subtle difference between training and testing data that are sampled from the same underlying distribution [38, 50]. This work mainly focuses on distribution shifts across image data that amount to major "style" discrepancy, including variations in illumination [6, 45], weather condition [29], and image quality degradation [12]. Various methods have been proposed to produce neural networks that generalize well to this type of distribution shift, including test time training [35], test time adaptation [43], training with noise [30], and novel network architectures [28], etc. Data augmentation is a popular method which is designed to increase the diversity of training data and prevent neural networks from over-fitting Recent studies in data augmentation have proposed to use novel augmentation operations [6, 13, 48, 51], policy searching [3. 10, 23), and adversarial approaches [17, 42, 55], etc. However, the potency of data augmentation can be limited by the choice of applicable augmentations [2, 5]. Feature space augmentation through feature interpolation [20, 41 on the other hand, is not restricted to the family of image transformations. Our method also works in feature space, but instead of interpolating, we perturb feature statistics adversarially.
3 Adversarial Batch Normalization
We propose Adversarial Batch Normalization (AdvBN), module that adversarially perturbs deep feature distributions such that the features confuse CNN classifiers. We iteratively compute adversarial directions in feature space by taking PGD steps on batch statistics. In the next section, we will train on these perturbed feature distributions in order to produce models robust to domain shifts.
Consider a pre-trained classification network, g. with L layers. We divide g into two parts, gid and g'+, where g.t." denotes layers m through n. Now, consider a batch of data, x, with corresponding labels, y. Formally, the AdvBN module is defined by
(I-OEn) o(f)
BN adv (z: (.g...)) v) = 8'g o(()
+5/ 5.h-/i -u(f), where f= gz))(x),
(1)
(6,, = arg max C 1 dr -S- + +5p (p-mm())
(8m)8))
(2)
subject to ||6 - 11100 S e, 11ó, 11/00 S
where L is the cross-entropy loss, and the maximization problem is solved with projected gradient descent. Note that the feature input of g g/+1,1 is simplified form of the AdvBN formulation in Eq (1), where the two off)ss 's cancel out. Simply put, the AdvBN module is a PGD attack on batch norm statistics which can be inserted inside network. 5,,,5, are vectors with length equal to the number of channels in the output of layer I, and we multiply by them entry-wise, one scalar entry per channel, similarly to Batch Normalization. Additionally, note that this module acts on a per-batch basis that features corresponding to the same image are perturhed differently across training epochs as training samples are randomly shuffled between epochs during training.
In Eq (I), we re-normalize the feature by adding adversarial statistics 8,, u(f), rather than simply 5p. that lo bounds and steps size do not need depend on v(f). Intuitively, we permit the mean of adversarial features to vary more when the clean features have a mean of higher magnitude.
Visualizing feature shifts. We adopt the VGG [33] based autoencoder from AdaIN [15], in which the encoder is the first few layers of a pre-trained VGG-19 19 image classification network. The decoder is trained to restore the input image from the output of the encoder. After we obtain autoencoder that can perform the identity mapping on input images, we plug in an AdvBN module after the encoder. To compute cross-entropy loss for the AdvBN module, the remaining downstream layers of the aforementioned VGG-19 classifier are used, which takes the encoded feature as input and outputs the class prediction. The features perturbed by AdvBN are then fed into the decoder to create our visualizations.
In Figure 2, images with adversarial feature distributions exhibit differences in color, texture, and edges. We draw two major conclusions from these visualizations which highlight the adversarial properties of these domains. The first one concerns textures: CNNs have been shown to rely heavily
3

---

(c)
(h)
Figure 2: Examples of ImageNet images with adversarial feature distributions shifted by Ad- vBN, visualized through a decoder. For each pair, the original image is on the top.
on image textures for classification [6]. Images from the adversarial domain, on the other hand, have inconsistent textures across samples. For example, the furry texture of a dog is smoothed in Figure (b), and the stripes disappear from a zebra in Figure 2 (d), whereas visible textures appear (f) and (g) of Figure 2. The second conclusion pertains color. Previous study [52] suggests that colors serve important information for CNNs. In the adversarial domain, we find suppressed colors (Figure (a), (c)) and unnatural hue (Figure 2 (c), (g)). Figure 3 illustrates how the appearance of reconstructed images shifts as adversarial perturbations to feature statistics become larger. See Appendix B for additional examples.
Original
m
m 2
m
m
m
5
m
m 7
Figure 3: The effect of adversarial strength on visualized examples. m denotes the number of PGD steps.
0 corresponds to images reconstructed by our autoencoder without AdvBN.
We use this visualization technique to process the entire ImageNet validation set and denote it as ImageNet-AdvBN in Figure 1. By evaluating different methods on this dataset, observe that performance on ImageNet-AdvBN consistently degraded, which validates the adversarial property of features generated by AdvBN. Experiments concerning performance ImageNet-AdvBN are listed Appendix B.
Training with Adversarial Batch Normalization
In this section, we use the proposed AdvBN module to train networks on the perturbed features. The goal is to produce networks that generalize well to unseen domains while maintaining performance on the training distribution, all without having to obtain auxiliary data from new domains.
We start with pre-trained model, g = g ++,,1 gh.,,, and we fine-tune the subnetwork, g++1,4, on clean and ad adversarial features simultancously. To this end, we solve the following min-max problem,
min E (x. max L(g 1+1.6 BN adv ( + L(g (41L ( g''(x).v)
(3)
where C denotes cross-entropy loss, and D is the distribution of batches of size n. In order to maintain network's performance on natural images, we adopt a similar approach to Xie et al. [46] by using auxiliary batch normalization in -1+ for adversarial features; we use the original BNs when propagating clean features, and we use auxiliary ones for adversarial features. See Algorithm for detailed description of our method.

---

Since we start with pre-trained models, we only need to fine-tune for 20 epochs, yielding improved robustness with little additional computation. Moreover, we only modify the parameters of later layers, so we do not need to backpropagate through the first half of the network. See Appendix C for an analysis of the training budget using our method. In the following section, we measure the performance, on several datasets, of our model fine-tuned using adversarial training with AdvBN.
Algorithm 1: Training with Adversarial Batch Normalization
Input:
Training data, pretrained network -+I. ag* gh learning rate PGD bound and PGD step
size T, loss function &
Result: Updated network parameters, 0, of subnetwork 96
141.L
for each training step do
Sample mini-batch x with lahel y; 1d (z);
Ohtain feature map f = g
Initialize perturbation: ó (6 6g);
Let Sado =:
for adversarial step I, ..... ........ m do
fado + 8g (f - p(f)) + 5n u(f);
Update 6:
d 8 T. sign(V Vv V& +1,1 (fadn). v)):
ō + clip (6. - c);
end
Jadv 8. (S m(f)) + 8p p(f):
Update using gradient descent:
8 - 8 - a Vollgo (fado). v) + ((ob+..f((f)u))))
end
return 0
Experiments
In section 5.1, we evaluate our method on image classification. We measure the generalization of models on ImageNet variant datasets that features different distributional shifts. We provide ablation studies our method Section 5.2. A feature divergence analysis is presented in Section 5.3 that validates the effectiveness of our method from different perspective. In section 5.4, we evaluate our method on semantic segmentation. We conduct cross-domen evaluations on traffic scene datasets with different weather conditions and traffic scenes.
5.1 Evaluation on ImageNet Variants
In this section, we evaluate our method in the context of image classification on ImageNet variant datasets. first evaluate the standalone performance of our method. We also include other baseline methods, including image space adversarial training adapted from the standard PGD adversarial training [25]. MoEx [20] is another related method that performs feature space augmentation through feature re-normalization. SIN [6] is trained both Stylized ImageNet and original ImageNet, which uses AdaIN [15] as the style transfer method.
In addition, we examine AdvBN as feature space augmentation method by showing its potential be complementary image space augmentation. We consider state-of-the-ar data augmentation methods, including AutoAugmen(3]] Fast AutoAugment (AA)]23). CutMix[48), AugMix[133 and AdvProp[46). We show that our method can further improve the generalization of models trained with advanced data augmentations. Results of all methods included in this section are based on the ResNet-50 model architecture. Our method also works well on other architectures, and we provide results in Appendix A.
Implementation details. Our model begins with an ImageNet pre-trained ResNet-50 [11]. We insert the AdvBN module the end the 2nd convolutional stage (conv2_3). We then fine-tune the model with our method following Algorithm for 20 epochs. The learning rate starts at 0.001 and decreases by factor of 10 after 10 epochs. Our batch size set to 256. We use SGD with momentum of 0.9 and weight decay coefficient 10 We search for the optimal number of adversarial
5

---

steps m by increasing m while fixing the step size to b 0.2, and we bound the perturbation with -0.1. The optimal m we find through this procedure m 6. When using AdvBN to improve given data augmentation method, we apply this fine-tuning procedure on a model trained with the data augmentation, except for AA. Due to the absence of pre-trained AA model, we manage to fine-tune base model jointly with a fixed AA policy and AdvBN, and we compare it to model solely fine-tuned with AA. All models trained with AdvBN that appeared in this subsection are obtained by following the same training routine and hyperparameter settings that specified above.
Datasets. To measure the generalization ability of image classification models, we evaluate our models on four variants of ImageNet [4]:
ImageNet-- [12] (under the Apache License 2.0) contains distorted images with categories of common image corruption applied, each with 5 levels of severity. Performance on this dataset measured by mean Corruption Error (mCE), the average classification error over a total 75 combinations of corruption type and severity level, weighted by their difficulty. ImageNNet-Instarree (ImageNet-Inss [45] is composed of ImageNet images processed with a total of 20 different Instagram aesthetic image filters. Filters are applied separately, and the dataset contains 20 sub-datasets, each corresponding one type of image filter.
ImageNet-Sketch [44] (under the MIT License) dataset of black and white sketches. The dataset includes 50,000 images in total, falling into 1,000 ImageNet categories, with 50 images per category. Images this dataset are collected independently from the original ImageNet validation set through Google Image queries. Details concerning the construction this dataset can found in Section 4.4 of Wang et al. [44].
Stylized ImageNet (ImageNet-Style) [6] (under the MIT License) consists of images from the ImageNet dataset, each stylized using AdaIN [15] with randomly selected painting.
Textures and colors of images in this dataset differ heavily from the originals.
Table 1: Evaluation ImageNet variants. All methods are implemented based ResNet-50. Performance on ImageNet-C measured by mean Corrupted Error (m3))[13).
Method
ImageNet-C mCE +
ImageNet-Ins. Top-1 acc. f
ImageNet-Sketch. Top-1 acc.1
ImageNet-Style Top-1 acc.1
Standard Training
76.7
67.2
24.1
7.4
Adv. Training
73.7
68.2
25.3
9.1
MoEx (w/ Cutmix)
74.8
70.0
24.0
5.0
SIN
73.8
68.5
26.9
10.4
AdvBN
72.7
69.5
27.9
11.9
AdvProp
70.7
69.2
18.0
9.0
AdvProp + AdvBN
69.5
69.3
28.7
12.6
Cutmix
74.7
70.3
23.8
5.3
Cutmix + AdvBN
72.1
70.9
27.2
8.2
AutoAugment" (AA) (AAA)
72.1
70.1
26.7
8.2
AA AdvBN
68.6
71.1
30.3
14.1
Fast AA
68.7
71.1
27.2
8.3
Fast AA + AdvBN
68.7
71.3
28.6
11.4
Augmix
68.4
70.4
28.5
11.2
Augmix + AdvBN
64.6
71.1
28.7
13.6
Model details. Models trained using other methods that we include in this section are ResNet-50 models released by the authors of the original work, except for methods of which an official ResNet- 50 is not available. The released MoEx model is trained collaboratively with CutMix. The "Adv. Training" baseline is adapted from PGD adversarial training, for which we adopt auxiliary batch normalization alleviate the performance degradation on non-adversarial images, and the model is obtained through fine-tuning on standard trained model. AdvProp does not provide a ResNet-50 model, and we use the open source implementation! and our reproduced model matches the accuracy reported in the aforementioned implementation. Our reported performance using AA, denoted as
hhttps://gitee httpp/////ggitoooo/////eeeeeeeuu http::/hhh......oooooooo///////a
(MIT License)

---

"AA"", is obtained through fine-tuning pre- pre-trained base model for the same number of epochs as AdvBN, using fixed set of augmentation operations found by AA, which is included as a reference to the performance of "AdvBN- AA" The augmentation policy that we use is from the original work and can be found in the open source implementation".
Results. In Table I, we evaluate the performance of models on the four variant datasets. The stan- dalone AdvBN improves the generalization of standard model on every dataset and is competitive with alternative methods. Additionally, we find that our method is complementary to imput space data augmentation, consistently boosting the performance of ofate-of-the-ar data augmentation methods. Note that our model has auxiliary BN layers, so its performance on the original ImageNet is well maintained, will be shown in the next subsection. Details concerning inference in Appendix A.
5.2 Ablation Study
Where should the AdvBN module be placed within a network? The proposed AdvBN module can be inserted after any layer network. In this section, we try AdvBN after other layers, namely conv3_4 4 and conv4_6. 6. For the ablation study, all of our models are obtained by following the same fine-tuning setting found in subsection 5.1, but with a fixed AutoAugment policy as data augmentation. In Table 2, we observe that conv4_6 6 yields the worst performance among all three ImageNet variants, indicating that using AdvBN at such deep layer is not as helpful as at shallower layers. We hypothesize two possible explanations for this phenomenon: (1) there are fewer trainable parameters when only very deep layers are fine-tuned;
(2) features are more abstract at deeper layers, and perturbing these high-level features can lead to extremely chaotic feature representations that are harmful for classification.
Table 2: Ablation studies the positioning of AdvBN. The base model is ResNet-50.
ImageNet topl acc. +
ImageNet-C: mCE. +
ImageNet-Ins. topl acc. f
ImageNet-Sketch. topl acc.
ImageNet-Style topl acc. 1
Model
Base model
76.1
76.7
67.2
24.1
7.4
= conv2_3
76.5
68.6
71.1
30.3
14.1
== conv3_
76.2
70.0
70.2
33.2
19.5
== conv4_ 6
75.3
75.0
68.5
26.1
11.0
deas ImseeNNe
ImageNes-C
ImsgeNc-Its.
Stale
1
39
30
14
15
35
(8)
(h)
(4)
(d)
(e)
Figure 4: The effects of adversarial strength. The y-axis of (b) ImageNet-C: is the mean Corrupted Error (mCE), and the others are top-1 accuracies. X-axes are the number of PGD steps m.
Adversarial strength. The strength of the adversarial attack in the adversarial training framework has a major impact model performance [25]. We test a range of PGD parameters to demonstrate how the strength of AdvBN affects model performance. We measure strength by the perturbation number of PGD steps m, where we fix to be 0.2 for all settings and fix the perturbation bound e mm 7] 0.1 for different m's.
Results concerning the impact of adversarial strength are shown in Figure 4. We can see that the clean accuracy on ImageNet decreases as the number of steps m grows. On the other datasets, we can observe a turning point, where the performance reaches optimality. This behavior is expected because small perturbations cause small changes to features which may help maintain the clean accuracy but cannot help improve model's generalization to other domains; overly large perturbations are also less beneficial as the resulting features can too noisy.
http:p/gggggttlocoooe/eeet/ttttt (MIT License)
7

---

5.3 Feature Divergence Analysis
We compare the features extracted by our network to those of a standard ResNet-50 trained on ImageNet. Following Pan et al. [28], we model features from each channel using a normal distribution with the same mean and standard deviation, and we compute the symmetric KL divergence between the corresponding distributions on the two datasets (A and B). For two sets of deep features, FA and (lB) is computed using the formula,
FB, each with C' channels, the divergence (F
1 Σ (KL(FAIFE) + (KL(FIIFAA) D(FAIlFR) =
(4)
KL (LFF))FF) log OB +
AA (u'a vB )²
1
(5)
20 where F denotes the features of i-th channel with mean n² and standard deviation of.
2
Sandani
Oze
Sandanl O2n
1.5
Stadded
Ouex
Feature divergence
Feature divergence
Feuture divergence
4
2.
0.5
10
11
12
13
14
15
16
8
10
11
12
13
14
15
16
0.0
10
11
12
15
14
15
16
Block index
Block index
Block index
(a) ImageNet vs. ImageNet-Ins.
(h) ImageNet vs. ImageNet-Sketch (c) ImageNet vs. ImageNet-Style
Figure 5: Feature divergence between pairs of datasets. Features are extracted by standard and an AdvBN fine-tuned ResNet50.
In Figure 5, we compare the baseline model with our three pairs of datasets in the fine-tuned layers. Since ImageNet-Instagran contains 20 filter versions, we use the "Toaster" filter found in [45] to cause the sharpest drop in classification performance.
Feature divergence in our network trained with AdvBN is substantially smaller in the deeper layers of the fine-tuned subnetwork. In other words, the distribution of deep features corresponding to shifted domains is very similar to the distribution of deep features corresponding to standard ImageNet data. The small divergence between feature representations explains the effectiveness of AdvBN from different angle and explains why our model generalizes well across datasets.
5.4 Generalization on Semantic Segmentation
Datasets. We present domain generalization results on the Synthia video sequences dataset [29], consisting of multiple sub-datasets featuring traffic situations under different weather, illumination, and season conditions. We conduct experiment on 10 sub-datasets that include two different road scenes: "Highway" and "New York-like City", each one with 5 different domain shifted variants: "dawn". "fog", "night", "spring" and "winter". Figure 6 shows sample images from each of the 10 sub-datasets. We use the left-front view images of each sub-dataset, and split dataset by randomly selecting 900 images for training and 500 for validation. We separately train two sets of models on the "Highway/ Dawn" and "New Yirk-like City/ Spring" datasets and evaluate them on all the 10 sub-datasets within the two road scenes.
Implementation details. The segmentation model we use is a ResNet-50 based segmentation network with dilated convolutions [47]. Our baseline models are trained for 80 epochs following the training protocol from [28]. We apply AdvBN by placing it after layer conv2_3 of the baseline model and fine-tuning on given sub-dataset for 30 epochs, with adversarial training parameters 7 0.2, e = 0.5, and 3 repeats.
ltta:://synthia-dataaee net/. subject
3.0.
8

---

Dawn
Fog
Night
Spring
Winter
Figure 6: Example images from the Synthia video sequences dataset. The top row contains traffic scenes of the "Highway" subset, and the bottom row is the "New York-like City".
Table 3: Segmentation results the Synthia dataset. Evaluation metric is mean IoU (Intersection over Union). The first column denotes the dataset used for training.
York-like City
Highway Night
Dawn
Fog
Night
Spring
Winter
Dawn
Fog
Spring
Winter
baseline
52.7
49.5
65.9
48.2
18.6
21.0
16.9
21.6
15.3
Adv. Training
51.6
50.7
50.2
65.8
19.7
21.2
28.0
20.4
26.3
21.4
NY.Like CJ Spring
AdvProp
52.3
48.9
48.0
71.9
48.2
18.9
22.0
13.8
24.3
17.5
MoEx
54.7
51.8
18.7
AdvBN
57.5
55.1
55.4
66.5
52.7
23.8
26.6
25.9
29.8
23.5
buseline
32.6
29.0
25.4
24.2
24.8
64.2
55.5
53.1
59.0
49.2
Adv. Training
33.5
30.7
27.9
27.5
26.77
64.0
56.4
54.0
39.5
50.3
Highway/ Dawn
AdwProp
30.8
47.2
47.6
MoEx
32.0
27.5
27.6
29.6
26.7
64.6
57.2
57.0
61.0
51.1
AdvBN
34.0
31.6
29.6
30.7
29.1
64.5
57.2
56.4
61.2
S3.2
We also include the results of several alternative methods from Section 5.1. For the image space adversarial training, we adopt the same training settings as we do for AdvBN, except for the adversarial training parameters (perturbation size and bound). We find the optimal perturbation size for adversarial training through grid search and report the best results. The optimal hyperparameters find
1. Auxiliary BNs not used for either method inference time. For MoEx and AdvProp, we train them for 110 epochs to match the number of optimization steps with our AdvBN fine-tuned model. Note that do include other data augmentation methods from Section 5.1 because they are originally designed for the image classification problems, which include operations that be tricky to applied dense prediction problems like semantic segmentation.
We evaluate the performance of semantic segmentation using the mean IoU (Intersection over Union) metric. In table 3, over 20 source-targe target domain pairs, show that AdvBN achieves the best cross-domain generalization performance. Our method also improves in-domain performance over standard training. while AdvProp achieves the highest performance under in-domain settings. Results in this section are consistent with our observation on image classification Section 5.1.
Conclusion and Discussion
Our work studies how perturbing feature statistics simulate distribution shifts in image data. We find that fine-tuning on images with adversarially shifted feature distributions improves model's robustness towards various domain shifts without using auxiliary data. As AdvBN operates purely in feature space, it is complementary to existing input space data augmentation methods, and can further improve the generalization of tfate-of-the-ar methods. Future work will be to adapt method to tasks beyond vision. known that adversarial perturbations in input space boost performance for language [55] and graph [17] models, and these data modalities may benefit from more structured feature-space perturbations.
Limitations and Impact. While AdvBN can offer impactful improvements for domain generaliza- tion, it may in some cases trade off performance on non-shifted in-distribution testing data. Moreover, real-world datasets and distributional shifts vary dramatically, and practitioners should be cautious rather than expecting that the performance seen on benchmark datasets, such as ImageNet variants, will translate to performance boosts in their own settings.

---

Acknowledgements
Shu and Goldstein were supported by the ONR MURI program, with additional support provided by DARPA GARD. Wu was supported by NSFC under Grant No. 62102092.
References
[1] Nils Bjorek, Carla P Gomes, Bart Selman, and Kilian Q Weinberger. Understanding batch normalization. In NIPS, 2018.
[2] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey E. Hinton. A simple framework for contrastive learning of visual representations. In ICML, 2020.
[3] Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoe V Le. Autoaugment: Learning augmentation strategies from data. In CVPR, 2019.
[4] Deng, W. Dong. R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. ImageNet: A Large-Scale Hierarchical Image Database. In CVPRO9, 2009.
[5] Robert Geirhos, Carlos RM Temme, Jonas Rauber, Heiko H Schütt, Matthias Bethge, and Felix A Wichmann. Generalisation in humans and deep neural networks. In NIPS, 2018.
[6] Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A. Wichmann, and Wieland Brendel. Imagenct-trained cmns are biased towards texture; increasing shape bias improves accuracy and robustness. In ICLR, 2019.
[7] Micah Goldblum, Liam Fowl, Soheil Feizi, and Tom Goldstein. Adversarially robust distillation. In AAAI, 2020.
[8] Cheng Yue Gong, Tongzheng Ren, Mao Ye, and Qiang Liu. Maxup: A simple way to improve generalization of neural network training. CoRR, abs/2002.09024, 2020.
[9] Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adver- sarial examples. In ICLR, 2015.
[10] Ryuichiro Hataya, Jan Zdenek, Kazuki Yoshizoe, and Hideki Nakayama. Faster autoaugment: Learning augmentation strategies using backpropagation. In ECCV. 2020.
[11] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. CVPR, 2016.
[12] Dan Hendrycks and Thomas G. Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. In ICLR, 2019.
[13] Dan Hendrycks, Norman Mu, Ekin Dogus Cubuk, Barret Zoph, Justin Gilmer, and Balaji Lakshminarayanan. Augmix: A simple data processing method to improve robustness and uncertainty. In ICLR, 2020.
[14] Han-Kat Kai Hsu, Chun-Han Yao, Yi-Hsuan Tsai, Wei-Chih Hung, Hung Yu Tseng, Maneesh Kumar Singh, and Ming-Hsuan Yang. Progressive domain adaptation for object detection. In WACV, 2020.
[15] Xun Huang and Serge Belongie. Arbitrary style transfer in real-time with adaptive instance normalization. In ICCV, 2017.
[16] Sergey loffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. ICML, 2015.
[17] Kezhi Kong, Guohao Li, Mucong Ding, Zuxuan Wu, Chen Zhu, Bernard Ghanem, Gavin Taylor, and Tom Goldstein. Flag: Adversarial data augmentation for graph neural networks. arXiv preprint arXiv:2010.09891, 2020.
[18] Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial machine learning at scale. In ICLR, 2017.
10

---

[19] Haebeom Lee, Taewook Nam, Eunho Yang, and Sung Ju Hwang. Meta dropout: Learning to perturb latent features for generalization. ICLR, 2020.
[20] Boyi Li, Felix Wu, Ser-Nam Lim, Serge J. Belongie, and Kilian Q. Weinberger. On feature normalization and data augmentation. CVPR. 2021.
[21] Yanghao Li, Naiyan Wang, Jianping Shi, Jiaying Liu, and Xiaodi Hou. Revisiting batch normalization for practical domain adaptation. In ICLR Workshop, 2017.
[22] Yijun Li, Chen Fang, Jimei Yang, Zhaowen Wang, Xin Lu, and Ming-Hsuall Yang. Universal style transfer via feature transforms. In NIPS, 2017.
[23] Sungbin Lim, Ildoo Kim, Taesup Kim, Chiheon Kim, and Sungwoong Kim. Fast autoaugment. In NewrIPS, 2019.
[24] Yawei Luo, Ping Liu, Tao Guan, Junqing Yu, and Yi Yang. Adversarial style mining for one-shot unsupervised domain adaptation. In NeurIPS, 2020.
[25] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In ICLR, 2018.
[26] Seyed-Mohsen Moosavi-Dezfoolii Alhussein Fawzi, and Pascal Frossard. Deepfool: simple and accurate method to fool deep neural networks. In CVPR, 2016.
[27] Muzammal Nascer, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Fatih Porikli. Stylized adversarial defense. CoRR, abs/2007.146722 2020.
(28) Xingang Pan, Ping Luo, Jianping Shi, and Xiaoou Tang. Two at once: Enhancing learning and generalization capacities via ibn-net. In ECCV, 2018.
[29] German Ros, Laura Sellart, Joanna Materzynska, David Vazquez, and Antonio M. Lopez. The synthia dataset: large collection of synthetic images for semantic segmentation of urban scenes. In CVPR, June 2016.
[30] Evgenia Rusak, Lukas Schott, Roland S. Zimmermann, Julian Bitterwolf, Oliver Bringmann, Matthias Bethge, and Wieland Brendel. A simple way to make neural networks robust against diverse image corruptions. In ECCV, 2020.
[31] Steffen Schneider, Evgenia Rusak, Luisa Eck, Oliver Bringmann, Wieland Brendel, and Matthias Bethge. Improving robustness against common corruptions by covariate shift adaptation. In NeurIPS, 2020.
[32] Ali Shafahi, Mahyar Najibi, Mohammad Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry S Davis, Gavin Taylor, and Tom Goldstein. Adversarial training for free! In NeurlPS, 2019.
[33] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In ICLR, 2015.
[34] Aman Sinha, Hongseok Namkoong, and John C. Duchi. Certifying some distributional robust- ness with principled adversarial training. In ICLR, 2018
[35] Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei A. Efros, and Moritz Hardt. Test-time training with self-supervision for generalization under distribution shifts. In ICML, 2020.
[36] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In ICLR, 2014.
[37] Hung- Yu Tseng, Hsin-Ying Lec, Jia-Bin Huang, and Ming- Hsuan Yang. Cross-domain few-shot classification via learned feature-wise transformation. In ICLR, 2020.
[38] Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robustness may be at odds with accuracy, In ICLR, 2019.

---

[39] Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. Adversarial discriminative domain adaptation. In CVPR, 2017.
[40] Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. Instance normalization: The missing ingredient for fast stylization. CoRR, abs/1607 abs/1607.08022, 2016.
[41] Vikas Verma, Alex Lamb, Christopher Beckham, Amir Najañi, Ioannis Mitliagkas, David Lopez- Paz, and Yoshua Bengio. Manifold mixup: Better representations by interpolating hidden states. In ICML, 2019.
[42] Riccardo Volpi. Hongseok Namkoong, Ozan Sener, John C. Duchi, Vittorio Murino, and Silvio Savarese. Generalizing to unseen domains via ad adversarial data augmentation. In NeurlPS, 2018.
[43] Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno A. Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. ICLR, 2021.
[44] Haohan Wang, Songwei Ge, Zachary Lipton, and Eric Xing. Learning robust global represen- tations by penalizing local predictive power. In NeurlPS, 2019.
[45] Zhe Wu, Zuxuan Wu, Bharat Singh, and Larry S. Davis. Recognizing instagram filtered images with feature de-stylization. In AAAI, 2020.
[46] Cihang Xie, Mingxing Tan, Boqing Gong, Jiang Wang, Alan L. Yuille, and Quoc V. Le. Adversarial examples improve image recognition. In CVPR, 2020.
[47] Fisher Yu and Vladlen Koltun. Multi-scale context aggregation by dilated convolutions. In ICLR, 2016.
[48] Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regularization strategy to train strong classifiers with localizable features. In ECCV,2019
[49] Dinghuai Zhang, Tianyuan Zhang, Yiping Lu, Zhanxing Zhu, and Bin Dong. You only propagate once: Accelerating adversarial training via maximal principle. In NeurIPS, 2019.
[50] Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P. Xing, Laurent El Ghaoui, and Michael I. Jordan. Theoretically principled trade-off between robustness and accuracy, In ICML, 2019.
[51] Hongyi Zhang, Moustapha Cissé, Yann N. Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. In ICLR, 2018.
[52] Richard Zhang, Phillip Isola, and Alexei A Efros. Colorful image colorization. In ECCV, 2016.
[53] Xinyu Zhang, Qiang Wang, Jian Zhang, and Zhao Zhong. Adversarial autoaugment. In ICLR, 2020.
[54] Long Zhao, Ting Liu, Xi Peng, and Dimitris N. Metaxas. Maximum-entropy adversarial data augmentation for improved generalization and robustness. In NeurIPS, 2020.
[55] Chen Zhu, Yu Cheng, Zhe Gan, Siqi Sun, Tom Goldstein, and Jingjing Liu. Freelb: Enhanced adversarial training for natural language understanding. In ICLR, 2020.
12


Reviews:
[{"review_id": "tmldwMdGKyQ", "reviewer": "Reviewer_316u", "summary": "The paper proposes a new method for augmenting training based on worst-case perturbations of batch statistics, batch mean and variance. The idea is that these statistics encode style information thus making a model robust to small changes of their values will promote style invariance. The authors evaluate their method on variants of the ImageNet dataset and compare its performance with existing data augmentation alternatives. They find that while the method itself does not always outperform these alternatives, combining it with them can improve performance.", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper studies an important question: how can simple synthetic transformations used during training improve robustness to unforeseen transformations of the data. The proposed method is rather simple and intuitive. At the same time, the performance improvement from using this method alone is too small to justify using it over other methods. Also, the improvement from combining it with other methods is still rather small in most cases. Nevertheless, I still believe that the findings could be of interest to the robustness community.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes, they do.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2.5"}, "extra_text": {}}, {"review_id": "QpswnZOsTCi", "reviewer": "Reviewer_4gjW", "summary": "The paper studies the problem of making predictive models robust to distributional shifts. This is achieved using an adversarial training strategy that adversarially perturbs batch norm statistics of a pre-trained model, which effectively captures variations in image style such as color, texture, etc. This worst-case optimization strategy exposes the model to larger variations in image statistics and semantics rather than just just pixel-based perturbations, and as a result is expected to produce better robustness under large distribution shifts. \n", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Strengths**\n+ The paper adds to emerging evidence that BN statistics can play a critical role in designing classifiers that are robust to unknown noisy changes and domain changes. \n+ Interesting technique to visualize the adversarially perturbed features, which many adversarial methods are not able to do. This is useful in understanding the kinds of attributes being perturbed during optimization. That the most relevant are color, and texture are also intuitive, corroborating previous studies that argue CNNs are very reliant on these properties for classification. \n+ I like the formulation that it is generic (network agnostic), and simple to implement.\n\n**Weaknesses**\n+ There are a few concerns with the work with regard to novelty — the idea of using adversarial training to obtain robustness to large semantic shifts instead of pixel-wise perturbations has been explored in the past (Gokhale et al., 2020 https://arxiv.org/abs/2012.01806). More specifically, the idea of adapting BN statistics to obtain robustness to distribution shifts has also been explored (Schneider et al., 2020, https://arxiv.org/abs/2006.16971; Nado et al., https://arxiv.org/abs/2006.10963); and other kinds of layer statistics (Wang et al., https://arxiv.org/abs/2006.10726)  albeit for test time adaptation, which is a different application than those considered in this paper. However, the broad idea that robustness in this manner can be achieved has been somewhat established. This paper is using the same approach in an adversarial training setting.\n+ Next, it seems the benchmarks on which robustness is measured are better suited for AdvBN (like stylized imageNet, sketch etc.) which mostly vary along texture and color anyway. On ImageNetC it is clear AdvBN on its own is unable to match the performance of existing baseline techniques like Adv Prop, Augmix etc. Many methods using adv robustness have also considered larger domain shifts (like in a domain generalization setting) which is more challenging, and will perhaps be a more fairer evaluation for AdvBN as well.\n+ Most evaluations are performed like ablations, with robustness methods mainly based on augmentation — whereas there are several recent approaches that have shown promise even otherwise which will help provide context to AdvBN’s performance benchmarks. This is the same issue on the semantic segmentation results — no other baselines are reported other than baseline and adv training, which are weak for this task which involves large semantic shifts — advBN is better by design. \n+ Question about the approach:  Why is only a single AdvBN layer used? How does the performance vary if more than one AdvBN layers are used or do they end up being redundant? ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes, the paper addresses some of its weaknesses in the broader problem of robustness under distributional shifts. ", "review_text": "", "extra_scores": {"ethical_concerns": "None", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3.5"}, "extra_text": {}}, {"review_id": "FQsSUEeYLAx", "reviewer": "Reviewer_WKvM", "summary": "This paper propose AdvBN as a variant of batch normalization, to increase the robustness to distributional shifts of existing CNNs.\n", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The authors propose to improve DNNs' robustness to different image styles by adversarially perturbing the features statistics during training, and fine-tuning the parameters to make them robust to the perturbed features.\nThe authors implement this idea as a DNN layer named AdvBN, which could be integrated into different DNN architectures.\n\nStrengths:\n* The intuition (perturbing feature statistics during fine-tuning to increase the robustness) makes sense.The authors first visualize the effects of perturbing feature statistics by an autocoder, to show that the feature statistics can indeed encode the \"style\" of different images. Then the authors propose to adversarially modify the feature statistics which could yield generate different image styles, and then make the classifier to be robust to these styles in the training time.\n* The authors integrate the proposed AdvBN on five data augmentation methods, and all test cases on four datasets show that the AdvBN can indeed improve the robustness to distribution shifts.\n* I particularly like the experiment designed in Section 5.3, which demonstrates that the ResNet-50 model with AdvBN can learn more robust feature representations across different image styles, when compared with a standard ResNet-50 model.\n\nWeakness:\n* In Table 1, since the authors perform fine-tuning for AdvBN, the total training time seem to be longer than baseline methods . The number of training steps should be carefully controlled to make a fair comparison.\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "30"}, "extra_text": {}}, {"review_id": "BDfXUOuZhe9", "reviewer": "Reviewer_1irr", "summary": "This paper proposes to improve robustness on ImageNet classification by adversarially optimizing the batch-norm statistics of a pretrained classifier. The initial layers of the classifier are frozen, and only the upper subnetwork of the classifier is finetuned after the addition of the adversarial batch-norm layer. The benefits of the method often appear in conjunction with image augmentation methods, where applying AdvBN on top of an image augmentation method tends to result in better performance.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The methodology of this paper is fairly simple, and the results are consistent across the board on various domains (several ImageNet variants, several classification architectures, and on a segmentation task). Ablations show the impact of altering the position of the AdvBN layer and on number of PGD steps. Overall, the paper reads smoothly and is clearly written.\n\nA few points that would be helpful to clarify:\n-  in A.3 it is mention that the original BN parameters are used for some datasets, and auxiliary BN is for others. It would be helpful to provide more details on how this choice is made, and full results on using both BN option to see how sensitive the performance is for this choice. Is main or auxiliary BN used for the segmentation experiment? \n- in Table 7, why are the reconstructed classifications without adversarial perturbation much worse than clean ImageNet classifications? It would be helpful to see what the reconstructions look like. \n- in Eqn 1, the notation $\\delta f$ seems a bit unclear. Is it standard deviation of the features f?\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Limitations and impact are addressed. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2.5"}, "extra_text": {}}]

Author rebuttal:

Meta-review: After a thorough discussion (particularly with reviewer 4gjW) there are now four reviews that unanimously recommend acceptance.
Thus, I will also recommend acceptance.
Decision: Accept (Poster)