Counterbalancing Teacher: Regularizing Batch Normalized Models for Robustness
Authors: Saeid Asgari
Fereshte Khani
Ali Gholami
Kristy Choi
Linh Tran
Ran Zhang
Venue: ICLR 2022 Submitted (2022)
Keywords: Robust representation learning
domain generalization

Abstract: Batch normalization (BN) is a ubiquitous technique for training deep neural networks that accelerates their convergence to reach higher accuracy. However, we demonstrate that BN comes with a fundamental drawback: it incentivizes the model to rely on frequent low-variance features that are highly specific to the training (in-domain) data, and thus fails to generalize to out-of-domain examples. In this work, we investigate this phenomenon by first showing that removing BN layers across a wide range of architectures leads to lower out-of-domain and corruption errors at the cost of higher in-domain error. We then propose the Counterbalancing Teacher (CT) method, which leverages a frozen copy of the same model without BN as a teacher to enforce the student network's learning of robust representations by substantially adapting its weights through a consistency loss function. This regularization signal helps CT perform well in unforeseen data shifts, even without information from the target domain as in prior works. We theoretically show in an overparameterized linear regression setting why normalization leads a model's reliance on such in-domain features, and empirically demonstrate the efficacy of CT by outperforming several methods on standard robustness benchmark datasets such as CIFAR-10-C, CIFAR-100-C, and VLCS.

Under review as a conference paper at ICLR 2022
COUNTERBALANCING TEACHER: BATCH NORMALIZED MODELS FOR ROBUSTNESS
REGULARIZING
Anonymous authors Paper under double-bling review
ABSTRACT
Batch normalization (BN) is a ubiquitous technique for training deep neural net- works that accelerates their convergence to reach higher accuracy. However, we demonstrate that BN comes with fundamental drawback: it incentivizes the model rely on frequent low-variance features that are highly specific the training (in-domain) data, and thus fails to generalize to out-of-domain examples, In this work, we investigate this phenomenon by first showing that removing BN layers across a wide range of architectures leads to lower out-of- and corruption errors at the cost of higher in-domain error. We then propose the Counterbalancing Teacher (CT) method, which leverages a frozen copy of the same model without BN teacher to enforce the student network's learning of robust representations by substantially adapting its weights through consistency loss function. This regularization signal helps CT perform well in unforeseen data shifts, even without information from the target domain as in prior works. We theoretically show in an overparameterized linear regression setting why normalization leads a model's s reliance on such in-domain features, and empirically demonstrate the efficacy of CT by outperforming several methods on standard robustness benchmark datasets such as CIFAR-IO-C, CIFAR-100-C, and VLCS.
INTRODUCTION
Batch normalization (BN), neural network layer that normalizes input features by aggregating batch statistics during training, is key component for accelerating convergence in the modern deep learning toolbox (loffe and Szegedy, 2015; Santurkar et al., 2018; Bjorck et al., 2018). It plays critical role in stabilizing training dynamics for large models optimized with stochastic gradient descent, and has since spurred a flurry of research in related modifications (Ba et al., 2016; Kingma and Ba, 2014) and its understanding (Gitman and Ginsburg, 2017; Santurkar et al., 2018; Luo et al., 2018; Kohler et al., 2018).
Despite its advantages, BN has recently been shown to be a source of vulnerability to adversarial perturbations (Galloway et al., 2019; Benz al., 2021). In work, we take this observation step further and demonstrate that BN also compromises a model's out-of-domain (OOD) generalization capabilities. Specifically, we demonstrate that normalization incentivizes the model to exploit highly predictive, low-variance features (Geirhos et al., 2018; 2020), that lead to poor classification accuracy when the test environment differs from that of training. Given the widespread use and benefits of normalization, we desire way to mitigate such drawbacks in models trained with BN.
To better understand this phenomenon, we investigate the effect of normalization in over ever-parametrizer regimes, where there exist multiple solutions and inductive bias (c.g., minimizing the norm of the weights) significantly impacts the estimated parameters. Similar to recent work in the theory of deep learning (Khani and Liang, 2021; Raghunathan et al., 2020; Nakkiran, 2019; Hastie et al., 2019; Liang et al., 2020), we study the min-norm solution in over- parrpaaraetrrized linear regression. Without normalization, the inductive bias selects a model that fits training data and minimizes fixed norm independent of data; with normalization, the same inductive bias selects a model that minimizes a data-dependent norm, leading the model to rely predictive features yield better performance in in-domiin where the features do not vary significantly, they cause performance plummet in OOD settings (e.g. data corruptions missing features). This
low-variance features. While such highly

---

Under review as a conference paper at ICLR 2022
Go
c00
P50
X
Lee (ogg TTPe)
FC
TFt
CF2
Pft
The Counterbalancing Teacher (CT) model's architecture. In the first step, the teacher encoder (i.e., unnormalized) Go is trained to map input x into label. In the second step, we freeze Ge. remove its classification head CG., and train the student encoder (i.c., with batch-norm) Fe while regularizing its learned representations FFx using distance function Let and rg.. Stopping gradients operation is shown by *t. Classifier heads and class probabilities are denoted by and p, respectively. During inference, we only use Fę.
Figure 1:
direct contrast models trained without BN that assign equal weight to all input features, which help to reduce their overfitting on the training set.
Drawing inspiration from our observation and the knowledge distillation literature (Hinton al., 2015; Romero et al., 2014), we propose a simple teacher-studen model to combine the best of both worlds: we leverage features derived both from a network without BN (teacher) and its clone with BN (student) to learn representations that achieve high standard and robust accuracies. We incorporate a regularization term in the loss function which encourages the features learned from the student encoder to have similar statistics and structure to those learned from the teacher, we name this model the Counterbalancing Teacher (CT) and show that it helps in achieving both higher robust and clean accuracy compared to a (batch) normalized model. In particular, CT retains good performance in OOD settings even without knowledge of statistics of the new domain.
Our results mark significant improvement over prior works, which have tackled similar problems by cither: (a) modifying the statistics a trained model (Schneider et al., 2020; Benz et al., 2021) using privileged information from the target domain; (b) augmenting the training data using set of predefined corruption functions (Hendrycks et al., 2019b). As recent studies (Vasiljevic et al., 6; Geirhos al., 2018; Taghanaki et al., 2020) show that such approaches often fail to generalize due to the tendency of neural networks to memorize data-specific properties, this motivates a shift towards de- veloping models that are inherently robust, independent of data augmentation or input transformation.
Empirically. we demonstrate that CT outperforms most existing data augmentation- baasea techniques and covariate shift adaption- baase methods (which require information from the test set) on mean corruption eтгоr on CIFAR 10-C and CIFARIO0 100-C (Hendrycks and Dietterich, 2019), and achieves state-of-the-art performance in domain generalization on the VLCS dataset (Torralba and Efros, 2011). We further test CT on corrupted 3D point-cloud data (Taghanaki et al., 2020) and show outperforms existing methods in terms of mean classification accuracy over multiple test sets. To the best of our knowledge, this is the first work to explore both theoretically and empirically why BN leads to a model's over- wver-reliance on frequent, low-variance features, which can adversely affect its performance on the downstream classification task. This is also the first work to present a robust representation learning framework for common input distortions without additional data augmentation strategies or information derived from the target domain.
In summary, our contribution is threefold:
1. We provide theoretical justifications for why normalization encourages a model to exploit low-variance features, and empirically evaluate how this behavior can adversely affect downstream classification accuracy.
2. We propose CT, a representation learning approach demonstrating that regularizing repre- sentations of a copy (i.e., teacher) can significantly improve a model's robustness.
network (i.e., student) using those from an unnormalized
3. We experimentally verify the robustness of the representations learned by CT to input distortions and domain shift variety of tasks and models.
2

---

Under review as a conference paper at ICLR 2022
2 PROBLEM STATEMENT AND ANALYSIS
We assume a supervised classification setting: given an input variable x E X' C Rd and set of
corresponding labels y 3 {1,..., k),, aim to learn classifier fc X y by minimizing
the empirical risk:
< argmin B () yi () argmin Σ ((fe(zz),
(I)
Here pa(z, y) is the underlying joint distribution where the dataset D = { |= is sampled from. In the following sections, we first discuss the effect of (batch) normalization on the solutions found in underspecified regimes, then elaborate on our approach to optimize the aforementioned empirical risk. In this context, we refer to a problem as underspecified or overparameterized when degrees of freedom of model is larger than the number of training samples.
2.1 THE EFFECT OF NORMALIZATION IN OVERPARAMETRIZED REGIMES
Modern deep learning frameworks usually incorporate many parameters (often larger than the number of training data points), which lead underspecified regimes. In other words, many distinct solutions solve the problem equally i.c., have the same training or even held-out loss (D' Amour al., 2020). In the underspecified regime, the inductive bias of the estimation procedure, such as choosing parameters with the minimum norm, significantly impacts the estimated parameters. In such regimes, we show that normalizing data incentivizes the model to rely on features with lower variance. We analyze the effect of normalization noiseless linear regression. This setup has been studied in many recent works for understanding some phenomena in deep networks (Khani and Liang, 2021; Raghunathan et al., 2020; Nakkiran, 2019).
the min-norm solution in overparametrized
Let X E Rn denote training examples and E R" denote their target. Considering that we in an parametrized regime (d > n), there should be an equivalence class of solutions. We assume that the inductive bias of the model is to choose the min-norm solution (the parameter with the minimum & norm). This is in line with the recent speculation that the inductive bias in deep networks tends to find a solution with minimum norm (Gunasekar et al., 2018). One can show that the convergence point of gradient descent run the least-squares loss is the min-norm solution.
Without normalization, the model chooses the min-norm solution which fits the training data:
غ= arg min ||c12 s.f. X(= =
Now we observe how normalization changes the estimated parameters. Let U be diagonal matrix where U/ denotes the standard deviation of the 7th feature. By normalization, we transform to XU (for simplicity, we assume the mean of each feature is 0, we can show that transforming points do not change the estimated parameter, see Appendix C.1 for details). this case the model estimates 3 as follows:
min 3 = arg min ||312 s.t. XU-1
(3)
and since we normalize data points at the test time as well, the estimated parameter used for prediction at the test time is U- B. Substituting instead of U-1 (thus UO = =8), we can write the equal formulation of 3 as:
i arg min | s.t. X0
For the same equivalence class of solutions a model with normalization (4) chooses different param- eters in comparison model without normalization (2). In particular, 2 chooses an interpolant with a minimum data independent norm. On the other hand, 4, chooses an interpolant with a min- imum data-dependent norm, which incentives the model to assign higher weights to low variance features. Note that projection Ô and is the same in column space of training points. Formally II = X (XxTT-1 X denote the column space of training points then IIθ = II. However, their projections to the null space of training points (7 IT) different. As result have
3

---

Under review as a conference paper at ICLR 2022
more data (smaller null space), 0 and { become closer, and converge when > d. Our analysis hold for classification with max mar-in, we only need to substitute X0 = Y by O Xe 2 (see Appendix C.2 for details).
We conjecture that minimizing the data-dependent norm in each layer leads to reliance on low variance (frequent) features, which result in better in-domain generalization as these feature do not exhibit high variations. Nonetheless, in a new domain where some (or all) of the training-doman features missing altered (e.g., when there is some data-agnostic corruption such as Gaussian noise), 2 performs better as its inductive bias is data independent. How should we change the regularization such that selects for model that performs well both and out-of-domain? Inspired by this analysis, we introduce a simple, yet powerful two-step approach that combines the normalized and unnormalized copies of the same network for robust representation learning.
COUNTERBALANCING TEACHER (CT)
3.1 TRAINING THE TEACHER ENCODER (L.E. UNNORMALIZED ENCODER)
For an arbitrary neural network encoder F parametrized by { with one or more batch normalization layers, we first create a clone of the network. We refer to the cloned network as and its parameters as 0. We then remove all the batch norm layers from the cloned network and train it using the classification (k class) objective of the task in hand C In our experiments, we minimize the cross-entropy loss as supervised classification objective:
Cao(pG v)
Σ yf log(p',),
(5)
henceforward we refer to Go as the Counterbalancing Teacher or simply CT, and F as the Student.
3.2 REGULARIZING FOR ROBUSTNESS USING THE COUNTERBALANCING TEACHER (CT) METHOD
We subsequently CT with frozen parameters to regularize the training objective of the Student network. Inspired from the work of Huang and Belongie (2017), we propose the following objective to regularize the representations of Fş (i.e. Student) using:
Lat (ren, TFc)
1 Σ
xxx 2
Fs and Go map the input & to latent representations r'F, Rh and rg. A respectively. h denotes the dimension of feature vectors. Finally, we add Lee to the cross entropy objective and minimize:
C(rg., PFF:F. = -5 (PF v) + XCrr(ro.
To calculate the first and and second terms in Equation 6 (RHS), we la normalize both rg, and r Another variant of the model would be to train both Ga and Fc simultaneously. however, as F converges faster, it dominates Go. We found the current 2-steps setting working better i.e., training and freezing Go in the first step. Similar to knowledge distillation, limitation of our CT method can be training an extra encoder as a regularizer, however, note that the encoder is removed during inference, therefore the processing time inference will be similar that of the original model. The in Eq. controls the amount of regularization.
EXPERIMENTAL RESULTS
In this section, we are interested in empirically investigating the following questions:
1. Do batch and other normalization techniques lead to learning frequent low variance features?
4

---

Under review as a conference paper at ICLR 2022
accurasy
WWARN
500
3
3
100
80
20
Test seit
MNIST classification results
(b) ablation
(c) Feature importance
Figure 2: (a) In the MNIST classification task, the model without BN (NoBN) achieves high classification accuracy even if the frequent training features are missing during inference. (b) Ablation results on the "None" test set. (c) The model with BN (2nd row) puts more emphasis on the red/blue squares as they are the frequent features while the NoBN model (3rd row) does not.
2. Would a trained model with normalization fail if the dominant features are missing or corrupted, and how would an unnormalized network behave in the same scenario?
3. Does our CT method lead to reasonable in- and corruption error across different architectures and data modalities?
accuracy well low
4.1 BATCH NORMALIZATION CAUSE OF FAILURE WHEN FREQUENT INPUT FEATURES ARE MISSING
We start with a synthetic experiment to test the hypothesis that batch normalization leads to learning low variance dominant features. We design binary classification problem by selecting two digits of "2" and "3" from the MNIST dataset and add a (Figure 2c first row) "frequent" features.
red and a blue square to all samples of class "3"
We examine this hypothesis both qualitatively (visualizing the important input features for the trained classifiers using GradCAM (Selvaraju et al., 2017)) and quantitatively by evaluating multiple test sets: 1) RB: similar to training samples, digit "3" includes both the red and blue squares, 2) R: digit "3" includes only red square, 3) B: digit "3" includes only blue square, and 4) None: contains square.
Ideally, classifier should not rely only on the dominant features to predict the class. However, as shown in the second row of Figure 2c, the network trained with batch normalization quickly picks up those low variance clues, which in this case either the red or the blue square. However, the same network without normalization takes into account other features as well. We verify this by quantifying classification accuracy on the different test sets. demonstrated in Figure 2a, since the model relies mostly on the colored squares, it drastically fails when those features are missing at test time (the "None" test set) which is not the case for the network without normalization.
4.2 BATCH NORMALIZATION IS CAUSE OF FAILURE WHEN INPUT FEATURES ARE CORRUPTED
In this experiment, we focus on scenario where models are trained with clean data while the test data is shifted by common data corruptions. Here, we use the WideResNet 40-2 and AllConvNet architectures and CIFAR-10-C dataset to consistent with the work of Hendrycks et al. (2019b). For simplicity and to directly study the effect of normalization under data shifts we do not use dropout and data augmentations this experiment. We further replace batch normalization with other common normalization techniques such layer normalization (LN) (Ba et al., 2016) and instance normalization (IN) (Ulyanov et al., 2016). As shown in Figure 3, the model with no normalization (NN) outperforms all the three types of normalization when evaluated the clean data all three techniques do generally well. With our CT model, corruption errors improves significantly WideResNet and ResNext, by 4% and 3%, respectively.
corrupted data. Whereas,
5

---

Under review as a conference paper at ICLR 2022
crror
Curroption errar
Curroption error
45
44
Clasiirc ton error
35
Glssiitiation eroor
34
23
19
wBN
No3N
(ourst
wBN
No3N
CT (ours)
(a) ResNext
(b) WideResNet
Figure 3: LIFAR-10-C error. For both the models, CT achieves lower corruption errors.
4.3 REGULARIZING BATCH NORMALIZED MODEL USING CT LEADS TO HIGHER ROBUSTNESS
Here we compare our counterbalance teacher model with recent approaches on robustness, multi- domain generalization, and corrupted point cloud classification using different data sets and models.
Robustness to common data corruptions. We compare the robustness our CT model two main groups of the recent approaches on robustness: 1) covariate shift adaptation and 2) data augmentation-bie based methods on CIFAR-1O-C and CIFAR-100- (Hendrycks and Dietterich, 2019). Covariate shift adaptation approaches (Benz et al., 2021; Schneider et al., 2020) are designed with the assumption that there exist some samples from (or close to test distribution) which can be used to calibrate the parameters of the batch normalization layers. However, we show that these approaches work only if BN statistics are adapted to particular corruption. This unrealistic, especially at inference time, occur. Additionally, we show adapting batch norm statistics to a single or a few distortion types can often fail generalize to unseen distortions (Geirhos et al., 2018; Vasiljevic et al., 2016) which is also the case for data augmentation- based methods.
often do not have prior knowledge of corruption types that may may not
CT vs. covariate shift adaptation methods for robustness. We consider three scenarios for adaptation of batch norm statistics; Let {c, i Z} be the corruptions that can be applied to the input of the network (we assume that the corruption severity level (Hendrycks and Dietterich, 2019) is all scenarios):
Adapt-one-test-one: BN statistics are adapted batch of samples from each corruption type c; at a time, and classification error is computed on the same This is repeated for each corruption type and classification error is averaged over each type (Benz al., 2021; Schneider et al., 2020). However, it is not trivial to differentiate samples based on corruption type and adapt each separately at test time.
(II) Adast-one-test-all:: BN statistics are adapted to a batch of samples from randomly selected C₁ and the classification error averaged over C. This experiment reveals the caveats of adaptation on single corruption, where model fails generalize other types of corruptions.
(III) Adapt-all-test-all BN statisties are adapted to batch of samples from random combination of samples from C. The goal is to aid adaptation of statistics by looking at samples from "all" corruption types. The classification error then averaged over C. This scenario is more realistic than (I) and (II), but still requires batch of samples from the target domain.
more realistic scenario of batch norm statistics adaptation. We assume access to some extra samples, but not their corruption types (i.e., adapt-all-test--aa Surprisingly, we regardless of batch size, such methods perform significantly worse both clean and corruption errors compared to our CT method on CIFAR- 10-C with WideResNet model. Further analysis on how adaptation-basee methods cause generalization failures included in Table 6.
In Figure 4, shed light

---

Under review as a conference paper at ICLR 2022
ackpp-al-tesit test-all
adipp--cn--tss-oo
W/S adaptation
CT (ours)
80
80
25
Cormuttonn ortor
20
30
15
10
32
120
512
1024
2049
32
129
612
1064
2048
Acspiation Batch Size
Adactation Batch Size
Figure 4: Corruption (left) and clean (right) errors by increasing adaptation batch size on CIFAR-1O-C. Covariate shift adaptation- based methods fail in more realistic setting addat-all-teste iee updating the batch norm statistics using randomly selected batches of samples from the target do- main, while CT performs significantly better without requiring the target domain knowledge/statisties.
SimCLR
BYOL.
Barlow Twins
CT (ours)
AllConvNet 30.87
34.12
34.35
14.0=0.55
CIFAR-10-C
WideResNet
23.12
26.04
25.80
11.6:+0.15
AllConvNet 57.51
63.76
60.07
39.8:10.06
CIFAR-100-C
WideResNet
57.59
56.40
55.88
33.7=10.12
Table 1: CT's mean coruppiio enror (mCE) compared common self-supervised and contrastive learning methods on CIFAR-10-C and CIFAR-100-C. All SSL models are first pre-trained (self- supervised), then fine-tuned (supervised).
Comparing the robustness of CT self-supervised methods on common input corruptions. In this experiment, we evaluate how recent self-supervised and contrastive learning methods perform on unseen input corruptions: SimCLR (Chen et al., 2020), BYOL (Grill et al., 2020), and Barlow Twins (Zbontar et al., 2021). As reported in Table 1, our method outperforms all self-supervised methods (with fine-tuned encoders) based on the mean corruption error for both CIFAR-IO-C and CIFAR- 100-C by large margin. For all the methods in Table we use the same individual augmentations (same operations as in AutoAugment) as in other experiments. See appendix B.2 for linear evaluation results. Concretely, note that these self- self-supervised representation learning methods proceed in two stages: pre-training an encoder via self-supervision; and learning linear classification head on top of the pre-trained encodings. To make fair comparison, modification was to convert this two-stage process into single joint-training procedure, by using an objective that combines these two terms: (1) a contrastive loss term that uses label information select positive and negative examples for training the encoder (similar to Supervised Contrastive Learning by (Khosla al., 2020)); and (2) a cross-entropy loss term for the downstream classification task. In this way, both the encoder and classifier benefit from labeled supervision.
Comparing CT to data augmentation-bass based methods. Hendrycks al. (2019b) has studied the effectiveness of different data augmentation techniques on making a model robust to common data corruptions. They proposed to create weighted mixture of input samples using multiple different augmentation types (AugMix) and showed this method can outperform other augmentation- based methods on robustness common corruptions. Since our CT model is inherently data nugmentation-inddeeeeenttt without any augmentation. it outperforms (by achieving 25. 25.7% mCE while the original mCE is 39.6%) several recent and widely used augmentation- -bsed approaches such as Standard (Hendrycks et al., 2019b), Cutout (DeVries and Taylor, 2017), CutMix (Yun et al., 2019), and Adversarial Training (Madry et al., 2017) on CIFAR-IO-C with WideResNet. When CT used with a few simple data augmentations (AutoAugment (Cubuk et al., 2018) individual operations, but randomly), as shown in Table 2, it outperforms most of the data augmentation based methods including the recent and advanced ones such Mixup (Zhang al., 2017) and AutoAugment (Cubuk et al., 2018), while it achieves comparable results to AugMix without the Aassen-Shannon divergence (ISD) loss on CIFAR-10- 10-C and CIFAR-100-C datasets with different networks.
7

---

Under review as a conference paper at ICLR 2022
Standard Cutout Mixup CutMix AutoAug AdvT AugMix CT (ours)
CIFAR-IO-C AllConvNet 30.8
32.9
24.6
31.3
29.2
28.1
15.0
14.0
WideResNet 26.9
26.8
22.3
27.1
23.9
26.2
11.2
11.6::0.15
CIFAR-100-C AllConvNet 56.4
56.8
53.4
56.0
55.1
56.0
42.7
39.8+0.06
WideResNet
53.3
53.5
50.4
52.9
49.6
55.1
35.9
33.7::0.12
Table 2: CIFAR-10-C and CIFAR-100-C mean corruption error (mCE) compared common data augmentation techniques. AdvT, and AutoAug refer to adversarial training, Auto Augment, respectively. Our CT approach outperforms six of seven methods while it achieves comparable results to AugMix which leverages complex augmentations.
CT's performance in domain generalization. In this experiment, we evaluate CT on domain generalization using the VLCS benchmark (Torralba and Efros, 2011). VLCS consists of images from five object categories shared by the PASCAL VOC 2007, LabelMe, Caltech, and Sun datasets, which are considered to four separate domains. We follow the standard evaluation strategy used in (Carlucci et al., 2019), where we partition each domain into a train (70%) and test set (30%) by random selection from the overall dataset. We use ResNet- 18 as the backbone to make a fair comparison with the state-of-the-art. As summarized in Table 3, CT outperforms the state-of-the-arr
on 3 out of 4 domains and by 1.83% on average.
Method
Caltech
LabelMe
Pascal
Sun
Average
DeepC (Li et al., 2018b)
87.47
62.06
64.93
61.51
68.89
CIDDG (Li et al., 2018b)
88.83
63.06
64.38
62.10
69.59
CCSA (Motian al., 2017)
92.30
62.10
67.10
59.10
70.15
SLRC (Ding and Fu, 2017)
92.76
62.34
65.25
63.54
70.15
TF (Li al., 2017)
93.63
63.49
69.99
61.32
72.11
MMD-AAE (Li et al., 2018a)
94.40
62.60
67.70
64.40
72.28
D-SAM (D' Innocente and Caputo. 2018)
91.75
57.95
58.59
60.84
67.03
Shape Bias (Asadi et al., 2019)
98.11
63.61
74.33
67.11
75.79
CT (ours)
99.21
65.87
74.10
71.20
77.60
Table 3: Multi-source domain generalization accuracy (%) the VLCS dataset with ResNet-18 as the base network for classification. All reported numbers are averaged over three runs.
CT's classification performance on corrupted 3D point cloud data. In this experiment, we use the RobustPointSet dataset Taghanaki al. (2020) which created for analysis of point classifiers in terms of robustness to 3D corruptions. We follow the same vraining-domain validation setting as in Taghanaki al. (2020). We train each model without data augmentation on the clean training set and select the best performing checkpoint test the models on the six distorted unseen test sets. This experiment shows the vulnerability of the models trained on original data to unseen input transformations. As shown in Table 4, PointNet-CT significantly improves classifcation accuracy on most of the shifted test sets, and 1.87% on average compared to original PointNet.
the clean validation set for each method. We then
Method
Original Noise Translation MissingPart Sparse Rotation Occlusion Avg.
PointNet
89.06
74.72
79.66
81.52
60.53
8.83
39.47
61.97
Poim
87.99
78.65
79.02
76.82
72.61
6.89
35.13
62.44
PointNet-CT (ours)
89.83
78.12
82.25
82.58
68.40
7.62
38.10
63.84
Table Robust classification accuracy on RobustPointSet. The Noise column for example shows the result of training on the Ori g nal train set and testing with the Noise test set. When we train PointNet using our CT method its performance significantly improves on average and particularly on Noise, Trans laz ion, MissingPan arr, and Sparse test sets.
8

---

Under review as a conference paper at ICLR 2022
5
RELATED WORK
Reducing classification error on common input corruptions. Hendrycks et al. (2019b) proposed data augmentation technique which mixes multiple augmented samples using random weights and showed improvements on robust accuracy. Rusak et al. (2020) proposed to mix Guassian noise with adversarial samples for robustness to common distortions. However, as in the real world, the space of distortions and their mixture is not finite, it is not trivial to devise set of augmentation types that will make a model robust all distortions. Moreover, making model robust to certain corruptions via augmentation does not generalize to others. Dodge and Karam (2017) leverage an ensemble approach for robustness against distortions, but they also assume corruptions are known beforehand.
Batch normalization's vulnerability. Several works have studied the effect of batch normalization in the context of robustness to adversarial perturbations. However, as the most relevant works to ours, Schneider et al. (2020) and Benz et al. (2021) suggested improving robustness to common data corruptions by updating the batch normalization parameters using statistics calculated form extra test distribution samples. However, extra test samples are not always available, and similar to data augmentation-bant -based methods, these approaches work only when a model is adapted to a specific distortion and tested on the same distortion type.
Teacher-student models in the context of robustness. The focus of the few existing methods on leveraging teacher-student models for robustness has been on adversarial perturbations for developing adversarially robust models either by training teacher encoders with adversarial (Goldblum et al., 2020; Papermot al., 2016) or augmented (Arani et al., 2021) examples. In contrast, CT method is citpanatttion-dddeeeeentttt Adversarially trained models might not necessarily be robust to common data corruptions, as showed in Table 2, our CT method performs significantly better (at least by 12%) on both CIFAR-1O-C and CIFAR-100-C datasets across different models compared to the adversarially trained model (AdvT). Even if the augmented samples are crafted by common data augmentations rather than adding adversarial noise, the robustness obtained by the augmentations does not generalize to unseen corruptions, even when they are from the same family of the augmentations a model has seen (Vasiljevic et al., 2016) during training.
Similarities to self-supervised methods. Our CT method is similar to ceff-supervised models (SSL) (Chen et al., 2020; Grill et al., 2020; He et al., 2020) in the sense that SSL also leverages copy of the same model but for encoding a different view of the input. SSL., the copy of the model is the same as the main model, while we discard batch normalization layers in our CT model from one of the copies. Hendrycks et al. (2019a) showed that SSL can improve error on distorted data, however their method-and and in general existing self-supervised methods-arr are highly dependent on auteeetgmenttio (inductive bias) for creating different views while our approach does not require extra views of the data. Note that our method does not employ any self-supervised loss function.
6 CONCLUSION
In this work, we investigated the effect of BN on model robustness. We provided a theoretical justification in the overparameterized linear regime for how BN compromises OOD generalization performance by encouraging the model to rely on highly predictive, low-variance input features. Then, we proposed a robust representation learning framework Counterbalancing Teacher (CT) - which leverages frozen copy of the same model without BN teacher to regularize the features learned by a student network to improve generalization. Empirically, we demonstrated that our method's learned representations are robust to common corruptions on suite of domain generalization tasks.
REFERENCES
Martin Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. Tensorflow; system for large-scale machine learning. In 12th {USENIX) symposium on operating systems design and implementation ((OSDI) 16), pages 265 283, 2016.
Elahe Arani, Fahad Sarfraz, and Bahram Zonooz. Noise resource for learning in knowledge distillation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 31 3129-3138, 2021.

---

Under review as a conference paper at ICLR 2022
Nader Asadi, Mehrdad Hosseinzadeh, and Mahdi Eftekhari. Towards shape biased unsupervised representation learning for domain generalization. arXiv preprint arXiv:1909.08245. 2019.
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. arXiv preprint arXiv:1607.064550 2016.
Philipp Benz, Chaoning Zhang, Adil Karjauv, and In So Kweon. Revisiting batch normalization for improving corruption robustness. In Proceedings of the IEEE/CVE Winter Conference on Applications of Computer Vision, pages 494-503, 2021.
Johan Bjorck, Carla Gomes, Bart Selman, and Kilian Q Weinberger. Understanding batch normaliza- tion. arXiv preprint arXi::/806.002:: 02375, 2018.
Fabio M Carlucci, Antonio D'Innocente, Silvia Bucci, Barbara Caputo, and Tatiana Tommasi. Domain generalization by solving jigsaw puzzles. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2229-2238, 2019.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference machine learning, pages 1597-1607. PMLR, 2020.
Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoe V Le. Autoaugment: Learning augmentation policies from data. arXiv preprint arXiv:/805.090 2018
Alexander D' Amour, Katherine Heller, Dan Moldovan, Ben Adlam, Babak Alipanahi, Alex Beutel, Christina Chen, Jonathan Deaton, Jacob Eisenstein, Matthew Hoffman, et al. Undenspecification presents challenges for credibility in modern machine learning. arXiv preprint arXiv:2011.03335. 2020.
Terrance DeVries and Graham W Taylor. Improved regularization of convolutional neural networks with cutout. arXiv preprint arxiv: 2017.
Zhengming Ding and Yun Fu. Deep domain generalization with structured low-rank constraint. IEEE Transactions on Image Processing, 27(1):304-311 313, 2017.
Samuel Dodge and Lina Karam. Quality resilient deep neural networks. arXiv:1703.081199 2017.
arXiv preprint
Antonio D'Innocente and Barbara Caputo. Domain generalization with domain-specific aggregation modules. In German Conference on Pattern Recognition, pages 187-198. Springer, 2018.
Angus Galloway, Anna Golubeva, Thomas Tanay, Medhat Moussa, and Graham W Taylor. Batch normalization is a cause of adversarial vulnerability. arXiv preprint arXiv:1905.00
2019.
Robert Geirhos, Carlos R Medina Temme, Jonas Rauber, Heiko Schütt, Matthias Bethge, and Felix A Wichmann. Generalisation in humans and deep neural networks. arXiv preprint arXiv: 1807000000.08750 2018.
Robert Geirhos, HonnHeeriik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix Wichmann. Shortcut learning deep neural networks. Nature Machine Intelligence, 2(1 :166556673, 2020.
Igor Gitman and Boris Ginsburg. Comparison of batch normalization and weight normalization algorithms for the large-scale image classification. arXiv preprini urXiv: 1e00:089999919 45, 2017.
Micah Goldblum, Liam Fowl, Soheil Feizi, and Tom Goldstein. Adversarially robust distillation. In Proceedings of the AAAI Conference Artificial Intelligence, volume 34, pages 3996-400000 2020.
Man-BBstien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre H Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Daniel Guo, Mohammad Gheshlaghi Azar, et al. Bootstrap your own latent: new approach to self-supervised learning. arXiv preprint arXiv:2006. 07733, 2020.
10

---

Under review as a conference paper at ICLR 2022
Suriya Gunasekar, Blake Woodworth, Srinadh Bhojanapalli, Behnam Neyshabur, and Nathan Srebro. Implicit regularization in matrix factorization. 2018 Information Theory and Applications Workshop (ITA), pages 1-10. IEEE, 2018.
Trevor Hastie, Andrea Montanari, Saharon Rosset, and Ryan Tibshirani. Surprises high- dimensional ridgeless least squares interpolation. arXiv preprint
2019.
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9729-97736 2020.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. arXiv preprint arXiv:/903. /2261, 2019.
Dan Hendrycks, Mantas Mazeika, Saurav Kadavath, and Dawn Song. Using self-supervised learning can improve model robustness and uncertainty. arXiv preprint arXiv: 12340, 2019a.
Dan Hendrycks, Norman Mu, Ekin D Cubuk, Barret Zoph, Justin Gilmer, and Balaji Lakshmi- narayanan. Augmix: A simple data processing method to improve robustness and uncertainty. arXiv preprint arw.v: 2019b.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge neural network. arXiv preprint arXiv:1503.00 02531, 2015.
Xun Huang and Serge Belongie. Arbitrary style transfer in real-time with adaptive instance nor- malization. In Proceedings of the IEEE International Conference on Computer Vision, pages 1501-1510, 2017.
Sergey loffe and Christian Szegedy. Batch normalization: Accelerating deep network training by redueing internal covariate shift. In International conference on machine learning, pages 448 456.. PMLR, 2015.
Fereshte Khani and Percy Liang. Removing spurious features can hurt accuracy and affect groups disproportionately. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pages 996-205,27 2021.
Prannay Khosla, Piotr Teterwak, Chen Wang. Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning. arXiv preprint wrXiv:2004.11362 2020.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
Jonas Kohler, Hadi Daneshmand, Aurelien Lucchi. Ming Zhou, Klaus Neymeyr, and Thomas Hofmann. Towards a theoretical understanding of batch normalization. stat, 1050:27, 2018.
Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M Hospedales. Deeper, broader and artier domain generalization. In Proceedings of the IEEE international conference computer vision, pages 5542-5550, 2017.
Haoliang Li, Sinno Jialin Pan, Shiqi Wang, and Alex C Kot. Domain generalization with adver- sarial feature learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5400-5409, 2018a.
Ya Li, Xinmei Tian, Mingming Gong, Yajing Liu, Tongliang Liu, Kun Zhang, and Dacheng Tao. Deep domain generalization via conditional invariant adversarial networks. In Proceedings of the European Conference on Computer Vision (ECCV), pages 624-639, 2018b.
Tengyuan Liang, Alexander Rakhlin, et al. Just interpolate: Kernel "ridgeless" regression can generalize. Annals of Statistics, 4893):1329-1347, 2020.
Ping Luo, Xinjiang Wang, Wenqi Shao, and Zhanglin Peng. Understanding regularization in batch normalization. arXiv preprint arXiv: 1809. 000400 1(6), 2018.

---

Under review as a conference paper at ICLR 2022
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv: 1eee:106600000 06083, 2017.
Saeid Motian, Marco Piccirillii, Donald A Adjeroh, and Gianfranco Doretto. Unified deep supervised domain adaptation and generalization. In Proceedings of the IEEE International Conference Computer Vision, pages 5715-5725, 2017.
Preetum Nakkiran. More data hurt for linear regression: Sample-wise double descent. arXiv preprint 1rXiv:19/2.07242, 2019.
Adhyyan Narang. Overparameterizze classification problems: How many support vectors do I have, and do increasing margins always bode well for generalization? Phl thesis, Department of Electrical Engineering and Computer Sciences, University of .... 2020.
Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. Distillation as a defense to adversarial perturbations against deep neural networks. In 2016 IEEE symposium security and privacy (SP), pages 582-597. IEEE, 2016.
Aditi Raghunathan. Sang Michael Xie, Fanny Yang. John Duchi, and Perey Liang. Understanding and mitigating the tradeoff' between robustness and accuracy. arXiv preprint arXiv:2002.10716 2020.
Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio. Fitnets: Hints for thin deep nets. arXiv preprint 141.:::412.65500 2014.
Evgenia Rusak, Lukas Schott, Roland S Zimmermann, Julian Bitterwolf, Oliver Bringmann, Matthias Bethge, and Wieland Brendel. A simple way to make neural networks robust against diverse image corruptions. In European Conference on Computer Vision, pages 33.69. Springer, 2020.
Shibani Santurkar, Dimitris Tsipras, Andrew Ilyas, and Aleksander Madry. How does batch normal- ization help optimization? arXiv preprint /805. /604, 2018.
Steffen Schneider, Evgenia Rusak, Luisa Eck, Oliver Bringmann, Wieland Brendel, and Matthias Bethge. Improving robustness against common corruptions by covariate shift adaptation. Advances in Neural Information Processing Systems, 33, 2020.
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-basee localiza- tion. In Proceedings of the IEEE international conference computer vision, pages 61 318-626, 2017.
Daniel Soudry, Elad Hoffer, Mor Shpigel Nacson, Suriya Gunasekar, and Nathan Srebro. The implicit bias of gradient descent on separable data. The Journal of Machine Learning Research, 19(1): 2822-2878, 2018
Saeid Asgari Taghanaki, Jieliang Luo, Ran Zhang, Ye Wang, Pradeep Kumar Jayaraman, and Krishna Murthy Jatavallabhula. Robustpointset: A dataset for benchmarking robustness of point cloud classifiers. arXiv preprint arXiv:2011.11572, 2020.
Antonio Torralba and Alexei A Efros. Unbiased look dataset bias. In CVPR 2011, pages 1521-1528. IEEE, 2011.
Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. Instance normalization: The missing ingredient for fast stylization. arXiv preprint arXiv:1607.08023 2016.
Igor Vasiljevic, Ayan Chakrabarti, and Gregory Shakhnarovich. Examining the impact of blur recognition by convolutional networks. arXiv preprint 11X1v:1611.05760, 2016.
Sangdoo Yun, Dongyoon Han, Scong Joon Oh. Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regularization strategy to train strong classifiers with localizable features. In Proceedings of the IEEF/CVF International Conference Computer Vision, pages 6023-6033 2019.
12

---

Under review as a conference paper at ICLR 2022
Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny, Barlow twins: Self-supervised learning via redundancy reduction. arXiv preprint arXiv:2103.03258 2021.
Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. arXiv preprint arXiv:1710.094412, 2017.
APPENDIX
A ADDITIONAL EXPERIMENTAL DETAILS
We implement our CT variants using Tensorflow Abadi et al. (2016).
A.1 MNIST MISSING FEATURES EXPERIMENT
We trained the MNIST models (architecture below) with SGD optimizer (with Tensorflow's 's default parameters) and batch size of 128 for 100 epochs. For the NoBN model, we simply removed the BN layers and trained all models without any data augmentation.
Table 5: MNIST small CNN architecture with batch normalization layers. Batch normalization layers (in gray) omitted in the Teacher network.
Layer
Conv2D (in=d, out=16, stride=1)
BatchNorm
ReLU
Conv2D (in=16, out=16, stride=1)
BatchNorm
6
ReLJU
MaxPool2D (stride=2)
Conv2D (in=16, out=32, stride=1)
9
BatchNorm
10
ReLU
11 Conv2D (in=32, out=32, stride=1)
BatchNorm
13
ReL.U
MaxPool2D (stride=2)
15
Dense (nodes=256)
BatchNorm
17
ReLU
Dense (nodes=c)
19
Softmax
A.2 COMPARING CT TO OTHER METHODS ON COMMON INPUT CORRUPTION
For both CIFAR-10-C and CIFAR-100-C train Go using Adam optimizer with learning rate of 0,0001 and batch size of 64. In the second step, we train Fe using SGD with Nesterov, initial learning rate 0.1 and decaying 0.00001 using cosine scheduler for 300 epochs.
A.3 APPLYING CT TO POINT CLOUDS
We trained both Go and Fe with Adam optimizer and learning rate of 0.001 (divided by 10 epochs 50 and 75) and batch size of 32 for 500 epochs.
A.4 APPLYING CT TO DOMAIN GENERALIZATION
We trained CT with SGD optimizer with learning rate of 0.001 and momentum of 0.9. We set batch size 32 and image size to 224x224, and train for 500 epochs.
13

---

Under review as a conference paper at ICLR 2022
A.5 SELF-SUPERVISED BASELINES
We train the self-supervised methods with batch size 256 for 1000 epochs while maintaining the orig- inal hyper-parameters. We fine-tune all self-supervised methods for 350 epochs with Adam Kingma and Ba (2014) and with starting learning rate of 0.001 with cosine decay, and lz coefficient set to 0.0005 to avoid overfitting
ADDITIONAL EXPERIMENTAL RESULTS
B. BATCH-NORM ADAPTATION ADDITIONAL RESULTS
Batch norm adaptation-bast based results for robustness to common input corruptions on CIFAR-IO-C.
Table Changes in clean error (CLN-E), mean corruption error (mCE), and corruption error with respect to changes in adaptation batch size (BS). In this table, the model's batch normalization layers are only adapted one corruption (CRP) based on the method presented in Benz al. (2021). This table shows an example of adapt-one- teap--n scenario. All numbers are reported with the WideResNNe44-22 model. Adaptation batch norm statistics Impulse noise, when observed test-time data is adequate, will improve the robustness to similar corruption types (Gaussian, Shot and Impulse noise). However it significantly reduces generalization to other corruption types. Top row shows original model performance without adaptation. Second row shows adapt results.
CRP
BS
CLN-E
mCE
Gauss.
Shot
Impulse
Defocus
Glass
Motion
Snow
Compress
None
14.2
37.54
56.26
47.93
49.28
29.54
56.44
38.51
32.35
32.71
a-0-t-o
32
23.83
26.07
37.72
35.03
34.82
16.84
40.03
24.86
27.86
31.1
Impulse
32
29.04
49.88
45.92
41
39.52
56.97
60.46
65.79
40.31
45.51
Impulse
16
33.06
55.02
51.39
45.73
43.62
62.27
66.23
71.03
45.62
51.12
Impulse
2
43.7
61.47
62.31
58.16
58.33
62.68
73.32
68.68
56.95
59.56
Defocus
32
20.39
41.98
70.47
62.49
59.07
21.54
65.94
35.58
41.16
38.9
Defocus
16
21.11
44.43
72.24
64.88
58.77
24.53
69.55
38.36
41.64
Defocus
2
44.47
62.1
74.39
70.69
68.91
51.03
74.98
54.16
64.18
68.92
B.2 LINEAR EVALUATION SELF-SUPERVISHE MODELS
Self-supervised linear evaluation results where encoders are frozen and only classification layer is trained.
Table 7: CT's mean corruption error (mCH) compared to common self celf-supervieed and contrastive learning methods on CIFAR- 10-C and CIFAR-100-C. Encoders are frozen and only classification layer is trained (linear evaluation).
SimCLR
BYOL
Barlow Twins
CT (ours)
AllConvNet
58.84
65.47
58.2
16.9
CIFAR-10-C
WideResNet
58.18
53.32
59.75
13.9
AllConvNet 82.91
91.2
86.16
42.6
CIFAR-100-C
WideResNet
84.08
83.38
85,61
43.1
B.3 PREDICTIVE PERFORMANCE OF MNIST DATASET VARIABLES.
The following analysis is performed to determine whether the dark and light pixels in the MNIST experiment are discriminative. We plot histograms of all pixel values for each class separately. The histograms look very similar, as shown in Figure 5, indicating that the dark light pixels are not predictive alone. Therefore we add red or blue squares/variabee which are both low variance and predictive.
14

---

Under review as a conference paper at ICLR 2022
Histogram of
245
1ed
dight
32-
30
235
1.
13
12
031
03
10
120
720
00
50
112
233
250
Figure 5: Histogram plots of class '2' vs. '3'
ADDITIONAL ANALYSIS ON THE EFFECT OF NORMALIZATION IN OVERPARAMETERRIZE REGIME
C.I SUBTRACTING MEAN DOES NOT CHANGE THE ESTIMATED PARAMETERS
In Section 2.1, we show that dividing each feature by its variance incentivize the model to rely more on features with low variance. Here show that subtracting mean from each feature and target does not change the estimated parameter in overparametrized regime. Recall that X E Raxd denote the training examples and Y R" denote their target. Let μ.x Rd be the mean of features in training data, and Joy E R denote the mean of the their targets.
Since μx a linear combination of the rows of and row operation does not change the projection matrix; therefore, the following two linear programs have the same solution:
&= arg min |||||
= arg min ||0||
s.t. X
(X 1x )0 =Y Ipy
C.2 THE EFFECT OF NORMALIZATION ON MAX-MARGIN CLASSIFIERS
In Section 2.1, analyze the effect of normalization in linear regression. Here show that the analysis hold for the max-margin classifiers as well. Soudry et al., Soudry et al. (2018), show that without any explicit regularization, gradient descent on logistic loss converges to the L2 maximum margin separator for all linearly separable datasets. In the overparametrized regime data are completely separable, there are also some investigation that as d increases all the data points serve support vectors Narang (2020).
Without normalization we have:
? = arg min ||</|2
s.t. YOXÇ21
Recall that U is a diagonal matrix where Un the standard deviation of the 7th feature. After normalization we have:
= arg min |1
s.t. XOX
(9)
At the test time we predict sign ii substituting U-1 with we have:
i = arg min |000| s.l. YO X021
(10)
Similar to linear regression scenario normalizing data change the inductive bias of estimator to choose a model that rely more low variance features. This data-dependent norm lead good performance in-domain where such features exhibit low variation but will perform poorly out of domain data are corrupted independently or are sampled from a different distribution with different statistic than the training distribution.
15


Reviews:
[{"review_id": "x9gl4pnJ3D", "reviewer": "Reviewer_63kd", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is mostly well written with a clear story, and nicely motivated by a few toy examples. The results outlined in the paper look very promising.\n\nHowever, especially after the strong focus on batch norm adaptation techniques in the introduction, I expected to see full results and improvements on ImageNet-C, -R, (-A) as common in the referenced papers. In contrast, the experimental results are quite sparse, and comparision to the most relevant competitors (techniques that leverage augmentations during training time) are limited to Table 2 (CIFAR scale). In-depth analysis of the properties of the method, along with ablation studies, is also missing. \n\nIn the following I will discuss the most important additional limitations/weaknesses of the paper.\n\n* Methods: The method section in 3.2 lacks clarity. Especially for batch norm, there are a lot of caveats in the exact implementation. Could you please add a precise description how the statistics are estimated, how the student / teacher update works, how student -> teacher update works, etc.? Given the simplicity of the method (which is a huge plus), it should be possible to re-implement the technique based on the information in §3, and I feel like this is not the case currently.\n* Motivating examples (§4.1): Especially on such small scale experiments, I find it important to strengthen results by running multiple seeds, and reporting error bars. Could these be added to all experiments in Figure 3, both for the barchart and the line plot? In addition, it could be interesting to add results for an “on the fly” normalization layer such as GroupNorm or LayerNorm to the set of models in Figure 3 (optional). Apart from this, I really like this experiment and how it is used to set the stage for the method.\n* Figure 4: The lines should not be connected, there is no obvious relationship between the entries on the x-axis. I suggest a simple scatterplot (one point per method) along with error bars is more suitable in this case.\n* Experimental results from §4.3 onwards: It is unclear to me which experiments were re-implemented, and which results are cited from previous work. Could you please let me know for tables 1, 2, 3 and 4 whether you ran all baseline implementations yourself, or whether any of the numbers in the tables is cited? From the appendix, it seems the contrastive learning numbers are reproduced, while the domain generalization results are cited? If that is the case, did you verify any of the baseline results to make sure the experiment setups match?\n* Self-supervised learning (Table 1): How do you justify this experiment? If all the labels are known, and the models are fine-tuned, why is it interesting to use self-supervised learning for obtaining the embeddings in the first place? Generally, I think this is a minor result, and should be less prominently mentioned in the paper.\n* Instead, It would strengthen the paper to extend Table 2, and add more methods. According to Robust Bench, the [state-of-the art in CIFAR10-C](https://robustbench.github.io/#div_cifar10_corruptions_heading) right now are much better than the reported 17.4%. On a WideResNet-40-2 model, Augmix implemented in the benchmark achives ~11-12%, while 13.5% are reported in the paper. How do you explain the difference? The same applies to CIFAR-C, where Augmix is reported with 35.89%. It would also be good to cite [Diffenderfer](https://arxiv.org/abs/2106.09129) who are state of the art now, although their method is clearly concurrent work (just accepted to NeurIPS'21).\n* VLCS: [Gulrajani and Lopez-Paz](https://arxiv.org/abs/2007.01434) showed that on domain generalization benchmarks like VLCS, after sufficient hyperparameter tuning, and by using more suitable and larger models (like a ResNet50), ERM is a very strong baseline. Results in Table 3 show exactly the issue described by Gulrajani and Lopez-Paz. A much stronger result would be a full evaluation on the DomainBed benchmark, if the authors would like to do a strong claim about Domain Generalization. (Also note that all of the cited methods were published before Gulrajani and Lopez-Paz).\n* Minor Point: Positioning within the related work: It is known (and already shown on ImageNet-C scale, cf. your references regarding BatchNorm adaptation) that adapting to “shuffled” corruptions (your “adapt-all-test-all” setup) does not work with batch norm adaptation, so this is not a new observation. This should be made clear in §4.2.\n\nA few questions:\n* Table 2 suggests that larger models (?) like the WideResNet narrow or even revert the effect of your technique. When moving to full scale data, models get even larger (ResNet50, ResNext101, etc.) and even better augmentation techniques than Augmix (SOTA right now is DeepAugment + Augmix augmentation combined) are used. Could you comment on the limitations of your method and your expectations? For instance, batch norm adaptation fails to work as models and datasets get larger--do you expect that a similar limitation will apply to your technique, that is not visible from the presented results?", "summary_of_the_paper": "The authors propose a new consistency loss for improving model robustness to common image corruptions. They use a student-teacher training setup where only the student network uses batch normalization at training time. Improvements are shown on small scale corruption datasets (CIFAR-C), a single domain generalization dataset (VLCS), and RobustPointSet. ", "summary_of_the_review": "The technique is simple, well-described, and seems to be effective on the considered toy datasets. What I am missing right now is compelling evidence that it actually works in practice, and in practically relevant settings. For this, several key results are missing:\n\n1. Full scale ImageNet-C (and -R) evaluation with relevant larger models (ResNet50, ResNext101 etc, following prior work). This will make the paper much more interesting for a wider audience interested in corruption robustness. In addition, I suggest runnign on ObjectNet --- BatchNorm adaptation does not work on this dataset, and it would be interesting to see whether CT is effective at mitigating this issue.\n2. Comparision to more recent domain generalization techniques, and especially reporting results on more datasets beyond VLCS. To make a claim that the method improves domain generalization, the authors should consider the DomainBed evaluation setting, or a similar, more comprehensive experiment protocol that avoids the issues discussed by Gulrajani & Lopez-Paz (2020). Alternatively, the authors might drop this claim, and move the table as a proof-of-concept to the supplement.\n3. I am missing a statement how hyperparameters were tuned. The supplement simply states a few values, but not how the authors arrived at them. In particular, they should be picked according to the *validation error on the training domain*. Ablations over a few crucial parameters will help here (for example, the learning rate). It is unclear why these were not added, given that the results are obtained on fairly small scale datasets.\n\nI think it is feasible and reasonable to expect these kinds of results in an ICLR paper. Point (1) only requires to train models on ImageNet (3 seeds, ideally) *once*, getting scores for the requested setups is easily doable afterwards.  I find it possible that on full ImageNet-C, more recent techniques like DeepAugment+Augmix on larger models (ResNet50) will outperform the proposed technique. Even then, I agree with the authors comment (cf. Table 2) that a huge advantage of the method is the removed need for well-designed data augmentations. However, I feel that this might come at the expense of only being effective in settings where batch norm adaptation is working. Since the technique is effective on corruptions, a great test would be the ObjectNet dataset, where batch norm adaptation fails to work.\n\nRegarding (2), this is not a strong requirement, the authors could equally decide to drop the claim regarding domain generalization.\n\nPoint (3) is again crucial---especially Table 2 indicates that the baselines in the paper were not sufficiently tuned.\n\nIf the authors manage to add these results (minimally 1, comment on 2, and clarifications on 3, along with addressing the detailed comment in my review), and show a realistic comparisions of tradeoffs to state of the art augmentation based models, I will increase my score. Right now, in my opinion the paper does not meet the bar for ICLR, as the number of truly relevant datasets, and comparisions to the state of the art is limited.\n\nI am happy to address questions/clarify the exact scope of points 1-3 above if this is needed.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "d997G4VBx5L", "reviewer": "Reviewer_HouJ", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Strengths:**\n\n- I love the clarity of exposition, especially section 2.1 where the insight of data-dependent versus independent norms is presented.  Also the experiments in section 4, which begin with a very clear description of the hypotheses to be tested.\n\n- The authors perform an extensive set of experiments for different models, datasets (and types).  Particularly impressive was the comparison of CT to other data augmentation-based methods\n\n**Weaknesses:**\n\n- Section 4.1 is a bit oddly presented.  In particular, the red vs blue experiment.  The third paragraph states:\n  > \"Ideally, a classifier should not rely only on the dominant features to\npredict the class. However, as shown in the second row of Figure 2, the network trained with batch normalization quickly picks up those low variance clues, which in this case is either the red or the blue\nsquare. However, the same network without normalization takes into account other features as well.\".\n\nI find it hard to draw a conclusion from this experiment.  Have the authors really controlled all the sources of variation in this experiment?  Are the synthetic features shown really the low(est) variance features that disambiguate 2 from 3 in MNIST?  I imagine there are many light / dark pixels that are also low-variance & predictive.  A histogram of variances would be helpful here in establishing the validity of the design.\n\n- Figure 4 is oddly presented.  The X-axis is categorical (different normalization for layers of the wide ResNet), and are not ordinal; presenting this as a line plot suggests that they are.  A grouped bar-plot or box-whisker plot (or violin plot) would be more direct.  Another issue is that the results of Figure 4 are discussed in section 4.2  with a condition not depicted:\n> As shown in Figure 4, the model with no normalization (NN) outperforms all the three types of normalization when evaluated on corrupted data\nNo data for (NN) is presented in Figure 4.  Was the name changed?\n\n- The central insight of the paper is that normalization of features leads to an over-reliance on low-variance features.  Yet after presenting the $L_{ct}$ regularizer, the authors state that they $l_2$ normalize both $r_{G_{\\theta}}$ and $r_{F_{\\zeta}}$.  It seems very strange to me that a corrective measure against normalization must still require normalization.  Is there no other way to account for the different scales of learned features?  *Should* the method account for them?  I'd like to see a version of the CT loss where different losses for equation (6) are used.\n\n- Table 2 was my favourite part of the whole paper, but I was sad to read that the authors only reported point estimates of mCE.\n\n- Given how the student-teacher regularized loss is defined in equation [7], it seems odd that the authors do not present an ablation study where they reduce the regularization effect tuning $\\lambda * \\mathcal{L}_{ct}$ as $\\lambda \\rightarrow 0$\n", "summary_of_the_paper": "The authors study the effects of BatchNorm layers on model robustness.  They demonstrate that models trained with BatchNorm are likely to rely on low-variance features which are performant in-domain, but are not helpful for out of domain data.  Leveraging this observation, they propose a student-teacher method (Counterbalancing Teacher) that creates a copy of the same model with all BatchNorm layers removed.  The demonstrate that this leads to very competitive performance with other techniques for reducing the performance hit of out domain evaluation: self-supervised methods and data augmentation methods.\n", "summary_of_the_review": "The authors present evidence that BatchNorm leads to a preference for parameters that minimize a data dependent norm, and demonstrate a simple student-teacher training method to correct for these effects.  There are a few alterations I'd really like to see, though I still believe that, given the required corrections, there is enough value in this particular insight and the experiments to see it published.  I would have dearly loved to see more emphasis on other measures of uncertainty for the networks (cf. https://github.com/uncertainty-toolbox/uncertainty-toolbox) rather than looking strictly at accuracy.  If my other criticisms can be addressed, I will increase my score.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "Qbn_NekPyfH", "reviewer": "Reviewer_SoaW", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n1. The paper is well-written and well-motivated. The proposed approach is simple, effective and does not require access to the out-of-domain distribution.\n2. The proposed approach is empirically evaluated against a number of baselines, including covariate shift adaptation methods, self-supervised methods, and data augmentation-based methods.\n\nWeakness:\n1. The theoretical analysis focuses on the limitation of batchnorm, but it is not clear how distillation resolves this problem. If a new model is distilled, one can argue that the effects of batchnorm, along with its limitations, are transferred to the new model, although it does not include the BN layer explicitly—analysis and insights are missing in the proposed counterbalancing teacher in how it address BN's limitation exactly.\n2. The authors should clarify why relying on low-frequency features are undesirable. This is an implicit theory in this paper without sufficient justification. ", "summary_of_the_paper": "The paper aims to address the problem of out-of-domain generalization with batch normalization layers. It first identifies the reliance on low-variance features in batch normalization, and proposes a counterbalancing teacher approach to distill from a BN-model to a non-BN model. Experiments verify the hypothesis of the paper and show that the proposed approach outperforms the various baselines on robustness to data corruption and domain generalization. The proposed approach is also evaluated on a 3D point cloud dataset.", "summary_of_the_review": "I think the simplicity and effectiveness of the proposed approach out-weights the limitations. This approach has potential for further improvements because it does not rely on access to the target data distribution.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "FxZ_qDK1Mus", "reviewer": "Reviewer_yprC", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper theoretically analyzes Batch Normalization on linear models, showing that it introduces and inductive bias that makes the model rely more on low-variance features. They argue this high reliance on low-variance features can cause the model to become brittle to distributon shifts where such features are disrupted, while non-normalized models tend to rely to all the input features in a more balanced way, which makes them more robust at the cost of a lower in-distribution accuracy.\n\nThe paper proposes a training method where a teacher model is first trained without BN, then a student model with the same architecture but with BN layers is trained both on the task loss and on a regularization loss that pushes its last hidden representation towards the one of the frozen teacher model. They show that this approach preserves in-distribution accuracy of BN while achieving better out-of-distribution accuracy on various tasks.\n\nQuestion for the authors: are the teacher and the student initialized with the same weights?\n\nOverall the paper is clearly written, the arguments are sound and the empirical evidence is solid.", "summary_of_the_paper": "The paper argues that Batch Normalization and similar normalization layers can make neural networks more fragile to data distribution shifts and proposes a distillation-based regularization method to ameliorate this issue while maintaining in-domain accuracy.", "summary_of_the_review": "\nOverall the paper is clearly written, the arguments are sound and the empirical evidence is solid.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors have proposed a new consistency loss for improving model robustness to common corruptions. With a student-teacher training setup, only the student network uses batch normalization at training time. Improvements are shown on small scale corruption datasets (CIFAR-C), a single domain generalization dataset (VLCS), and RobustPointSet.

Though, positive feedback were given on the quality of the story telling, and on an interesting motivation by a few toy examples, some concerns remained among the reviewers.
In particular applicability of the method as model and data sizes increases, e.g., on ImageNet-C, was questioned.
After Additional results were provided by the authors, the method seems to break as scales increases.
The way relevant baselines from previous work was also judged light and should be improved.
Hence, the paper could be improved to include more comparisons and more convincingly showing advantages of the method.
Decision: Reject