Bayesian Adaptation for Covariate Shift
Authors: Aurick Zhou
Sergey Levine
Venue: NeurIPS 2021 Poster (2021)
Keywords: uncertainty estimation
calibration
test-time adaptation
distribution shift

Abstract: When faced with distribution shift at test time, deep neural networks often make inaccurate predictions with unreliable uncertainty estimates.
While improving the robustness of neural networks is one promising approach to mitigate this issue, an appealing alternate to robustifying networks against all possible test-time shifts is to instead directly adapt them to unlabeled inputs from the particular distribution shift we encounter at test time.
However, this poses a challenging question: in the standard Bayesian model for supervised learning, unlabeled inputs are conditionally independent of model parameters when the labels are unobserved, so what can unlabeled data tell us about the model parameters at test-time? 
In this paper, we derive a Bayesian model that provides for a well-defined relationship between unlabeled inputs under distributional shift and model parameters, and show how approximate inference in this model can be instantiated with a simple regularized entropy minimization procedure at test-time. 
We evaluate our method on a variety of distribution shifts for image classification, including image corruptions, natural distribution shifts, and domain adaptation settings, and show that our method improves both accuracy and uncertainty estimation.

Training on Test Data with Bayesian Adaptation for Covariate Shift
Aurick Zhou, Sergey Levine
Department of Electrical Engineering and Computer Sciences University of California, Berkeley {aurick, svlevine aberkeley.cco.o.. edu
Abstract
When faced with distribution shift at test time, deep neural networks often make inaccurate predictions with unreliable uncertainty estimates. While improving the robustness of neural networks is one promising approach to mitigate this issue, an appealing alternate robustifying networks against all possible test-time shifts is to instead directly adapt them to unlabeled inputs from the particular distribution shift encounter at test time. However, this poses challenging question: in the standard Bayesian model for supervised learning, unlabeled inputs are conditionally independent of model parameters when the labels are unobserved, what unlabeled data tell us about the model parameters at test-time? In this paper, we derive a Bayesian model that provides for a well-defined relationship between unlabeled inputs under distributional shift and model parameters, and show how approximate inference in this model be instantiated with simple regularized entropy minimization procedure at test-time. We evaluate our method on a variety of distribution shifts for image classification, including image corruptions, natural distribution shifts, and domain adaptation settings, and show that our method improves both accuracy and uncertainty estimation.
Introduction
Modern deep learning methods can provide high accuracy in settings where the model is evaluated on data from the same distribution as the training set, but accuracy often degrades severely when there is mismatch between the training and test distributions [Hendrycks and Dietterich 2019 Taori jt al. 2020). In safety-critical settings, effectively deploying machine learning models requires not only high accuracy, but also requires the model reliably quantify uncertainty in its predictions in order to assess risk and potentially abstain from making dangerous, unreliable predictions. Reliably estimating uncertainty is especially important in settings with distribution shift where inaccurate predictions are more prevalent, but the reliability of uncertainty estimates often also degrades along with the accuracy as the shifts become more severe (Ovadia et al. 2019]. In real-world applications, distribution mismatch at test time is often inevitable. thus necessitating methods that can robustly handle distribution shifts, both in terms of retaining high accuracy but also providing meaningful uncertainty estimates.
As it can be diffieult to train single model be robust all potential distribution shifts might encounter at test time, we can instead robustify models by allowing for adaptation at test time, where finetune the network unlabeled inputs from the shifted target distribution, thus allowing the model to specialize in the particular shift it encounters. Since test-time distribution shifts often cannot be anticipated during training time, restrict the adaptation procedure operate without any further access to the original training data. Prior work on test-time adaptation (Wang et al.) 2020al Sun al. 2019b] focused on improving accuracy, and found that simple objectives like entropy minimization capable of providing substantial improvements under distribution shift (Wang et al.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

2020a
However, these prior works do not consider uncertainty estimation, and an objective like entropy minimization can quickly make predictions overly confident and less calibrated, leaving us without reliable uncertainty estimates and thus unable to quantify risks when making predictions. Our goal in this paper to design a test-time adaptation procedure that can not only improve predictive accuracy under distribution shift, but also provide reliable uncertainty estimates.
While number of prior papers have proposed various heuristic methods for test-time adapta- tion [Wang el al., 2020a Sun el al. 2019a it remains unclear what precisely unlabeled test data under covariate shift can actually tell us about the optimal classifier. this work, we take Bayesian approach to this question, and explicitly formulate Bayesian model that describes how unlabeled test data from a different domain can be related to the classifier parameters. Such model requires introducing an additional explicit assumption, as the classifier parameters are conditionally indepen- dent of unlabeled data in the standard model for discriminative classification [Seeger] 2000]. The additional assumption we introduce intuitively states that the data generation process at test-time, though distinct from the one training time (hence, under covariate shift) still more likely to produce inputs that have a single unambiguous labeling, even if that labeling is not known. We argue that this assumption is reasonable in practice, and leads to an appealing graphical model where approximate inference corresponds to Bayesian extension of entropy minimization.
We propose a practical test-time adaptation strategy, Bayesian Adapatation for Covariate Shift (BACS), which approximates Bayesian inference in this model and outperforms prior adaptive methods both terms of increasing accuracy and improving calibration under distribution shift. Our adaptation strategy is simple to implement, requires minimal changes to standard training procedures, and outperforms prior test-time adaptation techniques on variety of benchmarks for robustness to distribution shifts.
2 Bayesian Adaptation for Covariate Shift
In this section, we will devise a probabilistic graphical model that describes how unlabeled data in a new test domain can inform posterior about the model, and then describe practical deep learning algorithm that can instantiate this model in a system that enables test-time adaptation. We will begin by reviewing standard probabilistic models for supervised and discuss why such models are unable to utilize unlabeled data. We then discuss a probabilistic model proposed by |Seeger [2000] that does incorporate unlabeled data in semisupervised learning (SSL) setting, and propose extension to the model to account for distribution shift. Finally, we discuss the challenges in performing exact inference in our proposed model and describe the approximations introduce in order to derive tractable inference procedure suitable for test-time adaptation.
2.1 Probabilistic Model for Covariate Shift
In the standard Bayesian model for supervised learning (Figure T. we assume that the inputs X are sampled i.i.d. from a generative model with parameters 6, while the corresponding labels sampled from conditional distribution P(Y|X, 0) parameterized by 0. The parameters 9 and & are themselves random variables, sampled independently from prior distributions p(ő) and p(0). We observe only a dataset 'D (X,, Y 1= and then perform inference over the parameters 0 using Bayes rule:
0
x
V
N
D) & (() II pY.]X..)).
(I)
Figure 1: Proba- bilistic model for standard supervised learning, observing
To make predictions on new input Xn+11 we marginalize over the posterior distribution of the classifier parameters 0 to obtain the predictive distribution
N labeled data points.
(Yn+1 X D) = D(Yn+) ()p(0)) do.
Note that observing the (unlabeled) test input x (or more generally, any number of test inputs) does not affect the posterior distribution of parameters 0, and so within this probabilistic model, there no benefit observing multiple unlabeled datapoints before making predictions.

---

Inference for semi-supervised learning. By treating the parameters 0, Φ as a-priori independent, the standard model assumes there is no relationship between the input distri- bution and the process that generates labels, leading to the inability to utilize unlabeled data in inferring 0. To utilize unlabeled data, we can introduce assumptions about the relationship between the labeling process and input distribution using a model of the form in Figure Seeger 2000].
x
y
x
Figure 2: Model for SSL [Seeger 2000], ohserving N labeled pairs M unlabeled inputs, both generated from the same distri- bution. Observing unlabeled inputs x pro- vides information about the input distribution 9. and thus about
The assumption we use has simple intuitive interpreta- tion: we assume that the inputs that will be shown to our classifier have an unambiguous and clear labeling. This assumption is reasonable in many cases: if the classifier discriminates between automobiles and bicycles, that contain either an automobile or bicycle. Of course, this assumption is not necessarily true in all settings, but this intuition often agrees with how discriminatively trained models are actually used. This simple intuition can be formalized in terms of a prior belief about the conditional entropy of labels conditioned on the inputs.
reasonable that it will be presented with images
Similar to Grandvalet and Bengio 2004 we can encode this belief using the additional factor
p00 ((0) exp( - Ho (Y(x))
= (() exp (x -P (x,) [ p(Y (XX 0))
(3)
where m(0) is prior over the parameters 0 that agnostic of the input distribution parameter e.
Now, observing unlabeled test inputs 4 = ( ) provides information about the parameter ø governing the input distribution, which then allows to update our belief over the Iearned parameters 0 through Equation 2aaooonannnnn and thus allows inference to utilize unlabeled data within a Bayesian framework.
Extension to covariate shift. The previous probabilistic model for SSL assumes all inputs were drawn from the same distribution (given by Φ). However, our goal is use unlabeled data to adapt our extend the model incorporate covariate shift.
model different test distribution,
We now assume we have two tions; @ specifies the input distribution for our labeled training set, and è specifies the shifted distribution of unlabeled test inputs which we aim to adapt to. Under the assumption of covariate shift, the same classifier 0 is used to generate labels both the train and test domains, leading to the model in Figure 3
distribu-
X
We argue that our prior belief of low aleatoric uncertainty is reasonable for both the distribution induced by & and the one induced by Φi even if there is some distribution shift from the training set, we can still often expect the test data we are shown to have unambiguous labels. We can then incorporate both & and é into our belief over 0 with the factor
N
M
Figure 3: Our proposed probabilistic model for adaptation for covariate shift, observing a training with N labeled pairs and M labeled inputs from shifted test distribution.
p(0 Ø) x p(0)) exp(- (X))) exp( exp(- (Y
(4)
with the two scalar hy typerparameters a, â controlling how to weight the entropies from each distri- bution with the likelihoods of the labeled training set. While we can extend this model to include more labeled unlabeled input distributions, we focus on simply having one training distribution and one test distribution as it the most relevant for our problem setting of adapting to particular distribution shift at test time.
2.2 Approximations for Tractable Inference
Performing inference over 0 in the above model can be challenging in our test-time adaptation setting for several reasons. First of all, inference over 0 would also require performing inference over the
3

---

parameters of the generative models 6 and o to evaluate likelihoods in Equation 4 This is difficult in practice for two reasons: First, the inputs z might be high-dimensional and difficutt model, as in the case of images. Second, the amount of unlabeled data might be fairly small, and insufficient to accurately estimate the parameters Ø if we employ a highly expressive generative model, As such, we would much prefer to avoid explict generative modeling and to only perform inference over the discriminative model parameters 0 instead.
Another issue is that performing inference would require access to both the labeled training data and the unlabeled test data at the same time, while our test- rest-time adaptation setting assumes that we can no longer look at the training set when it is time to adapt to the test distribution. We will discuss how to address these issues and describe our method, Bayesian Adaptation for Covariate Shift (BACS), which provides a practical instantiation of inference in this Bayesian model computationally tractable test-time adaptation setting.
Plug-in approximation with empirical Bayes. We first propose to avoid explicit generative model- ing by using with a plug-in empirical Bayes approach. Empirical Bayes is a common approximation for hierarchical Bayesian models that simply uses point estimate of & rather than marginalizing over ø's (similarly for Φ). which would reduce the computation estimating only single generative model for each input distribution given by parameters o² and Φ= To eliminate the need train the parameters 6² of a generative model of the inputs altogether, we note that p(f]o, Φ) only depends on Φ and Φ through the input distributions p(xr()) p( Φ). We can then approximate Equation by plugging in the empirical distributions of x and ă in place of p(x]o"), p( Ø-), resulting in m
p(olà. Φ) p0))
exp exp
Σ H(Y)x, .) exp
H(Y])
172
Now, given labeled training set 'D and unlabeled test points U = distribution over parameters now has log probabilities (up to an additive normalizing constant)
{ the new posterior
logp(0/D,4) = log (0) + Σ log plu x. 0) Σ HY 0)
()) (5)
3=1
For convenience, we simply set a = 0, as additionally minimizing entropy on the labeled training set is unnecessary when we already maximizing the likelihood of the given labels with highly expressive models. Now, to infer 0 given the observed datasets D and U, the simplified log-density is
log Dogp(0)), log (0)
log p(us Xx, 0)
H(Y
(6)
Computing the cesimmmma-ppostriioii (MAP) solution of this model corresponds to simply optimizing model parameters 0 that on the supervised objective on the labeled training set in addition minimum entropy regularizer on the unlabeled input. However, we should not necessarily expect the MAP solution provide reasonable uncertainty estimates, as the learned model is being encouraged to make confident predictions on the test inputs, and so any single model will likely provide overconfident predictions. Marginalizing the predictions over the posterior distribution over parameters is thus essential to recovering meaningful uncertainty estimates, as the different models, though each being individually confident, can still express uncertainty through their combined predictions when the models predict different labels.
Approximate inference for test-time adaptation. We now diseuss how perform inference in the above model in a way suitable for test-time adaptation, where we adapt to the test data without any further access to the original training set. To enable this, we propose to learn approximate posterior
( ~ p D) = log (0)
log p(ya 0)
during training time, and then use this approximate training set posterior (0) in place of the training set when performing inference on the unlabeled test data. This gives us an approximate posterior with log density
á m
log (())D. = log q())
HY m j=1
(8)

---

Algorithm 1 Bayesian Adaptation for Covariate Shift (BACS)
Input:
Ensemble size k. Entropy weight ã Training Data: Xim, Mien. Test Data: X1am Output: Predictive distributions p(yl);) for each test input X,
Training: For each i E (1,..., k) compute an approximate density (()) = p( Xi:m B1:)) Adaptation:
for all ensemble members i E (L.....k) do Compute adapted parameters 0: arg maxg Σド= 11=1-H(Y)) 12 0) + log q: (8) end for For each test input Xy, marginalize over ensemble p(y)X;)) = 1 p(yls,, 0,)
H
Unlike the full training set, the learned approximate posterior density (()), which can be as simple as a diagonal Gaussian distribution, can be much easier to store and optimize over for test-time adaptation, and the training time procedure would be identical to any approximate Bayesian method that learns posterior density. In principle, we can now instantiate test-time adaptation by running any approximate Bayesian inference algorithm, such as variational inference or MCMC, to sample O's from the density in Equation [ and average predictions from these samples to compute the marginal probabilities for each desired test label.
Practical instantiation with ensembles. As previously mentioned, marginalizing over different models that provide diverse labelings of the test set is crucial to providing uncertainty estimates after adaptation via entropy minimization. We thus propose to use an ensembling approach [Laksh minarayanan et al. 2016] as a practical method to adapt to the test distribution while maintaining diverse labelings. Deep ensembles simply train multiple models from different random initializations. each independently optimizing the target likelihoods, and averages together the models' predicted probabilities at test time. They are able to provide effective approximations to Bayesian marginaliza- tion due to their ability to aggregate models across highly distinct modes of the loss landscape Foort et al. 2019 Wilson and Izmailov 2020).
Our method, BACS, summarized in Algorithm| I trains an ensemble of k different models on the training set, each with their approximate posterior (0) that captures the local loss landscape around each mode in the ensemble. Then at test time, we independently optimize each of the k models by minimizing Equation 8 (using the corresponding (8) for each ensemble member), and then average the predictions across all adapted ensemble members.
3 Related Work
Entropy minimization.
Entropy minimization has been used as a self-supervised objective in many
settings, including domain adaptation [Saito et al., 2019 Carlucci et al.] [017]] semisupervised learning GGrandvalet and Bengio, 2004 Berthelot et al. 2019 Lee and Lec| 2013). and few-shot
learning [Dillon et al., (2015). Grandvalet and Bengio [2004] proposed a probabilistic model incorporating entropy minimization for semisupervised learning (without distribution shift), but only use the probabilistic model to motivate entropy minimization as a regularizer for a MAP solution in order improve accuracy, which does not capture any epistemic uncertainty. In contrast, we are concerned with test-time adaptation under distribution shift, which requires introducing model of separate training-time and test-time input distributions, and with providing reliable epistemic uncertainty estimates, which we obtain via Bayesian marginalization. We also devise an approximate inference scheme to allow for efficient adaptation without access to the training data.
Test time entropy minimization (TENT) [Wang et al. 2020al uses entropy minimization as the sole objective when adapting to the test data (though without an explicit Bayesian interpretation) and adapts without any further access the training data, but only aims to improve accuracy and not uncertainty estimation. Similarly to Grandvalet and Bengio [2004), TENT only learns a single model using entropy minimization, whereas we show that explicitly performing Bayesian inference and marginalizing over multiple models is crucial for effective uncertainty estimation. TENT also heuristically proposes to only adapt specific parameters in the networks at test time for stability reasons, while our usage of a learned posterior density to account for the training set allows us to adapt the whole network, improving performance in some settings and eliminating the need for the heuristic design decision.
5

---

Domain Adaptation. Unsupervised domain adaptation [Ganin et al. 2015 Wang and Deng 2018 tackles a similar problem of learning classifiers in the presence of distribution shift between our train and test distributions. Most unsupervised domain adaptation works assume access to both the labeled training data as well as unlabeled test data at the same time, while we restrict adaptation to occur without any further access to the training data. One recent line of work, known as frurce-free domain adaptation [Liang et al. 2020 Kundu et al. 2020 Li et al. 2020j also restricts the adaptation procedure to not have access to the training data together with the unlabeled data from the test distribution. In contrast to these algorithms, we are concerned with using adaptation to improve both uncertainty estimation well accuracy, and our algorithm is additionally amenable to an online setting, where prediction and adaptation occur simultaneously without needing to see the entirety of the test inputs.
Uncertainty estimation under distribution shift (no adaptation). Various methods have been proposed for uncertainty estimation with distribution shift that do not incorporate test time adaptation. Ovadia et al.] 2019] measured the calibration of various models under various distribution shift without any adaptation, finding that deep ensembles [Lakshminarayanan et al.] 2016) and some other Bayesian methods that marginalize over multiple models perform well compared to methods that try to recalibrate predictions using only the source data. Beyond ensembles, other Bayesian techniques DDueeneeryy et al. 2020 Maddox et al.] 2019] have also demonstrated improved uncertainty esti- mation under distri bution shift compared to standard models. Various other techniques have been found to improving calibration under distribution shift through significant changes to the training procedure, for example utilizing different loss functions (Padhy al. 2020 Tomani and Buettner 2020). extensive data-augmentations (Hendrycks et al. 2019). or extra pre-training (Xie et al. (019b)]
Uncertainty estimation with both train and test distributions. We now discuss prior work that considers uncertainty estimation assuming access to both train and test data simultancously. Recent work studying calibration for domain adaptation algorithms (Pampari and Ermon 2020 Park et al. 2020 Wang et al., 2020b found that predictions are poorly calibrated in the target domain even the models were well-calibrated on the labeled source domain. These works all propose methods based on importance weighting between the target domain and the labeled source domain in order to recalibrate target domain predictions using only labels in the source domain. They are not directly applicable in our test-time adaptation setting, since they require estimates of density ratios between target and source distributions, which we cannot obtain without either a generative model of training inputs, or access to the training data during adaptation. Our method also differs in how we approach uncertainty estimation. Instead of using extra labeled data and post-hoc recalibration techniques for classifiers, our method uses Bayesian inference to provide meaningful uncertainty estimates.
For uncertainty estimation in regression problems, Chan et al. [2020] also propose to adapt Bayesian posteriors to unlabeled data by optimizing the predictive variance of Bayesian neural network at an input to serve binary classifier of whether the point is in-distribution not. thus encouraging higher variance for out-of- -ii--iidiitiittoinn points. Their method is again not applicable in our test-time setting because they require access to both the train and test data at once.
Uncertainty estimation with test time adaptation. Nado et al. 2020 evaluate various techniques for uncertainty estimation in conjunction with adapting batch-norm statistics to the shifted test domain, and again find that deep ensembles provide well-calibrated prediction in addition to improved accuracy. While our method similarly utilizes ensembles and adapts batch norm statistics, and we show that additionally adapting via entropy minimization at test time further improves predictive accuracy without sacrificing calibration.
Bayesian semi-supervised methods: Seeger 2000 proposed a probablistic model for incorporating unlabeled data in semi-supervise learning to motivate regularization for the classifier that depends on the input distribution MAP inference. However, they do not tackle uncertainty estimation and their model does not account for any distribution shift like ours does. Gordon and Horaáodez-Lohaoo [2020) perform Bayesian semi-supervised learning combining both generative and discriminative models. In contrast, our method does not need to learn generative model of the data, and explicitly tackles the problem of distribution shift instead of assuming the labeled and unlabeled data come from the same distribution. Another line of work [Ng et al.) 2018 Ma et al. 2019 Walker and Glocker [2019 Liu et al. 2020) propose Bayesian methods for semisupervised learning specialized graph-structured data.

---

CIFARIO-C
CIFARI00-C
Method
Acc
NLL
Brier
ECE
Acc
NLL
Brier
ECE
Vanilla
59.90
1.892
0.6216
0.2489
35.72
4.271
0.9797
0,3883
BN Adapt
82.36
0.8636
0.2909
0.1204
57.58
2.294
0.6401
0.2377
TENT (1 epoch)
84.29
0.7862
0.2629
0.1119
62.46
2.047
0.5828
0.2280
TENT (5 epoch)
85.16
0.8483
0.2603
0.1191
63.46
2.199
0.5987
0.2592
BACS (MAP) (1 epoch)
84.82
0.7808
0.2585
0.1119
63.05
2.090
0.5908
0.2411
BACS (MAP) (5 epochs)
85.20
0.8075
0.2575
0.1144
63,53
2.175
0,6009
0.2551
Vanilla Ensemble
61.72
1,535
0.5431
0,1684
38,66
3.343
0.8439
0.2462
Ensemble BN Adapt
85.99
0.4722
0.2043
0.03229
64,22
1.464
0.4793
0.0515
Ensemble TENT (1 epoch)
87.28
0.4351
0.1867
0.02868
67.83
1.318
0.4392
0.05909
BACS (ours) (1 epoch)
87.77
0.4260
0.1809
0,02986
68.33
1.324
0,4360
0.06519
Table 1: CIFAR-10/100 Corrupted results at the highest level of Corruption, averaged over all corruption types. With one epoch of adaptation, BACS consistently outperforms all baselines in terms of accuracy, NLL and Brier score, and can further improve with more training. In terms of ECE, all ensembled methods with adaptation performs similarly, and substantially outperforming the non-adaptive or non-ensembled baselines.
Experiments
In our experiments, we aim to analyze how our test-time adaptation procedure in BACS performs when adapting to various types of distribution shift, in comparison to prior methods, in terms of both the accuracy of the adapted model, and its ability to estimate uncertainty and avoid over-confident but incorrect predictions. We evaluate method and prior techniques across a range of distribution shifts, including corrupted datasets, natural distribution shifts, and domain adaptation settings.
Architectures and implementation. For our ImageNet [Deng et al. 2009 experiments, we use the ResNer50v2 [He al., 2016b) architecture, while for other datasets, we use ResNet26 (He et al.. 2016a]. For all methods utilizing ensembles, we use ensembles of 10 models, and report results averaged over the same seeds for the non-ensembled methods. While adapting our networks using the entropy loss, we also allow the batch normalization statisties to adapt to the target distribution. To obtain approximate posteriors for each ensemble member, we use SWAG-D [Maddox et al. [2019], which estimates a Gaussian posterior with diagonal covariance from a trajectory of SGD and requires minimal changes to standard training procedures. During adaptation, we initialize each model from the corresponding posterior mean, corresponding the solution obtained by Stochastic Weight Averaging [[zmailov et al. 2018). For methods that optimize on the test distribution, we report results after epoch of adaptation unless otherwise stated.
Comparisons. We compare our method against two state-of-the-arr prior methods for test-time adaptation: TENT [Wang et al. [2020a], which simply minimizes entropy on the test data with single model, without the additional posterior term accounting for the training set that we use in BACS, and ensembles adapted using the batch norm statistics of the shifted test set, as discussed by Nado et al. 2020]. We also compare to deep ensembles [Lakshminarayanan et al. 2016)] without any adaptation as a bascline for uncertainty estimation under distribution shift, as well as ensembles of models each adapted using TENT.
Metrics. In addition to accuracy, we also evaluate uncertainty estimation using the negative log likelihood (NLL), Brier score [Brier] [950] and expected calibration error (ECE) (Naiini al. 2015] metrics. NLL and Brier score are both proper scoring rules [Gneiting and Raftery] 2007]. and are minimized if and only the predicted distribution is identical to the true distribution. ECE measures calibration by binning predictions according to the predicted confidence and averaging the absolute differences between the average confidence and empirical accuracy within each bin.
Corrupted images. We first evaluate our method on CIFAR- CIFAR-100-C, and ImageNet-C Hendrycks and Dietterich] 2019]. where distribution-shiitee datasets are generated by applying different image corruptions at different intensities to the test sets of CIFAR10, CIFARI00 (Krizhevsky 2012), and ImageNet (Deng al., 2009) respectively. In Table we show comparisons the most severe level of corruption for CIFARIO-C and CIFAR100- With just a single epoch of adaptation at test time, both our method BACS and TENT ensembles substantially outperform other baselines in accuracy, NLL, and Brier score, with BACS improving slightly over TENT ensembles, showing that combining test-time entropy minimization with Bayesian marginalization can lead to strong improvements over either alone. We also note that simply adapting for more epochs with entropy
7

---

0.2
Verila Eroumble
0.1
Counrs)
0.0
0.7
25
0.2
00
ddicoa
dosbt
0907
gets
gersian
Susodee
ges
zaition
plutsta
dof
2009
Corraption
Figure 4: IetgeNet-C Results by Corruption. For each corruption type, we show the results for each method averaged over all levels of corruption. BACS improves over baselines for every corruption type except for contrast, where BACS has close to 0 accuracy at the most severe level of corruption.
minimization (as seen in TENT (5 epochs)), can further improve accuracy of a single model, but can actually lead to worse uncertainty estimates measured by NLL or ECE as predictions become excessively overconfident. In contrast, combining ensembling with entropy minimization consistently improves accuracy much more than simply adapting single model for more epochs, while also substantially improving uncertainty estimates.
In Table we show comparisons on ImageNet-C averaged over all cor- ruption levels and corruption types. BACS is able to outperform all prior methods in terms of accuracy, NLL, and Brier score. We note that en- sembling and batch norm adaptation actually worsen calibration, as mea- sured by ECE, compared to the sin- gle model or non-adapted baselines respectively, despite each technique providing substantial improvements on all other metrics. We see that an ablation of our method that only uses single model, BACS (MAP), outperforms the other non-ensemble methods in accuracy, NLL, and Brier score.
Method
Acc
NLL
Brier
ECE
Vanilla
41.79
3.127
0.7152
0.06439
BN Adapt
51.54
2.676
0.6564
0.1709
TENT
57.06
2.035
0.5536
0,0342
BACS (MAP)
58.06
2.036
0.5528
0.04270
Vanilla ensemble
46.03
2.788
0.6670
0.07256
Ensemble BN Adapt
58.30
2.428
0.6333
0.2594
Ensemble TENT
62.40
1.788
0.5131
0.1010
BACS (ours)
63.04
1.726
0.4962
0.08308
Table 2: ImageNet-C results averaged over all corruption types and levels. BACS outperforms all baselines in accuracy, NI.L, and Brier scores.
We also show results for accuracy and Brier score at each level of corruption in Figure BACS (ours) consistently outperforms baselines at each level of corruption (with one exception being accuracy at the highest level of corruption, where a single corruption with much lower accuracy drags down the mean be slightly below to that of Ensemble TENT).
ImageNet Net-R. R. In Table 3 we further evaluate robustness using ImageNet-R [Hendrycks et al. 2021 which consists images that are abstract renditions of 200 of the ImageNet classes. Similar to our Imagenet-C results, we find that BACS performs the best across accuracy, NLL, and Brier score, while being slightly outperformed in ECE by vanilla ensembles.
8

---

Impact of posterior term. In this section, we discuss the effects of us- ing the training set posterior density in
Method
Acc
NLL
Brier
ECE
Vanilla
36.40
3.288
0.7602
0.05667
BN Adapt
38.05
3.236
0.7646
0.09744
our objective (Equationf) X when adapt- ing at test time. Intuitively, the train- ing posterior density ensures that our adapted classifiers, while minimizing entropy on the target distribution, re- main constrained to still perform well on the training set and stay near the initial solution found during training. We empirically find that the posterior term can be important for preventing entropy minimization from finding degenerate solutions: adapting the whole network several ImageNet-C corruptions without the posterior term can lead to poor models that achieve close to O accuracy on many corruptions.
TENT
41.13
2.967
0.7167
0.02666
BACS (MAP)
43.55
2.909
0.7183
0.1074
Vanilla ensemble
40,38
3.011
0.7180
0.02339
Ensemble BN Adapt
42.82
3.019
0,7408
0,1696
Ensemble TENT
45.75
2.726
0.6815
0.
BACS (ours)
47.31
2.565
0.6625
0.04270
Table 3:
ImageNet-R results. BACS outperforms base- lines in accuracy, NLI., and Brier scores.
While TENT, which adapts via entropy minimization without any term accounting for the training data, uses an ad-hoc solution that restricts adaptation to only the learnable scale and shift parameters in the batch norm layers, our use of the training set posterior density is motivated directly from our proposed probabilistic model and does not require any heuristic choices over which parameters should or should not be adapted. In our ImageNet experiments (Tables 2 and 3). ablation of our method without ensembles, corresponding to the MAP solution in our proposed model, outperforms TENT. The difference between our ablation BACS (MAP) and TENT is precisely that BACS (MAP) adapts the whole network, but with an additional regularization term, while TENT adapts only small part of the network without any regularizer.
that an
However, we find the approximate posterior density can also be limiting in certain situationns which we can observe in experiments transferring from SVHN to MNIST (Appendix B.4). Here, there a large discrepancy between the training and test domains, and we find that adaptation with the posterior is unable to adjust the parameters enough and underperforms compared to an ablation of our method that simply removes the posterior term while still adapting the whole network.
5 Discussion
We presented Bayesian Adaptation for Covariate Shift (BACS), Bayesian approach for utilizing test-time adaptation to obtain both improved accuracy and well-calibrated uncertainty estimates when faced with distribution shift. We have shown that adapting via entropy minimization without Bayesian marginalization can lead to overconfident uncertainty estimates, while our principled usage of an approximate training posterior during adaptation outperform previous heuristic methods. These observation support our hypothesis that framing entropy minimization within a well-defined Bayesian model can lead to significantly more effective test-time adaptation techniques. Our method is straightforward to implement, requires minimal changes to standard training procedures, and improves both accuracy and uncertainty estimation for a variety of distribution shifts in classification.
Limitations and future work. One limitation of our approach is that it requires effective techniques for estimating the parameter posterior from the training set. While the study of such methods, and Bayesian neural networks more broadly, is an active area of research, it remains a significant practical challenge. It is likely that the simple Gaussian posteriors we employ provide a poor estimation of the true posterior and can overly constrain the network during adaptation. Therefore, a relevant direction for future work is to integrate BACS with more sophisticated Bayesian neural network methods.
Another promising direction for future work to explore other objectives that have had success in semi- supervisd learning settings, such as consistency based losses (Sajaadi et al.) 2016 Miyato jet al. 2017]] Xie et al.) [019a] Sohn et al.) 2020) or information maximization [ooess al. [2010]. which can be straightforwardly incorporated into our method as suitable priors on the relationship between the data distribution and classifier. More broadly, we hope that our work will spur further research into test-time adaptation techniques based on well-defined Bayesian models that describe
how unlabeled test data should inform our posterior estimates of the model parameters, even in the presence of distributional shift.

---

Acknowledgements
We thank the anonymous reviewers for their extremely valuable feedback and discussions, which have greatly improved our paper. This research was supported by the DARPA Assured Autonomy program and DARPA LwLL, with compute support from Google Cloud and the Tensorflow Research Cloud (TFRC) program.
References
D. Berthelot, G. Research, N. Carlini, I. Goodfellow, A. Oliver, N. Papernot, and C. Raffel. MixMatch:
A Holistic Approach to Semi- Super-suee Learning. Technical report, 2019. URL https://
G. W. Brier. Verification of forecasts expressed in terms of probability. Monthly Weather Review, 78 (I), 1950. ISSN 0027-0644. doi: (41177//220440005500077770000000
M. Carlucci, L.. Porzi, B. Caputo, E. Ricci, and S. R. Bulò. AutoDIAL: Automatic Domaln Alignment Layers. Proceedings of the IEEE International Conference on Computer Vision, 2017-0cctobr::007755555
2017. URL httthtttp/aaxtiii org http://.rrgabb///////// 080822
A. J. Chan, A. M. Alaa, Qian, and M. Van Der Schaar. Unlabelled Data Improves Bayesian Uncer- tainty Calibration under Covariate Shift. Technical report, 11 2020. URL http:::/pp::/// http://pproceedings
mlr. wtwps/vv1//////eeee chan20a. html
A. Coates, A. Ng. and H. Lee. An analysis of single-layer networks unsupervised feature learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pages 215-223, 2011.
Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference computer vision and pattern recognition, pages 248-255, 2009.
G. S. Dhillon, P. Chaudhari, A. Ravichandran, and Soatto. A Baseline for Few-Shot Image Classification. In International Conference Learning Representations, arXiv, 9 2015. URL http://arxiv. wwwpsiiiv.rrr.eooo www/oor///////eoeeeooooeeee abtp///.././/0000 09299929999099
M. W. Dusenberry, G. Jerfel, Y. Wen, Y.a. Ma, J. Snoek, K. Heller, B. Lakshminarayanan, and D. Tran. Efficient and Scalable Bayesian Neural Nets with Rank-1 Factors. arXiv preprint arXiv:2005.071 186, 2020.
N. Ford, J. Gilmer, N. Carlini, and D. Cubuk. Adversarial Examples Are a Natural Consequence of Test Error in Noise. 36th International Conference Machine Learning, ICMI. 2019, 2019-June: 4115-4139, 2019.
abs/ 19tp//99....11 10513v1
S. Fort, H. Hu, and B. Lakshminarayanan, Deep Ensembles: A Loss Landscape Perspective. arXiv, 12 2019. URL htRp:tt::/// htttp://arxiv.org/abbb//
02757
Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. Marchand, and V. Lempitsky. Domain-Adveersarial Training of Neural Networks. Advances in Computer Vision and Pattern Recognition, 1770783319583444))
5 2015. URL htttp://axxiv.rrg oba/
T. Gneiting and A. E. Raftery. Strictly proper scoring rules, prediction, and estimation. Jour- nal of the American Statistical Association, 102(477):359 378, 3 2007. ISSN 01621459, doi: 1998/01111450000000 0162145060000014337
URL httph:tt/ss/tt attps://www.tandfforll onlddfnnliig ine. cow/doi/ale htt///bs/1... 10. 1198/
R. Gomes, Krause, and Perona. Discriminative Clustering by Regularized Information Maxi- mization. Technical report, 2010. URL http://www cs ca/
J. Gordon and J. M. Hernández-Looatt Combining deep generative and discriminative models for Bayesian semi-supervised learning. Pattern Recognition, 100:107156, 4 2020. ISSN 00313203. doi: 10. 10.10 pato:./atco..2111.1077
10

---

Y. Grandvalet and Y. Bengio. Semi-supprrvised Learning by Entropy Minimization. In Advances in Neural Information Processing Systems, volume 17, 2004.
K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Computer Society Conference Computer Vision and Pattern Recognition, volume pages 770-778. IEEE Computer Society, 12 20. ISBN 978 4673884388 doi: 10.1 109/CVP..2016. URL http:htttp:/mmmeee net. org/ Sttaallenngs/LSSR//20001
K. He, X. Zhang. Ren, and J. Sun. Identity Mappings Deep Residual Networks. ECCV, 2016b.
URL https // /tt///axxxt /wwpvv.ivvv///////v///ax////aav/ org/ abs/1603, 050270
05027v3
D. Hendrycks and Dietterich. Benchmarking Neural Network Robustness to Common Corruptions and Perturbations. 7th Internationnal Conference on Learning Representations, ICLR 2019, 3 2019. URL /attp://arxiv... org/ awsaaass/...// 12261
D. Hendrycks, N. Mu, E. D. Cubuk, B. Zoph, J. Gilmer, and B. Lakshninarayanan. AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty. arXiv, 12 2019. URL
httt:://axiil: wwwpxiiv.rrr//bbe abs/ 0912.
D. Hendrycks, S. Basart, N. Mu, S. Kadavath, F. Wang, E. Dorundo, R. Desai, T. Zhu, S. Parajuli, M. Guo, D. Song, Steinhardt, and J. Gilmer. The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization. ICCV, 2021. httpps:///ax org/abs/ 2006. 16241v3
P. Izmailov, D. Podoprikhin, T. Garipov, D. P. Vetrov, and A. G. Wilson, Averaging Weights Leads to Wider Optima and Better Generalization. In UAI, 2018.
A. Krizhevsky. Learning Multiple Layers of Features from Tiny Images. University of Toronto, 2012.
N. Kundu, N. Venkat, R. V, and V. Babu. Universal Source-Free Domain Adaptation. Pro- ceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 4543-4552, 2020. URL https://arxx...rg/abbb////// www.o///////.....//// 04393v1
B. Lakshminarayanan, A. Pritzel, and C. Blundell. Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles. Advances in Neural Information Processing Systems, 2017- December:6403-641 12 2016. URL/tttp://arxi wwwp:////b..././/....
Y. LeCun, C. Cortes, and C. J. Burges. MNIST handwritten digit database. ATT Labs [Online]. Available: http://yann.lecue lecun.
www.:..wwwww..ww.niiiiie wwt.///m/////iisstttommmmmmm 2, 2010.
D.-h. Lee and D.-h. Lec. Pseudo-Label: The Simple and Efficient Semi- Supervieee Learning Method for Deep Neural Networks. ICML Workshop on challenges representation learning, 2013. URL http://citesedt
ist. psu. wdw/ http://eewoo///// /ummmmmydoiim11.....
664 3543
R. Li, Q. Jiao, W. Cao, H. Wong, and S. Wu. Model Adaptation: Unsupervised Domain Adaptation without Source Data. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 9638-9647, 2020. doi: 10. 101119//CCRR22200000000000
J. Liang, D. Hu, and J. Feng. Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation. arXiv, 2 2020. URL http://arxivv orgoorg//aaoo /bs// 2002.08546.
Z.-Y. Liu, S.-Y. Li, S. Chen, Y. Hu, and S.-J. Huang. Uncertainty Aware Graph Gaussian Process for Semi-Supera Semi-Supervised Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 34044): 4957-4964, 4 2020. ISSN 2159-5399, doi: 10. 11009/aai..344444559..... URL www www.aaai. org
J. Ma, W. Tang, J. Zhu, and Q Mei. A Flexible Generative Framework for Graph-based Semi- supervised Learning. arXiv, 2019. URL http://arxiv org/ htte/bbs/1990 10769
W. J. Maddox, P. Izmailov, T. Garipov, D. P. Vetrov, and A. G. Wilson. A simple baseline for bayesian uncertainty in deep learning. Advances Neural Information Processing Systems, pages 1132-13143, 2019.
11

---

T. Miyato, S.-i. Maeda, M. Koyama, and S. Ishii. Virtual Adversarial Training: A Regularization Method for Supervised and Semi- Supervised Learning, IEEE Transactions on Pattern Analysis and Machine Intelligence, 4 41(8):1979-199 19939, 4 2017. hRI http::://www...org///////// wwwp://wwwww/w/////////ob//s
03976
Z. Nado, S. Padhy, D. Sculley, A. D'Amour, B. Lakshminarayanan, and J. Snoek. Evaluating Prediction-Time Time Batch Normalization for Robustness under Covariate Shift. arXiv, 6 2020. URL
Sttt:///axxvv otg. www./og///////////////.//...ee// a0t.///.///00...../// 2006. 10963
M. P. Naeini, G. Cooper, and M. Hauskrecht. Obtaining well calibrated probabilities using bayesian binning, In Twenty-Ninth AAAI Conference on Artificial Intelligence, 2015.
Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, and A. Y. Ng. Reading Digits in Natural Images with Unsupervised Feature Learning. NIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2011.
Y. C. Ng. N. Colombo, and R. Silva. Bayesian Semi-supervised Learning with Graph Gaussian 1693--16446,
Processes, Advances in Neural Information Processing Systems, 2018. URLpttpp://axxi org/ www//a///...// 04379
Y. Ovadia, E. Fertig, J. Ren, Z. Nado, D. Sculley, S. Nowozin, J. V. Dillon, B. Lakshminarayanan, and J. Snoek. Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift. arXiv, 6 2019. URL http:://ariii:: http:://aaxd...rg/abbb//////////
org. htt///bb//1900 190.. 02530
S. Padhy, Nado. J. Ren, J. Liu, Snoek, and B. Lakshminarayanan. Revisiting One-vs-All Classifiers for Predictive Uncertainty and Ofutoo--Dstributton Detection in Neural Networks, arXiv, 2020. URL htR:hhtp://aree http://axxvvvv.e http:/awwwvvv......a......
org abs/ 05134
A. Pampari and S. Ermon. Unsupervised Calibration under Covariate Shift. arXiv, 2020. URL
http://arxiv.. org/ www.orgg////////eoe/ooooeeeeeeee 2006 16405
S. Park, O. Bastani, J. Weimer, and I. Lee. Calibrated Prediction with Covariate Shift via Unsupervised Domain Adaptation. arXiv, 2020. URL http://arxii wwwp:/vv.rrggbb//////0/ 00343
K. Saito, D. Kim, S. Sclaroff, T. Darrell, and K. Saenko. Semi-supervised Domain Adaptation via Minimax Entropy. Proceedings of the IEEE International Conference on Computer Vision, 2019-0Cctober:8049-800,, 4 2019. URL http::tttt:xiiii /ttg///rrxioiio/o/////// http////////./////
M. Sajjadi, M. Javanmardi, and T. Tasdizen. Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learning. Advances in Neural Information Processing Systems, pages 1171-1179, 6 2016. URL http:://aaa/ org/ aba/aaasbb00// 04586
M. Seeger. Input-dependent Regularization of Conditional Density Models. Technical report, 2000.
URL hallhttts//ifoocceenceee wpf.eeeeeeeee..e....e..llllll... ch/eecord/15/442
K. Sohn, D. Berthelot, C.-L. Li, Z. Zhang, N. Carlini, E. D. Cubuk, A. Kurakin, H. Zhang, and C. Raffel. FixMatch: Simplifying Semi-Supervvised Learning with Consistency and Confidence. arXiv, 1 2020. htRphttp://eee
S. Sun, G. Zhang, J. Shi, and R. Grosse Functional Variational Bayesian Neural Networks. 3 2019a. URL http:///rxiv. arxiv.xxivvv. org/ abs/ 1/5//9993.0555 05779
Y. Sun, X. Wang, Z. Liu, J. Miller, A. A. Efros, and M. Hardt. Test-Time Training with Self- Supervision for Generalization under Distribution Shifts. 9 2019b. htttt:////// abs/1909. 13231
R. Taori, A. Dave, V. Shankar, N. Carlini, B. Recht, and L. Schmidt. Measuring Robustness to Natural Distribution Shifts in Image Classification. arXiv, 7 2020. URL.Lccp://axxx..orrr//bbb/ 00644.
C. Tomani and F. Bucttner. Towards Trustworthy Predictions from Deep Neural Networks with Fast Adversarial Calibration. 12 2020. httphhttpp//arrvvvvorrg////
atw.///g///////.a..a/// 10923.
Walker and B. Glocker. Graph Convolutional Gaussian Processes. In International Conference on Machine Learning, 5 2019. URL http:://txiiii..tt/ www.axivvvv.iaaaaaa absp///s..119//.// 05739
12

---

D. Wang. E. Shelhamer, S. Liu, B. Olshausen, and T. Darrell. Tent: Fully Test-time Adaptation by Entropy Minimization. arXiv, 6 2020a. httpp//ttee/aaaaaar..oo/////////
10726
M. Wang and W. Deng. Deep Visual Domain Adaptation: A Survey, 3egp:eoeeeeencoooouuunnnn.222111 312
2 2018. doi: 10.1016/).ecucom.2016 w0w.:/..mm............ 0018.05.083. URL
X. Wang, M. Long, J. Wang, and M. 1. Jordan. Calivvaio with Lower Bias and Variance
in Domain Adaptation. arXiv, 7 2020b. URL http:://axxii. wttpa/wwwwww...//////./////////. 08259
A. G. Wilson and Izmailov. Bayesian Deep Learning and a Probabilistic Perspective of Generaliza-
tion. arXiv, 2 2020. URL /tLL:p////axxx...or/////22000000
Q. Xie, Dai, E. Hovy, M.-T. Mu.n.. and Q. V. Le. Unsuperrvssed Data Augmentation for Consistency Training. arXiv, 4 2019a. http::/: /ttpiiip//axxi..../ wttp 1 12848.
Q. Xie, M.-T. Luong, E. Hovy, and Q. V. Le. Self-training with Noisy Student improves ImageNet classification. Pruceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 10684-10699) 11 2019b. URL http: orggabs/11111 04252
13

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [No] Our contributions are in improving algorithms for robustness and adaptation, rather than any specific application. We expect the broader impacts of our work to be similar to those prior works that are concerned with robustness and adaptation, and do not see any negative societal impacts specific to our work.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full of assumptions all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main exper- imental results (either in the supplemental material or as a URL)? [Yes] Included in supplemental materials.
(b) Did you specify all the training details (c.g., data splits, hyperparameters, how they were chosen)? [Yes] See appendix A.
(c) Did you report error bars (e.g., with respect the random seed after running experi- ments multiple times)? [Yes] See expanded results in appendix with spread across different corruptions.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See appendix A.
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) your work uses existing assets, did you cite the creators? [Yes] All datasets are cited in the text of the experiments section.
(b) Did you mention the license of the assets? [Yes] See appendix A.3.
(c) Did you include any new assets either in the supplemental material URL? [No]
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [N/A]
(e) Did you discuss whether the data you using/curating contains personally identifiable
information or offensive content? [No] We use only standard, widely used datasets in experiments.
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "sjDWNmOyQIL", "reviewer": "Reviewer_3Wsi", "summary": "A principled (Bayesian) approach to adapt model parameters to potential covariate shift at test time with unlabeled test data. The proposed method does not require to keep training data around for adaptation at inference time, is relatively simple and flexible (initial model training can be carried out with any Bayesian method to learn a posterior density), and provides both high accuracy and strong calibration in experiments.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Overall appreciation (originality, quality, clarity, and significance)**\n- A very well written and executed paper overall\n- Language is clear and concise -- very easy to read throughout. Experiments are convincing. Related work is adequately covered. Only a few clarifications needed and typos to correct (see below)\n- Closely related to the approach introduced in “Tent: Fully Test-time Adaptation by Entropy Minimization” (Wang et al. 2020) although the proposed algorithm in this paper is derived in a more principled manner, and delivers stronger experimental results in most cases (all experiments except when test data distribution varies drastically from training, eg. SVHN Vs MNIST -- more on this below)\n\n**Clarifying questions**\n- Why do you restrict the adaptation procedure to operate without any further access to the original training data? For instance, just keeping the learnt approximate posterior and not the full training data is easier to store and optimize (as you allude to on line 162). Are these any other considerations? What are the potential limitations stemming from this assumption? If relaxed, could it potentially help address the issue you are observing with your method when test data differs drastically from training?\n- Your method relies on the availability of several (unlabelled) test points coming from a distribution (potentially) different from training. Did you look into how many such test points are needed for sufficient adaptation across your different experimental settings? \n- How do you select hyperparameters (e.g., alpha*)?\n- How do you interpret results from Table 2 (e.g., better ECE with a single model and no adaptation)?\n- Table 4: is the performance lift over TENT resulting from the ensembling or from the fact you adapt all parameters, not just the batch norm parameters?\n\n**Suggestions**\n- Paper is missing a good \"figure 1\" giving an intuition about / summarizing the approach\n- Would add a clarification regarding the availability of unlabeled test points for calibration. This is not always possible to get in certain applications, eg. when test points come in one-by-one\n- You are also assuming all test points come from the same test distribution, while in practice they may be coming from different distributions (e.g., evolution of underlying dynamics over time)\n- Line 192: would clarify here that TENT focuses on BN parameters (instead of lines 369-371) -- that would help better understanding experimental results\n\n**Minor points**\n- Line 34: typo “minmization’\n- Lines 145-147: you may be missing a word in this sentence\n- Line 147: drop “that” after “However”\n- Line 341: missing a “the” after “We also find that”\n- Line 396: need to replace BENT by BACS?\n- Line 398: drop duplicate word “objectives”\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Well covered overall", "review_text": "", "extra_scores": {"ethical_concerns": "None", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "7"}, "extra_text": {}}, {"review_id": "_LrS5ng_GZ", "reviewer": "Reviewer_RDBC", "summary": "This paper proposes a Bayesian inference framework for test-time adaptation under covariate shift setting.  The setting discussed in the paper is, we have a model trained in a fully supervised way, and observe unlabelled test data (without labels) that might have come from a distribution different from the training data distribution. The goal is to (quickly) adapt the model for the test data without access to the test labels. The proposed method also tries to properly model the epistemic uncertainty of the model to be adapted for the test data, and for this purpose, the problem is formulated as a Bayesian hierarchical model with the data generating distribution and the parameter generating distributions are coupled. Here, an important difference from the previous approaches is that for test data we have a different data generating distribution (conditioning on a different parameter for data generating distribution). The dependence between the model parameter distribution and data generating distribution is expressed in terms of energy-based density whose energies are written as predictive entropies.  The proposed approach entitled Bayesian Adaptation for Covariate Shift (BACS) is demonstrated to be effective in various covariate shift benchmark datasets.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I enjoyed reading the paper, it is clearly written and easy to follow. The related work section is extensive and faithfully covers the literature. \n\nI like the way the authors presented the covariate shift setting as a Bayesian model with coupled dependence between data generating distribution and the parameter distribution (more specifically, the distribution $p(\\theta|\\phi, \\tilde\\phi)$.  It is a classic trick to model such distribution as an energy-based model with entropy energy function (as authors stated in the paper, for example, Grandvalet and Bengio 2014), but the modification with test-time parameter $\\tilde\\phi$ is an interesting extension. The optimization objective derived as a result can be interpreted as a probabilistic version of TENT (Wang et al 2020a) without the regularization term originated from the model posterior obtained from the training set.  The resulting algorithm seems to be easy to implement.\n\nThe experimental results are extensive and convincing. BACS outperformed the baselines for most of the experiments done in terms of both prediction accuracy and uncertainty estimate. The appendix provides further results, especially the results under various corruption levels.\n\nQuestions & comments\n\n1. I guess the hyperparameter $\\tilde\\alpha$ (or $\\beta = 1/\\tilde\\alpha)$ plays an important role in controlling the behaviour of the algorithm. However, judging from the main text and the appendix, the paper does not provide a principled way of choosing it. How did you select the value $\\beta$? It would be hard to apply conventional tuning methods such as cross-validation or Bayesian optimization under test-time adaptation. Also, how sensitive the algorithm is to the choice of $\\beta$?\n\n2. In SVHN to MNIST experiments, BACS slightly underperformed against TENT in terms of prediction accuracy. This is maybe because of the amount of shift between SVHN and MNIST; since MNIST is quite different from SVHN, it requires more adaptation, so the regularization with the model posterior basically requires the adapted parameters to be close to the original model posterior parameters. The authors additionally showed that BACS without the posterior regularization (BACS-posterior) outperforms TENT, and this also implies the sensitivity of BACS to the choice of the parameter $\\beta$. Have you tested with different values of $\\beta$ for this setting? Also, I guess BACS-posterior can serve as a decent baseline to compare. How does it compare against BACS for the other experiments?\n\n3. Often, we don't even know whether given test data is indeed from a shifted distribution. I think it would be interesting to apply BACS to the test data coming from the distribution identical to the training data distribution and see how it affects the performance.\n\n4. I think the role of the posterior regularization (by the term $\\log q_i(\\theta)$) is reminiscent of the regularization techniques used in continual learning, where the goal is to prevent parameters deviate too much from the original one and thus result in catastrophic forgetting.  It would thus be interesting to consider the regularization terms used in continual learning literature for the test time adaptation problem.\n\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The paper properly discusses the limitation in section 5. I agree with the authors that the proposed approach does not have a negative societal impact.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6 hours"}, "extra_text": {}}, {"review_id": "WgJAY8vB73T", "reviewer": "Reviewer_EWK6", "summary": "The paper extends test-time entropy minimization through a novel Bayesian criterion for adaptation. It allows to improve performance during adaptation without degrading the model calibration, a shortcoming of standard entropy minimization. The model is evaluated on CIFAR-10C/-100C/ImageNet-C and compared to batch norm adaptation and test-time entropy minimization.", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The method is overall interesting and I think with a bit of improvements in the writing, the experiments and analysis, this could become a nice contribution. As of now, the results are inconclusive and ablations are lacking given that the loss function introduces a few new hyperparameters.\n\nMajor:\n* The method section needs work. The key idea of the method is currently hidden behind notation and theoretical motivations which are not experimentally verified and analysed. I realize this is matter of taste, but I am in favor of clearly stating out the learning algorithm, highlighting the differences to TENT-only training (ensembles, Bayesian log q(\\theta) update, and then putting the theoretical justification afterwards. The actual practical difference in the loss function to TENT seems minimal (yet elegant!) and this should be highlighted, I think it is actually a selling point of the method.\n* For the key table (Table 2) on large scale data, the method does not seem to fully work (?). The accuracy is only minimally better than TENT (and one issue of the TENT paper is that it does not contain a hyperparameter sweep for ImageNet scale experiments, so the true potential performance will likely be a bit higher than 58%) and the ECE is actually worse than the vanilla network. How would the model selection take place in practice? The model needs to be selected without access to any of the numbers present in Table 2. Also, I would find it important to show mean-corruption error (mCE, see comment further below) apart from accuracy.\n* Why does Table 1 only show the results at the highest corruption level? Could you provide the full picture by providing these numbers on all corruptions?\n* l. 598: It is unclear how the hyperparameters are chosen, especially since they differ from those reported in TENT. Also the new hyperparameter beta is simply selected as 0.001 for small scale experiments and 0.0001 for ImageNet-C, and the early stopping (after 1 vs. 5 vs. 10 epochs) is also arbitrarily selected. I assume good intent and that the authors simply did not test any other setting. If they did, they should include this information and state in case they picked the hyperparameter based on the test performance (which I would accept if it is stated in the main paper). An even better solution would be an ablation on the development corruptions in CIFAR-C/ImageNet-C (this would be 4 corruptions x 5 severities), and selecting beta based on the best accuracy in this dataset. An ablation across beta on said development set would be interesting as I think it will have an impact on the final performance.\n* On ImageNet-C, the *mean corruption error* is a very common metric and indeed proposed as the metric in the official paper. The authors should report this metric in the supplement and/or provide results that make it possible for others to compute it (i.e., at the very minimum, per-corruption accuracies/errors averaged across severities, even better would be a table with all corruptions and severities, or simply computing the mCE). It might also be that mCE will change the model ranking in the ImageNet-C table.\n* I think the comparison to TENT, at the very least its performance, is not fair. The author’s method uses ensembling over 10 models, while only a single TENT model is considered. It is well-known that ensembling improves accuracy. The authors should include results for adapting 10 models with TENT as a baseline. Even if that removes the improvement in terms of accuracy, I would expect that the calibration result will hold true (?)\n* The ECE score is sensitive to the number of bins. Could the authors run the following re-analysis of the results: Test bin sizes [5, 10, 20, 30, 50, …] and report whether this changes the scores on ImageNet-C in a non-trivial way? What was the rationale for picking 20 bins?\n* A ResNet-26 has 17.96 million parameters. In single precision float32, this amounts to 70Mb in weights and 35Mb in half-precision float16. SVHN train is 174Mb large, CIFAR 162Mb. In the light of this, I find it hard to disregard all previous work in domain adaptation on the basis of doing “test-time” adaptation. When the model is only barely smaller than the datasets.\n  * CIFAR10 and SVHN->MNIST results: Using BN adaptation and TENT on these datasets are outdated baselines. Both algorithms were proposed on robustness settings, not primarily on these classical domain adaptation scenarios. Methods like self-ensembling (French 2017), DIRT-T (Shu 2018) and follow up methods could be potential pointers.\n\nSuggestions:\n* I would propose to make a plot that better shows the trade-off between calibration and accuracy.\n* On one of the smaller scale dataset, could you perform the following experiment: Take TENT, compute mCE on the development corruptions, and tune based on ECE and accuracy (jointly using a suitable criterion). What is the upper bound performance of your baseline?\n* Consider including results on ImageNet-R for an instance of more \"natural\" distribution shift.\n* Ablations should be added for all newly introduced parameters, ideally on ImageNet scale (e.g. by running on the development corruptions in ImageNet-C)\n\nMinor:\n* l. 100 It is not clear why $\\theta$ would influence the data distribution in Figure 3.\n* l. 179 How are the predictions aggregated?\n* How do BACS (ours), BACS (MAP) and BACS - posterior compare?\n* I would recommend to replace supplementary figure 4 with violin or scatter plots instead of box plots, to better show the differences between the (very different) various domains in ImageNet-C.\n* The graphical models are misleading. It should be stated that the graphical models depict the predictive distribution, not the dataset distribution (since $\\theta$ -> $y$).\n* Table typesetting could be improved, e.g. by removing vertical lines in all tables.\n\n### Update of original review score based on author responses\n\nThe paper considerably improved during the rebuttal period, most notably for me is \n1. the commitment to get results on the \"real\"/frozen ImageNet-C test set for better comparability to other data for the camera ready version,\n2. the inclusion of strong ImageNet-R results,\n3. the addition of ensembling baselines for TENT and improvement of tables/figures in this regard,\n4. the commitment to situate the small scale domain adaptation results better in the existing literature (and possible tune down this section of the paper to make room for a more extensive discussion of the ImageNet-scale results).\n\nIn the light of the new results provided during the rebuttal phase, the discussion with the other reviewers, as well as the promise to add revised ImageNet-C results for the camera-ready, I decided to increase my score from (5) to (7) and recommend to accept the paper; I also increased the confidence score from (4) to (5). I thank the authors for the great discussion.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The paper discusses limitations to a sufficient extend in the conclusion section.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "-SBFEjdOvlz", "reviewer": "Reviewer_6Ge4", "summary": "This paper presents a Bayesian method for handling a covariate shift in a fully test-time adaptation setting. The authors build on Seeger (2000) to incorporate the unlabeled test examples in their model with an entropy-based empirical prior. They propose a graphical model having three latent variables of interest, the first controls the conditional distribution of the target given the data ($\\theta$), and two variables that govern the input distributions, one for the training data ($\\phi$) and the other for the test data ($\\tilde{\\phi}$). Posterior inference is done only for $\\theta$ while using a point estimate for $\\phi, \\tilde{\\phi}$ with empirical Bayes at the observed data points. To obtain the posterior, first, an approximation of it is obtained using the training data, and second, the approximation is corrected with the test data using an entropy-based term. The authors compared their method against several baselines on corrupted datasets and domain adaptation benchmarks.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper presents the following merits:  \n- Using the Bayesian formalization proposed in the paper makes sense and seems to be original in that context.  \n- The derivation in the paper is elegant and results in an intuitive posterior density.  \n- The resulting adaptation strategy is simple. It is great that training can be done efficiently with standard supervised techniques and that the test-time adaptation is not computationally intensive (when ignoring the costs imposed by using deep ensembles).  \n- The method shows the best results (in both accuracy and uncertainty quantification) in almost all cases against the compared baselines.  \n- Most experimental details required for reproducing the results were given. Code was submitted as well, which is noteworthy.  \n\nConcerns/limitations:  \n- The main concern I have with the paper is that it ignores relevant domain adaptation (DA) studies in the literature review and the experimental section. More specifically, it ignores papers that fall under the umbrella of source-free DA/DA without source data (e.g., [1, 2, 3]). In this setup as well the classifier is initially presented with examples from one domain and no longer has access to them after learning the model and when encountered with examples from a new domain. I think that the authors should have addressed this line of work and evaluate their method against top baselines in this field, at least in the offline setting.  \n- The method suggested in this paper operates under the assumption that the training data is not available during adaptation. Although I acknowledge the importance of this setting, often in real-life scenarios it is possible to maintain and make use of at least some of the training examples. I think that a discussion on how to use BACS in this setting is missing.  \n- I appreciate the diversity of experiments and the datasets used; however, for the DA part more challenging datasets could have been chosen (for example, datasets from Office-Home [4] or ViSDA-C [5]).  \n- Some experimental details are not clear from the text. (1) How did you chose alpha (or conversely beta) and the number of epochs for adaptation in each experiment. Did you have a validation set? If so, did you perform a grid search?  (2) More details about the corrupted datasets than just specifying the references will be great. (3) Did you need to tune hyper-parameters for baseline methods as well? if so, how was it done?  \n\nMinor comments:  \n- Some references should be altered from arXiv to the publication venue (e.g., [6, 7]).\n- According to NeurIPS guidelines, tables should not contain vertical rules.\n- It is not clear from the text what is the vanilla baseline.\n- Some typing mistakes: index is over i in the last line of Algorithm 1, \"is able improves\" (L319), \"We also include show tables\" (L675).  \n\nOverall it is a nice paper with solid results (compared to the baselines included). I think that the Bayesian treatment is justified. There are some issues with the paper, mainly in the experimental section, that should be addressed.  \n  \n[1] Li, R., Jiao, Q., Cao, W., Wong, H. S., & Wu, S. (2020). Model adaptation: Unsupervised domain adaptation without source data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 9641-9650).  \n[2] Kundu, J. N., Venkat, N., & Babu, R. V. (2020). Universal source-free domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 4544-4553).  \n[3] Liang, J., Hu, D., & Feng, J. (2020, November). Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation. In International Conference on Machine Learning (pp. 6028-6039). PMLR.  \n[4] Venkateswara, H., Eusebio, J., Chakraborty, S., & Panchanathan, S. (2017). Deep hashing network for unsupervised domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 5018-5027).  \n[5] Peng, X., Usman, B., Kaushik, N., Hoffman, J., Wang, D., & Saenko, K. (2017). Visda: The visual domain adaptation challenge. arXiv preprint arXiv:1710.06924.  \n[6] Sun, Y., Wang, X., Liu, Z., Miller, J., Efros, A., & Hardt, M. (2020, November). Test-time training with self-supervision for generalization under distribution shifts. In International Conference on Machine Learning (pp. 9229-9248). PMLR.  \n[7] Xie, Q., Dai, Z., Hovy, E., Luong, T., & Le, Q. (2020). Unsupervised Data Augmentation for Consistency Training. Advances in Neural Information Processing Systems, 33.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors addressed the potential limitations and negative societal impact of their work.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "10"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a Bayesian approach to covariate shift using unlabelled test data. There was extensive discussion of the paper, and number of concerns, many of which have moved towards a resolution throughout the response period. Key issues included:
1. Use the standard ImageNet-C test set, instead of the custom one in the submission.
2. Hyperparameter tuning
3. Show results for ensembles of baselines
4. Missing comparisons of DA baselines
5. Concerns about small-scale datasets
6. mCE computation inconsistencies

While it is concurrent work, and does not factor into the evaluation of this submission, it could also be useful to the readership to discuss the recent paper (https://arxiv.org/abs/2106.11905), which shows there are in fact risks of Bayesian model averaging under covariate shift, and explains why deep ensembles don't suffer from these risks.

Overall, this is nice work, and I am supportive of this paper. Please do carefully incorporate all reviewer comments in finalizing the camera-ready, especially point 1, where it is crucial to show an evaluation using the standard test-set for ImageNet-C.
Decision: Accept (Poster)