Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: A principled (Bayesian) approach to adapt model parameters to potential covariate shift at test time with unlabeled test data. The proposed method does not require to keep training data around for adaptation at inference time, is relatively simple and flexible (initial model training can be carried out with any Bayesian method to learn a posterior density), and provides both high accuracy and strong calibration in experiments.
main_review: **Overall appreciation (originality, quality, clarity, and significance)**
- A very well written and executed paper overall
- Language is clear and concise -- very easy to read throughout. Experiments are convincing. Related work is adequately covered. Only a few clarifications needed and typos to correct (see below)
- Closely related to the approach introduced in “Tent: Fully Test-time Adaptation by Entropy Minimization” (Wang et al. 2020) although the proposed algorithm in this paper is derived in a more principled manner, and delivers stronger experimental results in most cases (all experiments except when test data distribution varies drastically from training, eg. SVHN Vs MNIST -- more on this below)

**Clarifying questions**
- Why do you restrict the adaptation procedure to operate without any further access to the original training data? For instance, just keeping the learnt approximate posterior and not the full training data is easier to store and optimize (as you allude to on line 162). Are these any other considerations? What are the potential limitations stemming from this assumption? If relaxed, could it potentially help address the issue you are observing with your method when test data differs drastically from training?
- Your method relies on the availability of several (unlabelled) test points coming from a distribution (potentially) different from training. Did you look into how many such test points are needed for sufficient adaptation across your different experimental settings? 
- How do you select hyperparameters (e.g., alpha*)?
- How do you interpret results from Table 2 (e.g., better ECE with a single model and no adaptation)?
- Table 4: is the performance lift over TENT resulting from the ensembling or from the fact you adapt all parameters, not just the batch norm parameters?

**Suggestions**
- Paper is missing a good "figure 1" giving an intuition about / summarizing the approach
- Would add a clarification regarding the availability of unlabeled test points for calibration. This is not always possible to get in certain applications, eg. when test points come in one-by-one
- You are also assuming all test points come from the same test distribution, while in practice they may be coming from different distributions (e.g., evolution of underlying dynamics over time)
- Line 192: would clarify here that TENT focuses on BN parameters (instead of lines 369-371) -- that would help better understanding experimental results

**Minor points**
- Line 34: typo “minmization’
- Lines 145-147: you may be missing a word in this sentence
- Line 147: drop “that” after “However”
- Line 341: missing a “the” after “We also find that”
- Line 396: need to replace BENT by BACS?
- Line 398: drop duplicate word “objectives”

limitations_and_societal_impact: Well covered overall
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The paper extends test-time entropy minimization through a novel Bayesian criterion for adaptation. It allows to improve performance during adaptation without degrading the model calibration, a shortcoming of standard entropy minimization. The model is evaluated on CIFAR-10C/-100C/ImageNet-C and compared to batch norm adaptation and test-time entropy minimization.
main_review: The method is overall interesting and I think with a bit of improvements in the writing, the experiments and analysis, this could become a nice contribution. As of now, the results are inconclusive and ablations are lacking given that the loss function introduces a few new hyperparameters.

Major:
* The method section needs work. The key idea of the method is currently hidden behind notation and theoretical motivations which are not experimentally verified and analysed. I realize this is matter of taste, but I am in favor of clearly stating out the learning algorithm, highlighting the differences to TENT-only training (ensembles, Bayesian log q(\theta) update, and then putting the theoretical justification afterwards. The actual practical difference in the loss function to TENT seems minimal (yet elegant!) and this should be highlighted, I think it is actually a selling point of the method.
* For the key table (Table 2) on large scale data, the method does not seem to fully work (?). The accuracy is only minimally better than TENT (and one issue of the TENT paper is that it does not contain a hyperparameter sweep for ImageNet scale experiments, so the true potential performance will likely be a bit higher than 58%) and the ECE is actually worse than the vanilla network. How would the model selection take place in practice? The model needs to be selected without access to any of the numbers present in Table 2. Also, I would find it important to show mean-corruption error (mCE, see comment further below) apart from accuracy.
* Why does Table 1 only show the results at the highest corruption level? Could you provide the full picture by providing these numbers on all corruptions?
* l. 598: It is unclear how the hyperparameters are chosen, especially since they differ from those reported in TENT. Also the new hyperparameter beta is simply selected as 0.001 for small scale experiments and 0.0001 for ImageNet-C, and the early stopping (after 1 vs. 5 vs. 10 epochs) is also arbitrarily selected. I assume good intent and that the authors simply did not test any other setting. If they did, they should include this information and state in case they picked the hyperparameter based on the test performance (which I would accept if it is stated in the main paper). An even better solution would be an ablation on the development corruptions in CIFAR-C/ImageNet-C (this would be 4 corruptions x 5 severities), and selecting beta based on the best accuracy in this dataset. An ablation across beta on said development set would be interesting as I think it will have an impact on the final performance.
* On ImageNet-C, the *mean corruption error* is a very common metric and indeed proposed as the metric in the official paper. The authors should report this metric in the supplement and/or provide results that make it possible for others to compute it (i.e., at the very minimum, per-corruption accuracies/errors averaged across severities, even better would be a table with all corruptions and severities, or simply computing the mCE). It might also be that mCE will change the model ranking in the ImageNet-C table.
* I think the comparison to TENT, at the very least its performance, is not fair. The author’s method uses ensembling over 10 models, while only a single TENT model is considered. It is well-known that ensembling improves accuracy. The authors should include results for adapting 10 models with TENT as a baseline. Even if that removes the improvement in terms of accuracy, I would expect that the calibration result will hold true (?)
* The ECE score is sensitive to the number of bins. Could the authors run the following re-analysis of the results: Test bin sizes [5, 10, 20, 30, 50, …] and report whether this changes the scores on ImageNet-C in a non-trivial way? What was the rationale for picking 20 bins?
* A ResNet-26 has 17.96 million parameters. In single precision float32, this amounts to 70Mb in weights and 35Mb in half-precision float16. SVHN train is 174Mb large, CIFAR 162Mb. In the light of this, I find it hard to disregard all previous work in domain adaptation on the basis of doing “test-time” adaptation. When the model is only barely smaller than the datasets.
  * CIFAR10 and SVHN->MNIST results: Using BN adaptation and TENT on these datasets are outdated baselines. Both algorithms were proposed on robustness settings, not primarily on these classical domain adaptation scenarios. Methods like self-ensembling (French 2017), DIRT-T (Shu 2018) and follow up methods could be potential pointers.

Suggestions:
* I would propose to make a plot that better shows the trade-off between calibration and accuracy.
* On one of the smaller scale dataset, could you perform the following experiment: Take TENT, compute mCE on the development corruptions, and tune based on ECE and accuracy (jointly using a suitable criterion). What is the upper bound performance of your baseline?
* Consider including results on ImageNet-R for an instance of more "natural" distribution shift.
* Ablations should be added for all newly introduced parameters, ideally on ImageNet scale (e.g. by running on the development corruptions in ImageNet-C)

Minor:
* l. 100 It is not clear why $\theta$ would influence the data distribution in Figure 3.
* l. 179 How are the predictions aggregated?
* How do BACS (ours), BACS (MAP) and BACS - posterior compare?
* I would recommend to replace supplementary figure 4 with violin or scatter plots instead of box plots, to better show the differences between the (very different) various domains in ImageNet-C.
* The graphical models are misleading. It should be stated that the graphical models depict the predictive distribution, not the dataset distribution (since $\theta$ -> $y$).
* Table typesetting could be improved, e.g. by removing vertical lines in all tables.

### Update of original review score based on author responses

The paper considerably improved during the rebuttal period, most notably for me is 
1. the commitment to get results on the "real"/frozen ImageNet-C test set for better comparability to other data for the camera ready version,
2. the inclusion of strong ImageNet-R results,
3. the addition of ensembling baselines for TENT and improvement of tables/figures in this regard,
4. the commitment to situate the small scale domain adaptation results better in the existing literature (and possible tune down this section of the paper to make room for a more extensive discussion of the ImageNet-scale results).

In the light of the new results provided during the rebuttal phase, the discussion with the other reviewers, as well as the promise to add revised ImageNet-C results for the camera-ready, I decided to increase my score from (5) to (7) and recommend to accept the paper; I also increased the confidence score from (4) to (5). I thank the authors for the great discussion.
limitations_and_societal_impact: The paper discusses limitations to a sufficient extend in the conclusion section.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes a Bayesian inference framework for test-time adaptation under covariate shift setting.  The setting discussed in the paper is, we have a model trained in a fully supervised way, and observe unlabelled test data (without labels) that might have come from a distribution different from the training data distribution. The goal is to (quickly) adapt the model for the test data without access to the test labels. The proposed method also tries to properly model the epistemic uncertainty of the model to be adapted for the test data, and for this purpose, the problem is formulated as a Bayesian hierarchical model with the data generating distribution and the parameter generating distributions are coupled. Here, an important difference from the previous approaches is that for test data we have a different data generating distribution (conditioning on a different parameter for data generating distribution). The dependence between the model parameter distribution and data generating distribution is expressed in terms of energy-based density whose energies are written as predictive entropies.  The proposed approach entitled Bayesian Adaptation for Covariate Shift (BACS) is demonstrated to be effective in various covariate shift benchmark datasets.
main_review: I enjoyed reading the paper, it is clearly written and easy to follow. The related work section is extensive and faithfully covers the literature. 

I like the way the authors presented the covariate shift setting as a Bayesian model with coupled dependence between data generating distribution and the parameter distribution (more specifically, the distribution $p(\theta|\phi, \tilde\phi)$.  It is a classic trick to model such distribution as an energy-based model with entropy energy function (as authors stated in the paper, for example, Grandvalet and Bengio 2014), but the modification with test-time parameter $\tilde\phi$ is an interesting extension. The optimization objective derived as a result can be interpreted as a probabilistic version of TENT (Wang et al 2020a) without the regularization term originated from the model posterior obtained from the training set.  The resulting algorithm seems to be easy to implement.

The experimental results are extensive and convincing. BACS outperformed the baselines for most of the experiments done in terms of both prediction accuracy and uncertainty estimate. The appendix provides further results, especially the results under various corruption levels.

Questions & comments

1. I guess the hyperparameter $\tilde\alpha$ (or $\beta = 1/\tilde\alpha)$ plays an important role in controlling the behaviour of the algorithm. However, judging from the main text and the appendix, the paper does not provide a principled way of choosing it. How did you select the value $\beta$? It would be hard to apply conventional tuning methods such as cross-validation or Bayesian optimization under test-time adaptation. Also, how sensitive the algorithm is to the choice of $\beta$?

2. In SVHN to MNIST experiments, BACS slightly underperformed against TENT in terms of prediction accuracy. This is maybe because of the amount of shift between SVHN and MNIST; since MNIST is quite different from SVHN, it requires more adaptation, so the regularization with the model posterior basically requires the adapted parameters to be close to the original model posterior parameters. The authors additionally showed that BACS without the posterior regularization (BACS-posterior) outperforms TENT, and this also implies the sensitivity of BACS to the choice of the parameter $\beta$. Have you tested with different values of $\beta$ for this setting? Also, I guess BACS-posterior can serve as a decent baseline to compare. How does it compare against BACS for the other experiments?

3. Often, we don't even know whether given test data is indeed from a shifted distribution. I think it would be interesting to apply BACS to the test data coming from the distribution identical to the training data distribution and see how it affects the performance.

4. I think the role of the posterior regularization (by the term $\log q_i(\theta)$) is reminiscent of the regularization techniques used in continual learning, where the goal is to prevent parameters deviate too much from the original one and thus result in catastrophic forgetting.  It would thus be interesting to consider the regularization terms used in continual learning literature for the test time adaptation problem.



limitations_and_societal_impact: The paper properly discusses the limitation in section 5. I agree with the authors that the proposed approach does not have a negative societal impact.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper presents a Bayesian method for handling a covariate shift in a fully test-time adaptation setting. The authors build on Seeger (2000) to incorporate the unlabeled test examples in their model with an entropy-based empirical prior. They propose a graphical model having three latent variables of interest, the first controls the conditional distribution of the target given the data ($\theta$), and two variables that govern the input distributions, one for the training data ($\phi$) and the other for the test data ($\tilde{\phi}$). Posterior inference is done only for $\theta$ while using a point estimate for $\phi, \tilde{\phi}$ with empirical Bayes at the observed data points. To obtain the posterior, first, an approximation of it is obtained using the training data, and second, the approximation is corrected with the test data using an entropy-based term. The authors compared their method against several baselines on corrupted datasets and domain adaptation benchmarks.
main_review: The paper presents the following merits:  
- Using the Bayesian formalization proposed in the paper makes sense and seems to be original in that context.  
- The derivation in the paper is elegant and results in an intuitive posterior density.  
- The resulting adaptation strategy is simple. It is great that training can be done efficiently with standard supervised techniques and that the test-time adaptation is not computationally intensive (when ignoring the costs imposed by using deep ensembles).  
- The method shows the best results (in both accuracy and uncertainty quantification) in almost all cases against the compared baselines.  
- Most experimental details required for reproducing the results were given. Code was submitted as well, which is noteworthy.  

Concerns/limitations:  
- The main concern I have with the paper is that it ignores relevant domain adaptation (DA) studies in the literature review and the experimental section. More specifically, it ignores papers that fall under the umbrella of source-free DA/DA without source data (e.g., [1, 2, 3]). In this setup as well the classifier is initially presented with examples from one domain and no longer has access to them after learning the model and when encountered with examples from a new domain. I think that the authors should have addressed this line of work and evaluate their method against top baselines in this field, at least in the offline setting.  
- The method suggested in this paper operates under the assumption that the training data is not available during adaptation. Although I acknowledge the importance of this setting, often in real-life scenarios it is possible to maintain and make use of at least some of the training examples. I think that a discussion on how to use BACS in this setting is missing.  
- I appreciate the diversity of experiments and the datasets used; however, for the DA part more challenging datasets could have been chosen (for example, datasets from Office-Home [4] or ViSDA-C [5]).  
- Some experimental details are not clear from the text. (1) How did you chose alpha (or conversely beta) and the number of epochs for adaptation in each experiment. Did you have a validation set? If so, did you perform a grid search?  (2) More details about the corrupted datasets than just specifying the references will be great. (3) Did you need to tune hyper-parameters for baseline methods as well? if so, how was it done?  

Minor comments:  
- Some references should be altered from arXiv to the publication venue (e.g., [6, 7]).
- According to NeurIPS guidelines, tables should not contain vertical rules.
- It is not clear from the text what is the vanilla baseline.
- Some typing mistakes: index is over i in the last line of Algorithm 1, "is able improves" (L319), "We also include show tables" (L675).  

Overall it is a nice paper with solid results (compared to the baselines included). I think that the Bayesian treatment is justified. There are some issues with the paper, mainly in the experimental section, that should be addressed.  
  
[1] Li, R., Jiao, Q., Cao, W., Wong, H. S., & Wu, S. (2020). Model adaptation: Unsupervised domain adaptation without source data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 9641-9650).  
[2] Kundu, J. N., Venkat, N., & Babu, R. V. (2020). Universal source-free domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 4544-4553).  
[3] Liang, J., Hu, D., & Feng, J. (2020, November). Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation. In International Conference on Machine Learning (pp. 6028-6039). PMLR.  
[4] Venkateswara, H., Eusebio, J., Chakraborty, S., & Panchanathan, S. (2017). Deep hashing network for unsupervised domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 5018-5027).  
[5] Peng, X., Usman, B., Kaushik, N., Hoffman, J., Wang, D., & Saenko, K. (2017). Visda: The visual domain adaptation challenge. arXiv preprint arXiv:1710.06924.  
[6] Sun, Y., Wang, X., Liu, Z., Miller, J., Efros, A., & Hardt, M. (2020, November). Test-time training with self-supervision for generalization under distribution shifts. In International Conference on Machine Learning (pp. 9229-9248). PMLR.  
[7] Xie, Q., Dai, Z., Hovy, E., Luong, T., & Le, Q. (2020). Unsupervised Data Augmentation for Consistency Training. Advances in Neural Information Processing Systems, 33.
limitations_and_societal_impact: The authors addressed the potential limitations and negative societal impact of their work.
review_title: null

Meta-review: This paper proposes a Bayesian approach to covariate shift using unlabelled test data. There was extensive discussion of the paper, and number of concerns, many of which have moved towards a resolution throughout the response period. Key issues included:
1. Use the standard ImageNet-C test set, instead of the custom one in the submission.
2. Hyperparameter tuning
3. Show results for ensembles of baselines
4. Missing comparisons of DA baselines
5. Concerns about small-scale datasets
6. mCE computation inconsistencies

While it is concurrent work, and does not factor into the evaluation of this submission, it could also be useful to the readership to discuss the recent paper (https://arxiv.org/abs/2106.11905), which shows there are in fact risks of Bayesian model averaging under covariate shift, and explains why deep ensembles don't suffer from these risks.

Overall, this is nice work, and I am supportive of this paper. Please do carefully incorporate all reviewer comments in finalizing the camera-ready, especially point 1, where it is crucial to show an evaluation using the standard test-set for ImageNet-C.
Decision: Accept (Poster)