Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper introduces one algorithm for domain generalization and test-time template adjuster (T3A) for developing a model that performs well under conditions that are different from training conditions. The designed T3A is composed with two procedures: (1) compute a pseudo prototype representation for each class based on unlabeled online dataset, then (2) classify the data based on its distance to the pseudo prototypes. The proposed T3A can improve the performance on the unseen domains, and also outperform the state-of-the-arts.
main_review: Pros:
- Currently, domain generalization is a hot and important topic. Many works only focus on the training phase. However, T3A focuses on altering the testing phase based on the existing model, which is practical in the realistic world. 
- The proposed method in algorithm 1 is simple and easy to implement. 
- There is no training on feature extractor parameters, so the training is so fast.

Cons:
- In the proposed method, the feature extractors trained in each domain are fixed, and the goal is to solve a semi-supervised problem on a sample/prototype basis. Although not in the context of domain generalization, the reviewer thinks that this kind of problem setting was discussed before deep learning became popular. See, for example, reference [a].
-- [a] Driessens K., Reutemann P., Pfahringer B., Leschi C. Using Weighted Nearest Neighbor to Benefit from Unlabeled Data. PAKDD, 2006.
- A survey of sample/prototype-based semi-supervised learning methods prior to deep learning and a discussion of the superiority over these methods is needed.
- Except for the context of domain generalization, the novelty of the proposed method seems to be unclear.

limitations_and_societal_impact: - The overall algorithm utilizes meta learning for boosting the transferability between the source and target domain. But based on the current experiment, the proposed method can perform well on the close-set dataset. I’m wondering if this method can perform well on the partial domain adaptation.
- The paper also mentions that the template trained by source data can’t guarantee that it is a good template for the target dataset. I’m also wondering to what extent that the template on the source data can reflect the feature of the transferred information?
- The format of section 5 is wired. If there is only one subsection, it doesn’t have to be one subsection.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper proposes a new online method for test-time adaptation to domain shift. Specifically, the final layer of a network (i.e. the classifier) is adapted to test data using pseudo-prototypes, which combine the ideas of pseudo-labelling [1] and prototypical representations [2]. That is, pseudo-labels define the support set for each class, and the centroids of these support sets are the prototypes. Experiments demonstrate incremental but reliable performance improvements in the test domain using this optimization-free method.
main_review: **Summary**: Online test-time domain *adaptation* is an interesting and useful research direction, overcoming many of the impracticalities of domain *generalization* (no updating on the test data). The authors’ combination of two existing ideas -- pseudo-labelling [1] and prototypical representations [2] -- seem effective for this setting. The improvements are incremental but consistent. I believe the paper is just above the acceptance threshold, and would be improved with the following: (i) stronger baselines (see below); and (ii) a comparison to “offline” optimization-based methods [3,4] when using two different model selection strategies---training-domain validation-set and test-domain validation-set (oracle) validation sets (see below).

**Originality**: A new combination of well-known techniques for the purpose of *online* test-time adaptation. Related work is adequately cited, making clear how this work differs from source-free domain adaptation (offline vs. online) and prototypical networks (assume access to a few labels in the test domain vs. pseudo-labels).


**Quality**: Overall, the submission is technically sound, the claims well supported, and the methods appropriate. However, the empirical evaluation could be improved with:
- *Additional baselines*
     - *Full Tent method*: Tent-C and Tent-BN are very weak baselines -- they are quite far from the proposed Tent method [4]. In addition, many networks *do* still use batch normalization, including some of the backbones used in this paper (e.g. ResNet50). Thus I think a comparison to the actual Tent method (i.e. updating all BN parameters) is warranted, e.g. for the ResNet models. I think the difficulty of model selection will ensure the proposed method still “wins”, but the comparison is important.
     - *SFDA methods in an online setting*: Do they do poorly? They seem like necessary baselines to show the advantages of the proposed method (e.g. better sample-efficiency due to less updates, or better performance due to easier model selection). E.g. use SHOT[3] with 200 examples per test batch, rather than the current 32. This is the online setting of Tent [4].
     - *Pseudo-labelling*: could the final layer be updated with pseudo-labelling to isolate the benefits of: (i) prototypes vs. pseudo-labelling; and (ii) being optimization-free (easier model selection)?
- *Model selection*: This is a huge advantage of the proposed method over optimization-based methods (e.g. Tent or SFDA methods). Thus, I think it warrants a little more emphasis / analysis:
     - *Lines 261--262*: \emph{These parameters must be selected before the deployment}
     - Compare the performance of your method to these optimization-based methods when using: (i) training-domain validation-set; and (ii) test-domain validation set (oracle). This would very clearly illustrate the limitations of these methods, and thus the advantage of the proposed one. E.g. a figure similar to Figure 1a for different model selection methods.
     - *Figure 1c*: I think this figure is a bit unfair/misleading as: (i) Tent-C is used, rather than the actual Tent method[4], despite ResNet50 having BN layers; and (ii) the poor entropy-reduction of Tent-C is likely explained by poor model selection (training-domain validation set), and this point is only very briefly noted in lines 208--209---not in the caption.

**Clarity**: The submission is clearly written and well organized.

**Significance**: The importance and usefulness is relatively clear. The proposed method improves the state-of-the-art for *online* test-time adaptation on networks with *no batch-normalization*. The significance and usefulness would be much improved by: (i) comparing to the actual Tent method [4] on the resnet models which use BN; and (ii) illustrating the poor suitability of existing *offline* SFDA methods to this “online” problem, and as a result, the need for the proposed method.

**Additional feedback**: 
- *A model can be confident but wrong.* The authors seem to assume that this never happens. E.g. on line 128, “more confidence predictions are more correct” -- this needs to be weakened, e.g. “more confident predictions are often more correct”... but not always. Adversarial examples are one… example, but I’m sure it can happen even when not hand-crafted. I think this is a minor limitation worth acknowledging (also applies to other entropy-reduction methods too).
- *Domain generalization vs. domain adaptation.* I am not sure I agree with calling this a domain *generalization* method, rather than an online method for source-free domain *adaptation*. DG usually implies *no updating* on the test data.



[1] Lee, D. H. (2013). Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In *Workshop on Challenges in Representation Learning, ICML*, 3, pp. 896.

[2] Snell, J., Swersky, K., & Zemel, R. (2017). Prototypical Networks for Few-shot Learning. In *Advances in Neural Information Processing Systems*, 30, pp. 4077-4087.

[3] Liang, J., Hu, D., & Feng, J. (2020). Do we really need to access the source data? Source hypothesis transfer for unsupervised domain adaptation. In *International Conference on Machine Learning*, pp. 6028-6039.

[4] Wang, D., Shelhamer, E., Liu, S., Olshausen, B., & Darrell, T. (2020). Tent: Fully Test-Time Adaptation by Entropy Minimization. In *International Conference on Learning Representations*.

limitations_and_societal_impact: Yes, the authors have addressed the limitations and potential ethical concerns.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper proposed a test-time adaptation (TTA) solution for domain generalization (DG) problems. It exploits prototypical representations, which are used to classify test samples and are refined over time. The method performs well, when compared with standard DG methods and the recently introduced TTA method "TENT".
main_review: Strengths

- The method proposed method is reasonable, and scores favorably with competing algorithms in standard DG benchmarks.

- The paper is overall well written, with Figures well designed and properly placed. 

- The hyper-parameter selection is transparent, also for what concerns the competing algorithm. The plots in Figure 2 are very helpful in assessing the robustness of the proposed method with respect to the baseline across different runs.

Weaknesses

- According to Lines 167/168, the prototypes are refined every time a new test sample is processed: this is consistent with Wang et al. ICLR 2021 (TENT), but it brings an advantage over standard DG methods that the method is compared against - which make their decision by evaluating single samples independently. It would be good to include also some results where the prototypes are not updated, to have a fairer comparison with DG methods. For what concerns the protocol followed in this work and by Wang et al. ICLR 2021, it would be good to also include a comparison against Schneider et al. ICLR 2021 ("Improving robustness against common corruptions by covariate shift adaptation"): it is essentially a simpler version of TENT, where only the BN statistics is updated.

- I believe the comparison with TENT (and with the suggested Schneider et al., ICLR 2021) should be carried out by using a standard ResNet with BN: I am not fully convinced that the precautions taken (lines 246/247) put the competing algorithms in a favorable starting point. It should be fairly simple to test the proposed method with a ResNet-18/50 with BN, am I correct? In this way, the comparison with such related work (the most related to the proposed method) would utmost fair.

Additional comments

- Line 162: why is the support set initialized with the weights of the network, and not with the average between the source features? In Eq.3 new features are included in the support set, and in Eq.5 the average between the elements of the support set is carried out: from the latter Eq., it seems that they are all feature vectors, without weight vectors. Is it a typo or am I getting this wrong?

- The motivation in lines 126-129 are not very strong: it is true that test-time adaptation is yet to be used in DG, but it was tested in very related problems - for instance, under corrupted test samples or natural domain shifts.

- In the related work, it should be included [Chidlovskii et al., "Adaptation in the Absence of Source Domain Data", SIGKDD 2016] for what concerns source-free adaptation, which I believe it is the first work tackling this problem formulation.

- Researchers started exploiting prototypical representations for classification before they were used in the cited few-shot learning works. See, e.g., Mensink et al., "Distance-based image classifi- cation: Generalizing to new classes at near-zero cost" TPAMI 2013.

- Lines 253-258, I think [47] should be [16]

- I do not believe that Remark 1  in Sec.3.2 is relevant: it is widely accepted that finetuning the final layer carries good performance in general.
limitations_and_societal_impact: Yes.
review_title: null

Meta-review: The paper is proposing a test-time adaptation method for domain generalization. The proposed idea is quite simple and effective. It uses the predictions as pseudo labels and only updates the final classification layer without any gradient-based optimization. Hence, it is not only simple and effective but also lightweight. All reviewers agreed that the proposed method has merits; however, there were some concerns with the empirical study. The authors provided significant additional empirical support during rebuttal and reviewers appreciated it. I believe it is a solid paper that will likely have a significant impact on the community.
Decision: Accept (Spotlight)