Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: Test-time adaptation methods seek to improve generalization error from training on the source data to testing on the target data by updating the model on test data to mitigate shift between source and target.
This work re-examines the assumptions for this setting, and in particular identifies adapting _online_ in one pass and adapting _without_ altering the training loss as key considerations for practicality.
Multi-pass adaptation requires too much computation, delaying prediction, and modifying training requires re-training on all of the source data.
While arguing for these strict limits on the latency and the loss, this work relaxes the restriction to not use source data by allowing "light-weight" information (that is, statistics, clusters, etc.).
This variant of test-time training is named sequential test-time training (sTTT).

The test-time anchored clustering (TTAC) method is proposed to address this setting.
The anchored clustering is a joint clustering of the source and target, where the class-wise source clusters are the anchors, and the predicted test clusters are optimized to match the anchors in mean and variance by minimizing the KL divergence.
Both clusterings are computed as mixture-of-Gaussian distributions with one Gaussian per class.
The updates to the target clusters are regularized by exponential moving average (EMA) and stabilized by filtering out noisy predictions according to a queue of predictions on past test data (pseudo-label filtering).
Lastly, filtered out samples can still contribute to an update of the cluster statistics by global feature alignment.

Experiments show that TTAC rivals or improves on test-time adaptation and source-free adaptation baselines.
A variety of settings (one-pass/multi-pass and same/different training) and datasets (CIFAR-10/100-C, CIFAR-10.1, VisDA-C, and ModelNet-40-C) are evaluated.
strengths_and_weaknesses: Strengths

- This work helps explore a newer setting by emphasizing certain dimensions of the test-time adaptation problem, in particular (1) the use of source data, (2) online or one-pass vs. offline or multi-pass updates, and (3) altering training.
  As a newer setting, these choices need to be interrogated. The choice of sTTA in this work is different than some but not all prior work (in particular, BUFR [A] works in the same setting).
- The experiments explore a variety of settings (one-pass/multi-pass and same/different training) and datasets (CIFAR-10/100-C, CIFAR-10.1, VisDA-C, and ModelNet-40).
  Across these settings and datasets TTAC does as well or better than common baselines for test-time adaptation and source-free adaptation.
- The proposed TTAC method keeps improving more with additional data during testing compared to BN or T3A, which plateau at 20% of the test set.
- The use of source data does not alter training. It still requires the full source data for fitting the anchored clusters of TTAC offline, but fitting these clusters is agnostic to architecture, training loss, etc.
- Without source data, in the source-free adaptation setting, a variant of TTAC still works (Table 3) although its accuracy is close to that of existing methods (SHOT, TENT).
- The ablation study covers each component of the method and every setting considered.

Weaknesses

- sTTT is close to existing notions of test-time training, test-time adaptation, and source-free adaptation.
  sTTT underlines one-pass adaptation and minor use of source data, but test-time adaptation already focuses on one-pass adaptation _without_ source data, and several source-free methods report one-pass and multi-pass results (SHOT and BUFR, for example).
  A missing related paper, BUFR, has already argued for minimal use of source data---in their case, the collection of feature histograms---to improve adaptation peformance with results in the setting this work calls sTTT (see one-pass results).
- TTAC lacks technical novelty, as it applies several known components from source-free and test-time adaptation without a particularly strong point of innovation.
  - exponential moving average estimation: EATA [B], BN [C]
  - pseudo-label filtering: SHOT [18] has a prototypical clustering and pseudo-labeling filtering step that serves the same purpose as PLF
  - global feature alignment: TTT+ [19], as acknowledged in this work
  - queuing data to stabilize estimates: TTT+ [19], as acknowledged in this work
  - incremental mean and variance updates are well-known (Equation 8)
- The use of a queue for past data increases the computation for inference and adaptation. Algorithm 1 suggests the use of two devices, one for prediction and one for adaptation, but this is double the resources of fully test-time methods like BN or TENT.
- The experiments do not always align with the motivation and claims. In the fully source-free setting on CIFAR-10-C, the accuracy of TTAC is comparable to other methods like SHOT and TENT. Although sTTT emphasizes one-pass adaptation, Table 4 focuses on multi-pass adaptation, and results on VisDA-C in one pass are not reported.
  Although there is an emphasis on practicality and efficiency, results are not reported on ImageNet-C, which should be possible for an efficient method. It is potentially worrisome that most results are reported on the smallest dataset, CIFAR-10-C, and even on CIFAR-100-C the gap with existing methods shrinks (Table 1).
- (Minor) The writing needs editing throughout for word choice and grammar. (See "Other Feedback" in the next section for typos spotted.)


[A] BUFR: Source-Free Adaptation to Measurement Shift via Bottom-Up Feature Restoration. Eastwood et al. ICLR'22 and arXiv'21. 

[B] EATA: Efficient Test-Time Model Adaptation without Forgetting. Niu et al. ICML'22 and arXiv'22 (~2 months before the NeurIPS deadline). 

[C] BN: Improving robustness against common corruptions by covariate shift adaptation. Schneider et al. Neurips'20.

**Update**: I have raised my score given the benchmarking on ImageNet-C and profiling of computation time.
questions: Questions

- Why is ImageNet-C not evaluated? How does TTAC perform on it? It is a gold standard benchmark, and results on CIFAR-10/100-C do not always reflect results at larger scale.
- How many updates are made when updating the network on the test input and sampling queue? Is there a single update per batch, as in TENT, or multiple updates like in TTT or MEMO?
- How much computation is needed for TTAC? Could you measure it in number of forward passes/backward passes or wallclock time per test sample? In particular, what is the overhead of the queueing?

Other Feedback

- Figure 1 is complex! To make it more accessible and informative, consider taking certain points and moving them into their own figure. For instance, PLF could perhaps be explained on its own.
- Note that SHOT has a journal edition in PAMI'21 with improved results. It is preferable to compare to the latest and strongest edition of the method.
- Here are a few typos noticed during review: "it is inevitable to access" on line 58 could be "still requires access to".
  On line 125 there is a missing period after "alone". On line 241 "Ours sTTT" should be "Our sTTT". There are more.

limitations: - There is no explicit discussion of limitations.
- The computational overhead of queuing data for statistics and parameter updates needs acknowledgement and ideally measurement. TTAC is more expensive than BN or TENT.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper proposes a clustering-based method for sequential test-time training, where the test data under distribution shifts arrives in a stream manner. The core idea of the proposed method is to match the cluster statistics between the source and target domain. To obtain a robust estimate of the cluster statistics, the paper introduces a pseudo label filtering strategy as well as global feature alignment. The empirical results are better than several previous methods on multiple datasets. 
strengths_and_weaknesses: Strengths
* The categorization of test-time algorithms proposed in the paper is neat and convincing. Indeed, the evaluation protocol in test-time training and source-free adaptation has been sometimes inconsistent. The paper does a great job to streamline the evaluation protocols. 
* The proposed anchored clustering also looks technically sound. While clustering, pseudo labelling and feature alignment have been separately explored in recent methods, the paper nicely connects them in a principled way. I also enjoy reading the detailed modifications of these components, e.g., eq 8.
* The empirical evaluation of the method looks complete and rigorous. It’s great to see that the proposed TTAC is compatible with other methods like SHOT and TTT++. 

Weaknesses
* Many prior test-time algorithms work only under a narrow set of distribution shifts. I suspect the proposed method may be subject to some particular settings too. It would be good if the authors could clearly discuss limitations and failure cases.
* In particular, the proposed method introduces additional hyperparameters for peusde label filtering. It would be good to know how these parameters affect the performance.

questions: * How sensitive is the proposed method to the threshold of pseudo label filtering?

limitations: The authors did not clearly discuss limitations of their method.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: In spite of the numerous efforts put into test-time training (TTT), the experimental settings are unclear, resulting in unfair comparisons.
This paper revisits TTT assumptions and classifies TTT protocols. This paper aims to tackle on the most realistic and challenging TTT protocol (one-pass test time training with no modifications to training objective). Motivated by the confused evaluation protocols for TTT, the authors propose test-time anchored clustering (TTAC) approach to align target domain features to the source ones in sequential test-time training. On five TTT datasets, TTAC consistently beats state-of-the-art approaches.
strengths_and_weaknesses: Strength:
- This paper is well-written and the whole part including motivation, method and experiment are easy to understand.
- Clear and detailed descriptions of the proposed method, which provide enough information to reproduce.
- This paper presents a categorization of TTT protocols based on two key factors so that comparisons of TTT methods within each category could be fair.
- This paper shows the experimental results for various experimental protocols for each method, which can serve as a baseline for subsequent TTT-related studies.

Weakness:
- This paper is not the first attempt to use statistical information between the source domain and the target domain to alleviate domain discrepancy. [1] was not mentioned even though it is very relevant to this paper in that it reduces the difference between the centroid (mean) of the source domain and the centroid (mean) of the target domain via pseudo labels or uses a moving average scheme. Although this paper uses covariance in addition to mean, novelty is not enough.
- On line 125, there must be a space between alone and we.


[1] Xie, Shaoan, et al. "Learning semantic representations for unsupervised domain adaptation." ICML 2018.
questions: - Sensitivity analysis for key hyperparameters such as thresholds used for pseudo label filtering related was not presented. I would like to know the trends related to key hyperparameters.
limitations: There is no description of the limitations or potential negative social impacts of the proposed method.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: Test-time training is an emerging approach for building robust machine learning models to distribute shifts. The main contribution of this paper summarizes the following two points. (1) They distinguish experiment setups, from the perspective of whether it uses on-pass adaptation or multi-pass adaptation and whether it needs to alter the training phase.  The distinction is important as prior works often compared different setups, leading to unfair comparisons. (2)  They provide a new method on realistic test-time training setup, called test-time anchored clustering. Specifically, they use a mixture of Gaussians to model the clusters in the target domain, and use pseudo-label to match it with the mixture distribution of source data in closed form. Empirical results show that the proposed method outperforms existing methods on several benchmark datasets. 

strengths_and_weaknesses: ---
**Strengths**

(1) The paper is well written and easy to follow. 

(2)  They pointed out important confusion about evaluation in the fields of test-time training. I strongly agree that the confusion often leads to unfair comparisons, and the clarification of this paper makes it easier for subsequent research to conduct fairer comparisons, which I think very important for scientific progress.  

(3) This paper provides extensive empirical validations, carefully investigating how each component of the proposal contributes to the performance gain. 

---
**Weaknesses**

(1) Why do we need to represent each distribution by a mixture of Gaussian distributions? In other words, is it better than simply aligning the statistics (e.g, mean and covariance) of source data of a category and that of the target data predicted as the category? The former approach has been already proposed in test-time adaptation recently [1], it is better to clarify the difference. 

[1] Kojima, Takeshi et al. “Robustifying Vision Transformer without Retraining from Scratch by Test-Time Class-Conditional Feature Alignment.” IJCAI2022

(2) Comparing sTTA and other TTA setup, one notable difficulty of sTTA might be how to choose the hyperparameter. Because there is no clue about test-distribution when we choose the hyper parameter (e.g., learning rate, optimizer, tau for the threshold, etc) in sTTA, there is a reasonable way to determine the hyper parameter. From the current manuscript, unfortunately, how to tune the hyperparameter for each test-time training method is unclear for me, which makes me wonder whether the reported results are actually fair or not. 

(3) According to Table 5, major performance gains come from global feature alignment using KLD. In my understanding, several prior methods (e.g., TFA and TTT+) use similar feature alignment loss to match the statistics of source and target distribution. Does the results suggest KLD is better than other feature alignment approaches, or does there any difference between setup among these methods?

(Minor comments)
- line 124 needs a period. 
- line 131: should clarify what is u and sigma (d-dimensional mean vector and covariance matrix?)

questions: (1) Why do we need to represent each distribution by a mixture of Gaussian distributions? In other words, is it better than simply aligning the statistics (e.g, mean and covariance) of source data of a category and that of the target data predicted as the category?

(2) How did you tune the hyper parameters for test-time training methods? 

(3) Why is SHOT categorized as N-O or N-M? In my understanding, the original SHOT contains several proposals including label-smoothing techniques for source training, and thus require special objective function for source training. Does the value reported here not use label-smoothing? 

(4) According to Table 5, major performance gains come from global feature alignment using KLD. In my understanding, several prior methods (e.g., TFA and TTT+) use similar feature alignment loss to match the statistics of source and target distribution. Does the results suggest KLD is better than other feature alignment approaches, or does there any difference between setup among these methods?

limitations: This paper does not provide limitation section in the main text. I recommend the author to discuss the potential limitation of the proposed method (e.g. hyper parameter tuning mentioned in weakness in my review), and potential drawback of test-time training itself (e.g., do we really want to adapt the model during test-time in the safety critical application?).
review_title: null

Meta-review: metareview: The paper proposes a clustering-based method for sequential test-time training, where the test data under distribution shifts arrives in a stream manner. The authors also explore a newer setting by emphasizing certain dimensions of the test-time adaptation problem: they distinguish experiment setups, from the perspective of whether it uses on-pass adaptation or multi-pass adaptation and whether it needs to alter the training phase. The distinction is important as prior works often compared under different setups, leading to unfair comparisons. The authors did a good job to address the reviewers' concerns in the author-reviewer discussion phase, and at the end, all reviewers unanimously support the acceptance. AC also did not find a particular weakness for rejection. As the test-time adaptation has emerged as a realistic solution to make ML models more robust to work, AC thinks that the contribution of this paper is of interest to a broad range of NeurIPS audience and would guide future research on the topic. Overall, AC is happy to recommend the acceptance.
confidence: Certain
Decision: Accept