Revisiting Realistic Test-Time Training: Sequential Inference and Adaptation by Anchored Clustering
Authors: Yongyi Su
Xun Xu
Kui Jia
Venue: NeurIPS 2022 Accept (2022)
Keywords: Test-Time Training
Domain Adaptation
Pseudo Labeling

Abstract: Deploying models on target domain data subject to distribution shift requires adaptation. Test-time training (TTT) emerges as a solution to this adaptation under a realistic scenario where access to full source domain data is not available and instant inference on target domain is required. Despite many efforts into TTT, there is a confusion over the experimental settings, thus leading to unfair comparisons. In this work, we first revisit TTT assumptions and categorize TTT protocols by two key factors. Among the multiple protocols, we adopt a realistic sequential test-time training (sTTT) protocol, under which we further develop a test-time anchored clustering (TTAC) approach to enable stronger test-time feature learning. TTAC discovers clusters in both source and target domain and match the target clusters to the source ones to improve generalization. Pseudo label filtering and iterative updating are developed to improve the effectiveness and efficiency of anchored clustering. We demonstrate that under all TTT protocols TTAC consistently outperforms the state-of-the-art methods on six TTT datasets. We hope this work will provide a fair benchmarking of TTT methods and future research should be compared within respective protocols. A demo code is available at https://github.com/Gorilla-Lab-SCUT/TTAC.

Revisiting Realistic Test-Time Training: Sequential Inference and Adaptation by Anchored Clustering
Yongyi Su Xun Xu Kui Jia 13f
South China University of Technology "Institute for Infocomm Research Speng Cheng Laboratory
eesuyongyi seaennyy@mmall.m ttt.mmmmauu........mmmmmmamaaaae edu. alex. xun xu@gmail..... com
kuijiaEscut, edu. cn
Abstract
Deploying models on target domain data subject to distribution shift requires adaptation. Test-time training (TTT) emerges a solution to this adaptation under a realistic scenario where access to full source domain data not available and instant inference TTT, there is a confusion over the experimental settings, thus leading to unfair comparisons. In this work, we first revisit TTT assumptions and categorize TTT protocols by two key factors. Among the multiple protocols, we adopt a realistic sequential test-time training (sTTT) protocol, under which further develop test-time anchored clustering (TTAC) approach to enable stronger rest-time feature learning. TTAC discovers clusters in both source and target domain and match the target clusters to the source ones to improve generalization. Pseudo label filtering and iterative updating developed improve the effectiveness and efficiency of anchored clustering. We demonstrate that under all TTT protocols TTAC consistently outperforms the state-of-the-ar methods on six TTT datasets. We hope this work will provide fair benchmarking of TTT methods and future research should be compared within respective protocols. A demo code is available at httth:t:/// / / thub. com/ Lat.eilla-Laa-SUT/AAA
target domain is required. Despite many efforts into
Introduction
The recent success in deep learning is attributed to the availability of large labeled data [15] 40 and the assumption of i.i.d. between training and test datasets. Such assumptions could be violated when test data features a drastic difference from the training data, e.g. training on synthetic images and test on real ones, and this is often referred to as domain shift [22 2 To tackle this issue, domain adaptation (DA) [32 emerges and the labeled training data and unlabeled testing data are often referred to as source and target data/domains respectively.
The existing DA works either require the access to both source and target domain data during training (5] or training on multiple domains simultaneously 39 The former approach renders the methods restrictive to limited scenarios where source domain data is always available during adaptation while the latter ones are computationally more expensive. To alleviate the reliance on source domain data, which may be inaccessible due to privacy issues or storage overhead, source-free domain adaptation (SFDA) emerges which handles DA on target data without access to SFDA is often achieved through self-training [19, self self-supervised
source data [9 16. 37
35 20
"gqual contribution
CCorrespondecce to Skujja@@sct.eduu.cm & <alex.xun.xu @ www.gml.coommo
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

learning (00] or introducing prior knowledge [9] and it requires multiple training epochs on the full target data to allow convergence. Despite easing the dependence on source data, SFDA has major drawbacks in more realistic domain adaptation scenario where test data arrives in a stream and inference or prediction must be taken instantly, and this setting often referred to as test-time training (TTT) or adaptation (TTA) [27 31 13 20]. Despite the attractive feature of adaption at time test, we notice a confusion of what defines a test-time training and as result comparing apples and oranges happens frequently in the community. In this work, we first categorize TTT by two key factors after summarizing various definitions made in existing works. First, under a realistic TTT setting, test samples are sequentially streamed and prediction must be made instantly upon the arrival of a new test sample. More specifically, the prediction of test sample Xr, arriving at time stamp T, should not be affected by any subsequent samples, (Xi)==+11...0 Throughout this work, we refer to the sequential streaming as one-pass adaptation protocol and any other protocols violating this assumption are called multi-pass adaptation (model may be updated on all test data for multiple epochs before inference). Second, we notice some recent works must modify source domain training loss, e.g. by introducing additional self-supervised branch, to allow more effective TTT 127 20 This will introduce additional overhead in the deployment of TTT because re-training on some source dataset, e.g. ImageNet, is computationally expensive. In this work, we aim to tackle on the most realistic and challenging TTT protocol, i.e. one-pass test time training with no modifications training objective. This setting is similar to TTA proposed in [31 except for not restricting access to a light-weigh information from the source domain. Given the objective of TTT being efficient adaptation test-time, this assumption computationally efficient and improves TTT performance substantially. We name this new TTT protocol as sequential test time training (sTTT).
We propose two techniques enable efficient and accurate STTT. i) We are inspired by the recent progresses in unsupervised domain adaptation (28) that encourages testing samples to form clusters in the feature space. However, separately learning cluster in the target domain without regularization from source domain does not guarantee improved adaptation (28). To overcome this challenge, we identify clusters in both the source and target domains through a mixture of Gaussians with each component Gaussian corresponding to one category. Provided with the category- -wise statistics from source domain as anchors, we match the target domain clusters to the anchors by minimizing the KL-Divergence as the training objective for sTTT. Therefore, we name the proposed method test-time anchored clustering (TTAC). Since test samples are sequentially streamed, we develop an exponential moving averaging strategy to update the target domain cluster statistics to allow gradient- based optimization. ii) Each component Gaussian in the target domain updated by the test sample features that are assigned to the corresponding category. Thus, incorrect assignments (pseudo labels) will harm the estimation of component Gaussian. To tackle this issue, we are inspired by the correlation between network's is stability and confidence and pseudo label accuracy 17 24], and propose to filter out potentially incorrect pseudo labels. Component Gaussians are then updated by the samples that have passed the filtering. To exploit the filtered out samples. we incorporate global feature alignment [20) objective. We also demonstrate TTAC is compatible with existing TTT techniques, e.g. contrastive learning branch [20]. if source training loss is allowed to be modified. The contributions of this work are summarized as below.
In light of the confusions within TTT works, we provide a categorization of TTT protocols by two key factors. Comparison of TTT methods is now fair within each category.
We adopt a realistic TTT setting, namely STTT. To improve test-time feature learning, we propose TTAC by matching the statistics of the target clusters to the source ones. The target statistics are updated through moving averaging with filtered pseudo labels.
The proposed method is complementary to existing TTT method and is demonstrated on six TTT datasets, achieving the state-of-the-ar performance under all categories of TTT protocols.
2 Related Work
Unsupervised Domain Adaptation. Domain adaptation aims to improve model generalization when source and target data are not drawn i.i.d. When target data are unlabeled, unsupervised domain adaptation (UDA) 5 29 learns domain invariant feature representations on both source and target domains to improve generalization. Follow-up works improve UDA by minimizing a divergence 66 25 38), adversarial training [[] or discovering cluster structures in the target data (28).
2

---

Apart from formulating UDA as a task-specific model, re-weighting has been adopted for domain adaptation by selectively up-weighting conducive samples in the source domain 14 36.. During model training, the existing approaches often require access to the source domain data which, however, may be not accessible due to privacy issues, storage overhead, etc. Therefore, deploying UDA in more realistic scenarios has inspired research into source-free domain adaptation and test-time training/adaptation..
Source-Free Domain Adaptation. Without the access to source data, source-free domain adaptation
(SFDA) develops domain adaptation through self-training 19 16 self-supervised training [20].
clustering in the target domain [37] and feature restoration 4 It has been demonstrated that SFDA performs well on seminal domain adaptation datasets even compared against UDA methods [28].
Nevertheless, SFDA requires access to all testing data beforehand and model training must be carried out iteratively on the testing data. In a more realistic DA scenario where inference and adaptation must be implemented simuftaneously, SFDA will no longer be effective. Moreover, some statistical information on the source domain does not pose privacy issues and can be exploited to further improve adaptation on target data.
Test-Time Training. Collecting enough samples from target domain and adapt models in an offline manner restricts the application to adapting to static target domain. To allow fast and online adaptation, test-time training (TTT) 27 33 or adaptation (TTA) [3 emerges. Despite many recent works claiming to be test-time training, we notice severe confusion over the definition of TTT. In particular, whether training objective must be modified (27 20 and whether sequential inference on
target domain data is possible [1 13 Therefore, to reflect the key challenges in TTT, we define a setting called sequential test-time training (sTTT) which neither modifies the training objective nor violates sequential inference. Under the more clear definition, some existing works, e.g. TTT [27] and TTT++ (20) is more likely to be categorized into SFDA. Several existing works [3] can be adapted to the sTTT protocol. Tent 31 proposed adjust affine parameters in the batchnorm layers to adapt to target domain data. Nevertheless, updating only a fraction of model weights inevitably leads to limited performance gain on the target domain. T3A [13] further proposed to update classifier prototype through pseudo labeling. Despite being efficient, updating classifier prototype alone does not affect feature representation for the target domain. Target feature may not form clusters at all when the distribution mismatch between source and target is large enough. In this work we propose to simultaneously cluster the target domain and match target clusters to source domain classes, namely anchored clustering. To further constrain feature update, we introduce additional global feature alignment and pseudo label filtering. Through the introduced anchored clustering, achieve test-time training of more network parameters and achieve the state-of-the-ar performance.
3 Methodology
In this section we first introduce the anchored clustering objective for test-time training through pseudo labeling and then describe an efficient iterative updating strategy. An overview of the proposed pipeline is illustrated Fig.
3.1 Anchored Clustering for Test-Time Training
Discovering cluster structures in the target domain has been demonstrated effective for unsupervised domain adaptation (28) and we develop an anchored clustering on the test data alone. We first use mixture of Gaussians to model the clusters in the target domain, here each component Gaussian represents one discovered cluster. We further use the distributions of each category in the source domain anchors for the target distribution to match against. In this way, test data features can simultaneously form clusters and the clusters are associated with source domain categories, resulting in improved generalization to target domain. Formally, we first write the mixture of Gaussians in the source and target domains Pa (z) = Σn (x.Ne Eek), Pr(z) = E, 3 (Ho,, Σxx), where {#k E Rdd Σ₁ E Rdxd] represent one cluster in the source/target domain and d is the dimension of feature embedding.
Anchored clustering can be achieved by matching the above two distributions and one may directly minimize the KL-Divergence between the two distribution. Nevertheless, this is non-trivial because the KL.-Divergence between two mixture of Gaussians has no closed-form solution which prohibits efficient -rasee optimization. Despite some approximations exist [10], without knowing the
3

---

Source Domain (oftline)
Streeming Testing Data
Pseudo Label Filter (P.LF))
f()
Pugh
Anchored Clustering
fieratively
updata
Fixed-Length Queue
&& preudo tabel
Hipr kan)
Semporal changing direction
Ittrrttivaly update
PO
h()
P.L.F
Steratively update
Frozen
NOV.3D
NG4 35)
Stol-
Kiga
Lae
Anchor. Cluster.
K06.30
Nom
Leaa
Align
Nist
Figure I: Overview of TTAC pipeline. i) In the source domain, we calculate category- and global statistics as anchors. ii) In the testing stage, samples are sequentially streamed and pushed into fixed-length queue. Clusters in target domain are identified through anchored clustering with pseudo label filtering. Target clusters are then matched to the anchors in source domain achieve test-time training.
semantic labels for each Gaussian component, even good match between two mixture Gaussians does not guarantee target clusters aligned to the correct source ones and this will severely harm the performance test-time training. In light these challenges, we propose a category- alignment. Specifically, we allocate the same number of clusters in both source and target domains and each target cluster is assigned to one source cluster. We can then minimize the KL-Divergence between each pair of clusters in Eq.
Lac DDI(WN(Ho Ext) E(k))
HHH((( Exr)) + H(N( Esk), Exk))
The KL-Divergence can be further decomposed into the entropy H (Makk Esk)) and cross-entropy H (N (Hak, 2, (P(k, ))))). )) It is commonly true that the source reference distribution (x) is fixed thus the entropy term a constant C and only the cross-entropy the closed-form solution to the KL-Divergence between two Gaussian distributions, we now write the anchored clustering objective as,
is be optimized. Given
Lac E(log 2T + (111 - Mss) "x (piR - (ssk)) + (r( [xs) +
(2)
The source cluster parameters can be estimated in an offline manner. These information will not cause any privacy leakage and only introduces small computation and storage overheads. In the next section, we elaborate clustering in the target domain.
3.2 Clustering through Pseudo Labeling
In order test-time train network with anchored clustering loss, one obtain target cluster parameters (Mac, Fe J. For a minibatch of target test samples B = } J at timestamp 4, we first denote the predicted posterior as pt so softmau tfftaaaahi (h( E [0. 1 where softmar(-). f(()) and f(-) respectively denote a standard softmax function, the classifier head and backbone network. The pseudo labels are obtained via 91 arg max, P Given the predicted pseudo labels we could estimate the mean and covariance for each component Gaussian with the pseudo labeled testing samples. However, pseudo labels are always subject to model's discrimination ability. The eтrоr rate for pseudo labels is often high when the domain shift between source and target is large, directly updating the component Gaussian subject to erroneous pseudo labels, a.k.a. confirmation bias D. To reduce the impact of incorrect pseudo labels, we first adopt a light-weight temporal consistency (TC) pseudo label filtering approach. Compared to co-teaching [ meta-learning [18] based methods, this light- method does not introduce additional computation overhead and

---

is therefore more suitable for test-time training. Specifically, to alleviate the impact from the noisy predictions, calculate the temporal exponential moving averaging posteriors p [0. below
1|N* as
p = (1- P. +
s.t.
P P
(3)
The temporal consistency filtering is realized as in Eq. 4 where TIC threshold determining the maximally allowed difference in the most probable prediction over time. If the posterior deviate from historical value too much, will be excluded from target domain clustering.
FTC ((P) P TTC), s.t. = max &
(4)
Due to the sequential inference, test samples without enough historical predictions may still pass the TC filtering. So, we further introduce an additional pseudo label filter directly based on the posterior probability as,
E.P 1P > TPP)
(5)
By filtering out potential incorrect pseudo labels, we update the component Gaussian only with the leftover target samples as below.
Σ FFTO
PPPIPIP 1 (0: k)f(2))
1 (1:- = k))((,))) (=x))(((x)) - pek)
Mekk
ΣF F.P 10, = k))
Ee
Σ EFTCEPP1(0-
3.3 Global Feature Alignment
As discussed above, test samples that do not pass the filtering will not contribute to the estimation of target clusters. Hence, anchored clustering may not reach its full potential without the filtered test samples. To exploit all available test samples, we propose to align global target data distribution to the source one. We define the global feature distribution of the source data as ÿs (x) A (Ms: Eg) and the target data as (I) = N (14, Et). To align two distributions, we again minimize the KL--Divergence as,
Lyn D DRL (x)(()
Similar idea has appeared in 20 which directly matches the moments between source and target domains 33 by minimizing the F-norm for the mean and covariance, i.e. || = + +|E, - Σ, I However, designed for matching complex distributions represented drawn samples, central moment discrepancy (38) requires summing infinite central moment discrepancies and the ratios between different order moments are hard to estimate. For matching two parameterized Gaussian distributions KL-Divergence is more convenient with good explamation from a probabilistic point of view. Finally, we add small constant to the diagonal of Σ for both source and target domains increase the condition number for better numerical stability.
3.4 Efficient Iterative Updating
Despite the distribution for source data can be trivially estimated from all available training data in a totally offline manner, estimating the distribution for target domain data is not equally trivial, in particular under the sTTT protocol. In related research 20), dynamic queue of test data features are preserved to dynamically estimate the statistics, which will introduce additional memory footprint [20]. To alleviate the memory cost we propose iteratively update the running statistics for Gaussian distribution. Formally, we define t-th test minibatch as B = {z:)i=I-..No- Denoting the running mean and covariance at step as je and I', we present the rules to update the mean and covariance in Eq. g More detailed derivations and update rules for per cluster statistics are deferred the Appendix.
1d = p1-1 +50 E'=
+ a' Σ ((f(z)) (T-1))((((a)) (f(xz) 11-1)- - -E--1- - 5t 8r
2.E6
(8)
1
8º = a' Σ(f()) -
N = N + 18%, a' = No
5

---

Additionally, N grows larger overtime. New test samples will have smaller contribution to the update of target domain statistics when N is large enough. As result, the gradient calculated from current minibatch will vanish. To alleviate this issue, we impose a clip on the value of a' as below. As such, the gradient maintain minimal scale even if N is very large.
N < Netip others
a'
NO Nelp
(9)
3.5 TTAC Training Algorithm
We summarize the training algorithm for the TTAC in Algo. For effective clustering in target domain, we allocate fixed length memory space, denoted as Ne x3 to store the recent testing samples. In the sTTT protocol, we first make instant prediction on each testing sample, and only update the model when NB testing samples are accumulated. TTAC can be efficiently implemented, e.g. with two devices, one is for continuous inference and another is for model updating.
Algorithm 1: Test- Time Anchored Clustering Training Algorithm
input :A new testing sample batch B {=i]t=1..NN.
# Update the testing sample queue C. C = C Bl-NN//Nn, C = C* UB
for 1 to Nitr do
for minibatch (x k= in C do
# Obtain the predicted posterior and pseudo labels
P. = softmar (ootmarrh((f(a)) (x()), if = arg maxx({))
# Calculate the global and per-cluster running mean and covariance by Eq.
n², Ex {u (2%)
# Optimize the combined loss by Eq. and Eq
L= Lac Xbya
update network f to minimize C
4 Experiment
In this section, we first compare various existing methods based on the two key factors. Evaluation is then carried out on six test-time training datasets. We then ablate the components of TTAC. Further analysis on the cumulative performance, qualitative insights, etc. are provided at the end.
4.1 Datasets
We evaluate on 6 test-time training datasets and report the classification error rate (%) throughout the experiment section. To evaluate the test-time training efficacy on corrupted target images, we use CIFARIO-C/CIFARI COCCCAIOO--CCCCCCCC ( each consisting of 10/100 classes with 50,000 training samples of clean data and 10,000 corrupted test samples. We further evaluate test-time training on hard target domain samples with CIFARIO.1 23), which contains around 2,000 difficult testing images sampled over years of research the original CIFAR-10 dataset. To demonstrate the ability to do test-time training for synthetic data to real data transfer we further use VisDA-C 22.. which is challenging large-scale synthetic-to-reat object classification dataset, consisting of 12 classes, 152,397 synthetic training images and 55,388 real testing images. To evaluate large-scale test-time training, we use ImageNet-C [9] which consists of 1,000 classes and 15 types of corruptions on the 50,000 testing samples. Finally, to evaluate test-time training on 3D point cloud data, we choose ModelNet40-C [26). which consists of 15 common and realistic corruptions of point cloud data, with 9,843 training samples and 2,468 test samples.
4.2 Experiment Settings
Hyperparameters. We use the ResNet-50 8 for image datasets and the DGCNN 34
ModelNet40-C. We optimize the backbone network f(-) by SGD with momentum on all datasets. On CIFARIO-C/CIFAN 100-C and 1IFAR10.1, 0.0001, 0.01 respectively. On VisDA-C we use BS = 128 and LRR = 0.0001, and on ModelNet40-C we use BS 64 and LR 0.001. More details of hyperparameters
use (batchsize) BS 256 and (learning rate) LR = 0.01,
be found in the Appendix.
Test- Time Training Protocols. We categorize test-time training based on two key factors. First, whether the training objective must be changed during training the source domain, use Y and N

---

to indicate if training objective is allowed to be changed or not respectively. Second, whether testing data is sequentially streamed and predicted, we use O to indicate a sequential One-pass inference and M to indicate non-sequential inference, a.k.a. Multi-pass inference. With the above criteria, we summarize 4 test-time training protocols, namely N-O, Y-O, N-M and Y-M, and the strength of the assumption increases from the first to the last protocols. Our STTT setting makes the weakest assumption, i.e. N-O. Existing methods are categorized by the four TTT protocols, we notice that some methods can operate under multiple protocols
Competing Methods. We compare the following test-time training methods. Direct testing (TEST) without adaptation simply do inference on target domain with source domain model. Test-time training (TTT-R) [27] jointly trains the rotation-based self-supervised task and the classification task in the source domain, and then only train the rotation- rotation-based self-supervised task in the streaming test samples and make the predictions instantly. The default method is classified into the Y-M protocol. Test-time normalization (BN) 12] moving average updates the batch normalization statistics by streamed data. The default method follows N-M protocol and can be adapted to N-O protocol. Test-time entropy minimization (TENT) 31 updates the parameters of all batch normalization by minimizing the entropy of the model predictions in the streaming data. By default, TENT follows the N-O protocol and can be adapted to N-M protocol. Test-time classifier adjustment (T3A) [ computes target prototype representation for each category using streamed data and make predictions with updated prototypes. T3A follows the N-O protocol by default. Source Hypothesis Transfer (SHOT) [99] freezes the linear classification head and trains the target-specific feature extraction module by exploiting balanced category assumption and self--upervised pseudo-labeling in the target domain. SHOT by default follows the N-M protocol and we adapt it to N-O protocol. TTT++ [20] aligns source domain feature distribution, whose statistics are calculated offline, and target domain feature distribution by minimizing the F-norm between the mean covariance. TTT++ follows the Y-M protocol and we adapt it to N-O (removing contrastive learning branch) and Y-O protocols. Finally, we present our own approach, TTAC, which only requires a single pass on the target domain and does not have to modify the source training objective. We further modify TTAC for Y-O, N-M and Y-M protocols, for Y-O and Y-M we incorporate an additional contrastive learning branch [20]. We could further combine TTAC with additional diversity loss and entropy minimization loss introduced in SHOT [19, denoted as TTAC+SHOT.
4.3 Test-Time Training on Corrupted Target Domain
We present the test-time training results CIFAR and ModelNet40-C datasets in Tab. and the results on ImageNet-C dataset in Tab. 2
We make the following observations from the results.
STTT (N-O) Protocol. We first analyze the results under the proposed sTTT (N-O) protocol. Our method outperforms all competing ones by a large margin. For example, 3% improvement is observed on both CIFARIO- and CIFARION 100-C from the previous best (TTT+)) and 5-13% improvement is observed on ImageNet-C compared with BN and TENT, and TTAC is superior in average accuracy and outperforms out of 15 types of corruptions compared with SHOT on ImageNet-C. We further combine TTAC with the class balance assumption made in SHOT (TTAC+SHOT), With the stronger assumptions out method can further improve upon TTAC alone, in particular on ModelNet4O-C dataset. This result demonstrates TTAC's compatibility with existing methods.
Alternative Protocols. We further compare different methods under N-M, Y-O and Y-M protocols. Under the Y-O protocol, TTT++ (20) modifies the source domain training objective by incorporating contrastive learning branch [3]. To compare with TTT++, we also include the contrastive branch and
observe a clear improvement on both CIFARIO-C and CIFAR 100-C datasets. More TTT methods can be adapted to the N-M protocol which allows training the whole target domain data multiple epochs. Specifically, we compared with BN, TENT and SHOT. With TTAC alone we observe substantial improvement all three datasets and TTAC can be further combined with SHOT demonstrating additional improvement. Finally, under the Y-M protocol, we demonstrate very strong performance compared to TTT-R and TTT++. It is also worth noting that TTAC under the N-O protocol can already yield results close to TTT++ under the Y-M protocol, suggesting the strong test-time training ability of TTAC even under the most challenging TTT protocol.
4.4 Additional Datasets
TTT on Hard Samples. CIFAR1O.1 contains roughly 2,000 new test images that were re-sampled after the research original CIFAR-10 dataset, which consists of some hard samples and reflects the
7

---

Table 1: Comparison under different TTT protocols. Y/N indicates modifying source domain training objective or not. O/M indicate one pass or multiple passes test-time training. C10-C, C100-C and MN40-C refer to CIFARIO-C, CIFAR100-C and ModelNer4O-C datasets respectively. All numbers indicate error rate in percentage.
Method TEST
TTT Protocol
Assum. Strength
C10-C
C100-C
MN40-C
29.15
60.34
34.62
BN 12
N-O
Weak
15.49
43.38
26.53
TENT 31
N-O
Weak
14.27
40.72
26.38
A3A 13
N-O
Weak
15.44
42.72
24.57
SHOT 19
N-O
Weak
13.95
39.10
19.71
TTT+
N-O
Weak
13.69
40.32
TTAC (Ours)
N-O
Weak
10.94
36.64
22.30
TTAC+SHOT (Ours)
N-O
Weak
10,99
36,39
19.21
TTT++ 20
Y-O
Medium
13.00
35.23
TTAC (Ours)
Y-O
Medium
10.69
34.82
BN 12
N-M
Medium
15.70
43.30
26.49
TENT 31
N-M
Medium
12,60
36.30
21.23
SHOT 19
N-M
Medium
14.70
38.10
15.99
TTAC (Our))
N-M
Medium
9.42
33.55
16.77
TTAC+SHOT (Ours)
N-M
Medium
9.54
32.89
15.04
TTT-R 27
Y-M
Strong
14.30
40.40
TTT+ 20
Y-M
Strong
9.80
34.10
TTAC (Ours)
Y-M
Strong
8.52
30.57
Table 2: Test-time training ImageNet-C under the sTTT (N-O) protocol.
Method
Birt
Coetr
Defoc
Elast
Fog
Frost
Gauss
Glass
Impul
Opeg
Moin
Pixel
Shot
Snow
Zoom
Avg
TEST
38.82
89.55
82.23
87.13
64.84
76.83
97.34
90.50
97.76
68.31
83.60
50.37
96.74
82.22
74.31
80.70
BN (N-O)
32.33
50.93
81.28
52.98
42.21
64.13
83.25
83.64
82.52
59.18
66.23
19.45
82.59
62.34
52.51
63.04
49.23
SHOT (N-O)
30.69
37.69
61.97
41.30
34.74
54.19
76.33
71.94
74.24
46.50
47.98
38.88
70.60
46.09
40.74
51.59
TTAC (N-O)
30.36
38.84
69.06
39.67
36.01
50.20
66.18
70.17
64.36
45.59
51.77
19.72
62.43
44.56
42.80
50.11
normal domain shift in our life. The results in Table. demonstrate our method is better able to adapt to the normal domain shift.
TTT on Synthetic to Real Adaptation. VisDA-C is a large-scale benchmark of synthetic-to-ren object classification dataset. The setting training on synthetic dataset and testing on real data fits well with the real application scenario. On this dataset, we conduct experiments with our method under the N-O, Y-O and Y-M protocols and other methods under respective protocols, results are presented in Table. 5 We make the following observations. First, our method (TTAC Y-O) outperforms all methods except TTT++ under the Y-M protocol. This suggests TTAC is able be deployed in the realistic time training protocol. Moreover, if training on the whole target data is allowed, TTAC (Y-M) further beats TTT+ by large margin, suggesting the effectiveness TTAC under wide range of TTT protocols.
Table 3: Test-time training on CIFARIO.1.
Table 4: Source-free STTT on CIFARIO-C
TEST
BN
TTT-R
TENT
SHOT
TIT++
TTAC
TEST
BN
TENT
T3A
SHOT
TTAC
TTACKSHOT
29.15
15.49
14.27
15.44
13.95
13.74
13.35
4.5
Ablation Study
We conduct ablation study CIFAR10-C dataset for individual components, including anchored clus- tering, pseudo label filtering, global feature alignment and finally the compatibility with contrastive branch [20]. For anchored clustering alone, we use all testing samples to update cluster statistics. For pseudo label filtering alone, we implement as predicting pseudo labels followed by filtering, then pseudo labels are used for self-training. We make the following observations from Tab. O Under both N-O and N-M protocols, introducing anchored clustering pseudo label filtering a alone improves over the baseline, e.g. under N-O 29.15% > 14.32% for anchored clustering and 29.15% - 15.00% for pseudo label filtering. When anchored clustering is combined with pseudo label filtering, we observe a significant boost in performance. This is due to more accurate estimation category-wise cluster in the target domain and this reflects matching directly in the feature space may be better than minimizing cross-entropy with pseudo labels. We further evaluate aligning global features alone with KL-Divergence. This achieves relatively good performance and obviously outperforms the L2

---

Table Test-time training on VisDA. The numbers for competing methods are inherited from 20
Method
Plane
Beyel
Bus
Car
Horse
Knife
Meyel
Person
Plant
Sktbrd
Train
Truck
Per-olass
TEST
56.52
88.71
62.77
30.56
81.88
99.03
17.53
95.85
51.66
77.86
20.44
99.51
65.19
BN (N-M)
44.38
$6.98
33.24
55
37.45
66.60
16.55
59.02
43.55
60.72
31.07
82.98
48.99
TENT (N-M)
31
13.43
77.98
20.17
48.15
21.72
82.45
12.37
35.78
21.06
76.41
34.11
98.93
45.21
SHOT (N-M)
5.73
13.64
23.33
42.69
7,93
86.99
19.17
11.63
11.09
15.06
43.26
25.04
TFA (N-M)
20
28.25
32.03
33.67
64.77
20.49
56.63
22.52
36.30
24.84
35.20
25.31
64.24
37.02
TTT+ (Y-M)
20
4.13
26.20
21 21.50
31.70
7.43
83.30
7.83
21.10
7.03
7.73
6.91
51.40
23.03
TTAC (N-O)
18.54
40.20
35
63.11
23.83
39.61
15.51
41.35
22.97
46.56
25.24
67.81
36,71
TTAC (Y-O)
7.19
29.
22.52
56.58
8.14
18.41
8.25
22.28
10.18
23.98
13.55
67.00
24.01
TTAC (Y-M)
2.74
17.73
18.91
43.12
5.54
12.24
4.66
15.90
4.77
10.78
9.75
62.45
17.38
GIFAR1S
CIFRR100
B4-0--0
(v-d)
TTT+
OURS (44)
OLR5 (-:1
Trll Sapley
Test-time cumulative
(b) TTT+ Feature
(c) TTAC Feature
Figure 2: (a) Comparison of test-time cumulattve error under one-pass protocol. (b) T-SNE visualiza- tion of TTT++ feature embedding. (c) T-SNE visualization of TTAC feature embedding.
distance alignment adopted in (20). Finally, we combine all three components and the full model yields the best performance. When contrast learning branch is included, TTAC achieves even better results.
Table 6: Ablation study for individual components CIFARIO-C dataset.
TTT Protocol
N.O
Y-O
N-M
Y-M
Anchored Cluster.
Pseudo Lahel Filter.
Fear, Align
KLD
Dist.
KLD
Contrust. Branch 20
Aug
29.15
14.32
10.01
8.52
4.6 Additional Analysis
Cumulative performance under STTT. We illustrate the cumulative error under the STTT protocol in Fig. 2 (a). For both datasets TTAC outperforms competing methods from the early stage of test-time training. The advantage is consistent throughout the TTT procedure.
TSNE Visualization of TTAC features. We provide qualitative results for test-time training by visualizing the adapted features through T-SNE [30]. In Fig. 2 (b) and Fig. (c), we compared the features learned by TTT++ 20 and TTAC (Ours). We observe a better separation between classes by TTAC, implying improved classification accuracy.
Source-Free Test-Time Training. TTT aims adapt model to target domain data by doing simulta- neous training and sequential inference. It has been demonstrated some light- information, e.g. statistics, from source domain will greatly improve the efficacy TTT. Nevertheless, under more strict scenario where source domain information is strictly blind, TTAC can still exploit classifier prototypes to facilitate anchored clustering. Specifically, we normalize the category- weight vector with the norm of corresponding target domain cluster center prototypes. Then, build source domain mixture of Gaussians by taking prototypes as mean with a fixed covariance matrix. The results CIFAR 10-A presented in Tab. It is clear that even without any statistical information from source domain, TTAC still outperforms all competing methods.
Test Sample Queue and Update Epochs. Under the sTTT protocol, we allow all competing methods to maintain the same test sample queue and multiple update epochs on the queue. analyse the significance of the sample queue and update epochs, we evaluate BN, TENT, SHOT and TTAC on CIFAR 10-C and ImageNet-C level 5 snow corruption evaluation set under different number of update epochs on test sample queue and under without queue protocol, i.c. only update model w.r.t.

---

the current test sample batch. As the results presented in Tab. / we make the following observations. i) Maintaining a sample queue can substantially improve the performance of methods that estimate target distribution, e.g. TTAC (11.91 + 10.88 on CIFAR 10-C) and SHOT (15.18 - 13.96 on CIFARIO-C). This is due to more test samples giving better estimation of true distribution. ii) Consistent improvement can be observed with increasing update epochs for SHOT and TTAC. We ascribe this to iterative pseudo labeling benefiting from more update epochs.
Table 7: Comparing with and without test sample queue and different numbers of model update epochs. w/ Queue maintains test sample queue with 4096 samples; w/o Queue maintains a single mini-batch with 256 and 128 samples on CIFARIO-C and ImageNet-C respectivvll
CIFARIO-C
ImageNet-C
Queue
w/o Queue
w/ Queue
wlo Queuc
#Epochs
3
4
2
BN
15.84
15.99
16.04
16.00
15.44
62.34
62.34
62.59
TENT
13.35
13.83
13.85
13.87
13.48
47.82
49.23
48.39
SHOT
13.96
13.93
13.83
13.75
15.18
46.91
51.46
TTAC
10.88
10.80
10.58
9.96
11.91
45.44
44.56
46.64
Computation Cost Measured in Wall-Clock Time. Test sample queue and multiple update epochs introduce additional computation overhead. To investigate the impact on efficiency, we measure the overall wall time the time elapsed from the beginning to the end of test-time training, including all I/O overheads. The per-sample wall time is then calculated as the overall wall time divided by the number of test samples. We report the per-sample wall time (in seconds) for BN, TENT, SHOT and TTAC in Tab. [ Junde different update epoch settings and without queue setting. The Inference row indicates the per-sample wall time in a single forward pass including the data I/O overhead. We observe that, under the same experiment setting, BN and TENT are more computational efficient, but TTAC only twice more expensive than BN and TENT if test sample queue preserved (0.0083 v.s. 0,0030/0.004| (11.91 v.s. 13.48). In summary, TTAC is able to strike balance between computation efficiency and performance depending on how much computation resource is available. This suggests allocating separate device is only necessary when securing best performance is the priority.
while the performance of TTAC w/o queue is still better than TENT
Table 8: The par-sample wall time (measured in seconds) on CIFARIO-C under STTT protocol.
w/ Queue
w/o Queue
#Epochs
3
BN
0.0136
0.0220
0.0293
0.0362
0.0030
TENT
0.0269
0.0399
0.0537
0.0663
0.0041
SHOT
0.0479
0.0709
0.0942
0.1183
0.0067
TTAC
0.0516
0.0822
0.1233
0.1524
0.0083
Inference
0.0030
0.0030
0.0030
0.0030
0.0030
5 Conclusion
Test-time training (TTT) tackles the realistic challenges of deploying domain adaptation on-the-fly. In this work, we are first motivated by the confused evaluation protocols for TTT and propose two key criteria, namely modifying source training objective and sequential inference, to further categorize existing methods into four TTT protocols. Under the most realistic protocol, i.e. sequential test-time training (sTTT), we develop a test-time anchored clustering (TTAC) approach to align target domain features to the source ones. Unlike batchnorm and classifier prototype updates, anchored clustering allows all network parameters to be trainable, thus demonstrating stronger test-time training ability. We further propose pseudo label filtering and an iterative update method to improve anchored clustering and save memory footprint respectively. Experiments on six datasets verified the effectiveness of TTAC under STTT well other TTT protocols
Acknowledgement This work was supported part by the National Natural Science Foun- dation of China (NSFC) under Grant 62106078, Guangdong RD key project of China (No.: 20198010155001), and the Program for Guangdong Introducing Innovative and Enterpreneurial Teams (No.: 20172T07X183).
10

---

References
[1] Eric Arazo, Diego Ortego, Paul Albert, Noel E O'Connor, and Kevin McGuinness. Pseudo-labeling and confirmation bias deep semi-supervised learning. International Joint Conference on Neural Networks. 2020.
[2] Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman Vaughan. A theory of learning from different domains, Machine learning, 2010.
[3] Ting Chen, Simon Komblith. Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference machine learning, 2020.
[4] Cian Eastwood, Ian Mason, Chris Williams, and Bernhard Schölkopf. Source- adaptation measure- ment shift via bottom-up up feature restoration. International Conference on Learning Representations. 2022.
[5] Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. Interna- tional conference on machine learning, 2015.
[6] Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test. The Journal of Machine Learning Research, 2012.
[7] Bo Han, Quamming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. Co-teaching: Robust training of deep neural networks with extremely noisy labels. In Advances in neural information processing systems, 2018.
[8] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2016.
[9] Dan Hendrycks and Thomas Dictterich. Benchmarking neural network robustness to common corruptions and perturbations. In International Conference on Learning Representations, 2019.
[10] John R Hershey and Peder A Olsen. Approximating the kullback leibler divergence between gaussian mixture models. In IEEE International Conference on Acoustics, Speech and Signal Processing, 2007.
[11] Judy Hoffman, Fric Tzeng, Tacsung Park, Jun-Yan Zhu, Phillip Isola, Kate Saenko, Alexei Efros, and Trevor Darrell. Cycada: Cycle-consistent adversarial domain adaptation. In International conference on machine learning, 2018.
[12] Sergey loffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference machine learning, 2015.
[13] Yusuke Iwasawa and Yutaka Matsuo. Test-time classifier adjustment module for model-agnostic domain generalization. In Advances in Neural Information Procesxing Systems, 2021.
[14] Jing Jiang and ChengXiang Zhai. Instance weighting for domain adaptation in nlp. In ACL, 2007.
[15] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 2012.
[16] Jogendra Nath Kundu, Naveen Venkat, R Venkatesh Babu, et al. Universal source-free free domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, 2020.
[17] Dong-Hyun Lee et al. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on challenges in representation learning, 1CML, 2013.
[18] Junnan Li, Yongkang Wong, Qi Zhao, and Mohan S Kankanhalli. Learning to learn from noisy labeled data. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, 2019.
[19] Jian Liang, Dapeng Hu, and Jiashi Feng. Do we really need to access the source data? Source hypothesis transfer for unsupervised domain adaptation. In International Conference on Machine Learning, 2020.
[20] Yuejiang Liu, Parth Kothari, Bastien van Delfi, Baptiste Bellot-Gurlet, Taylor Mordan, and Alexandre Alahi. Tut++: When does supervied test-time training fail or thrive? In Advances in Neural Information Processing Systems, 2021.
[21] Xingchao Peng, Ben Usman, Neela Kaushik, Judy Hoffiman, Dequan Wang. and Kate Saenko, Visda: The visual domain adaptation challenge, arXiv preprint arXiv:1710.06924. 2017.
11

---

[22] Joaquin Quiioonro-Candela. Masashi Sugiyama, Anton Schwaighofer, and Neil D Lawrence. Dataset shift in machine learning. Mit Press, 2008.
[23] Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do imagenet classifiers generalize to imagenet? In International Conference on Machine Learning, 2019.
[24] Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin A Raffel, Ekin Dogus Cubuk, Alexey Kurakin, and Chun-Liang Li. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. Advances in Neural Information Processing Systems, 2020.
[25] Baochen Sun and Kate Saenko. Deep coral: Correlation alignment for deep domain adaptation. In European conference computer vision, 2016.
[26] Jiachen Sun, Qingzhao Zhang, Bhavya Kailkhura, Zhiding Yu. Chaowei Xiao, and Z Morley Mao. Benchmarking robustness of 3d point cloud recognition against common corruptions. arXiv preprint arXiv:2201. /2296, 2022.
[27] Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei Efros, and Moritz Hardt. Test-time training with superviin for generalization under distribution shifts. In International Conference on Machine Learning, 2020.
[28] Hui Tang, Ke Chen, and Kui Jia. Unsupervised domain adaptation via structurally regularized deep clustering. In Proceedings of IEEE/CVE conference on computer vision and pattern recognition, 2020.
[29] Eric Tzeng, Judy Hoffmann, Ning Zhang, Kate Saenko, and Trevor Darrell. Deep domain confusion: Maximizing for domain invariance. arXiv preprint 14X1::1412.347,, 2014.
[30] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 2008
[31] Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. In International Conference Learning Representations, 2021.
[32] Mci Wang and Weihong Deng. Deep visual domain adaptation: A survey. Neurocomputing, 2018.
[33] Qin Wang, Olga Fink. Luc Van Gool, and Dengxin Dai. Continual test-time domain adaptation. In Proceedings of IEEE/CVF International Conference on Computer Vision, 2022.
[34] Yue Wang, Yongbin Sun, Ziwci Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon. Dynamic graph can for learning point clouds. Acm Transactions On Graphics (tog), 2019.
[35] Haifeng Xia, Handong Zhao, and Zhengming Ding. Adaptive adversarial network source- free domain adaptation. In Proceedings of the IEEE/CVE International Conference on Computer Vision, 2021.
[36] Hongliang Yan, Yukang Ding, Peihua Li, Qilong Wang, Yong Xu, and Wangmeng Zuo. Mind the class weight bias: Weighted maximum mean discrepancy for unsupervised domain adaptation. In Proceedings of IEEE conference on computer vision and pattern recognition, 2017.
[37] Shiqi Yang, Yaxing Wang, Joost van de Weijer, Luis Herranz, and Shangling Jui. Generalized source-free domain adaptation. In Proceedings of the IEEE/CVF International Conference Computer Vision, 2021.
[38] Werner Zellinger, Thomas Grubinger, Edwin Lughofer, Thomas Natschläger, and Susanne Saminger- Platz. Central moment discrepancy (cmd) for -omain-invariant representation learning. In International Conference on Learning Representations, 2016.
[39] Kaiyang Zhou, Ziwei Liu, Yu Qiao, Tao Xiang, and Chen Change Loy. Domain generalization: A survey. arXiv e-prints, pages Xivv 2021.
[40] Zhi-Hua Zhou. A brief introduction to weakly supervised learning. National science review, 2018
12

---

Checklist
The checklist follows the references. Please read the checklist guidelines carefully for information on how to answer these questions. For each question, change the default [TODO] to [Yes], [No], or [N/A] . You are strongly encouraged to include justification your answer, either by referencing the appropriate section of your paper or providing a brief inline description. For example:
Did you include the license the code and datasets? [Yes] See Section ??.
Did you include the license to the code and datasets? [No] The code and the data are proprictary.
Did you include the license to the code and datasets? [N/]]
Please do not modify the questions and only use the provided macros for your answers. Note that the Checklist section does not count towards the page limit. In your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/answers below.
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations your work? [Yes] We have discussed the assump- tions and evaluation protocols explicitly.
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)? [Yes]] We release the code GitHub repository, whose URL is provided at the abstract part.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] Please see the experiment section and Appendix.
(c) Did you report error bars (e.g.. with respect to the random seed after running experi- ments multiple times)? [N/A] The randomness negligible.
(d) Did you include the total amount of compute and the type of resources used (e.g.. type of GPUs, internal cluster, or cloud provider)? [Yes] Please see the Appendix.
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) your work uses existing assets, did you cite the creators? [Yes] We have properly cited all datasets used and works we compared against.
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material or URL? [No]
No additional data is involved.
(d) Did you discuss whether and how consent was obtained from people whose data you'l 're using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content? [N/A] No personally identifiable information used.
5. If you used crowdsoureing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid participants and the total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "vQVSK7su7iE", "reviewer": "Reviewer_6icX", "summary": "Test-time training is an emerging approach for building robust machine learning models to distribute shifts. The main contribution of this paper summarizes the following two points. (1) They distinguish experiment setups, from the perspective of whether it uses on-pass adaptation or multi-pass adaptation and whether it needs to alter the training phase.  The distinction is important as prior works often compared different setups, leading to unfair comparisons. (2)  They provide a new method on realistic test-time training setup, called test-time anchored clustering. Specifically, they use a mixture of Gaussians to model the clusters in the target domain, and use pseudo-label to match it with the mixture distribution of source data in closed form. Empirical results show that the proposed method outperforms existing methods on several benchmark datasets. \n", "questions": "(1) Why do we need to represent each distribution by a mixture of Gaussian distributions? In other words, is it better than simply aligning the statistics (e.g, mean and covariance) of source data of a category and that of the target data predicted as the category?\n\n(2) How did you tune the hyper parameters for test-time training methods? \n\n(3) Why is SHOT categorized as N-O or N-M? In my understanding, the original SHOT contains several proposals including label-smoothing techniques for source training, and thus require special objective function for source training. Does the value reported here not use label-smoothing? \n\n(4) According to Table 5, major performance gains come from global feature alignment using KLD. In my understanding, several prior methods (e.g., TFA and TTT+) use similar feature alignment loss to match the statistics of source and target distribution. Does the results suggest KLD is better than other feature alignment approaches, or does there any difference between setup among these methods?\n", "limitations": "This paper does not provide limitation section in the main text. I recommend the author to discuss the potential limitation of the proposed method (e.g. hyper parameter tuning mentioned in weakness in my review), and potential drawback of test-time training itself (e.g., do we really want to adapt the model during test-time in the safety critical application?). ", "rating": 6, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "---\n**Strengths**\n\n(1) The paper is well written and easy to follow. \n\n(2)  They pointed out important confusion about evaluation in the fields of test-time training. I strongly agree that the confusion often leads to unfair comparisons, and the clarification of this paper makes it easier for subsequent research to conduct fairer comparisons, which I think very important for scientific progress.  \n\n(3) This paper provides extensive empirical validations, carefully investigating how each component of the proposal contributes to the performance gain. \n\n---\n**Weaknesses**\n\n(1) Why do we need to represent each distribution by a mixture of Gaussian distributions? In other words, is it better than simply aligning the statistics (e.g, mean and covariance) of source data of a category and that of the target data predicted as the category? The former approach has been already proposed in test-time adaptation recently [1], it is better to clarify the difference. \n\n[1] Kojima, Takeshi et al. “Robustifying Vision Transformer without Retraining from Scratch by Test-Time Class-Conditional Feature Alignment.” IJCAI2022\n\n(2) Comparing sTTA and other TTA setup, one notable difficulty of sTTA might be how to choose the hyperparameter. Because there is no clue about test-distribution when we choose the hyper parameter (e.g., learning rate, optimizer, tau for the threshold, etc) in sTTA, there is a reasonable way to determine the hyper parameter. From the current manuscript, unfortunately, how to tune the hyperparameter for each test-time training method is unclear for me, which makes me wonder whether the reported results are actually fair or not. \n\n(3) According to Table 5, major performance gains come from global feature alignment using KLD. In my understanding, several prior methods (e.g., TFA and TTT+) use similar feature alignment loss to match the statistics of source and target distribution. Does the results suggest KLD is better than other feature alignment approaches, or does there any difference between setup among these methods?\n\n(Minor comments)\n- line 124 needs a period. \n- line 131: should clarify what is u and sigma (d-dimensional mean vector and covariance matrix?)\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "nUeJrmeR5o-", "reviewer": "Reviewer_Af9y", "summary": "Test-time adaptation methods seek to improve generalization error from training on the source data to testing on the target data by updating the model on test data to mitigate shift between source and target.\nThis work re-examines the assumptions for this setting, and in particular identifies adapting _online_ in one pass and adapting _without_ altering the training loss as key considerations for practicality.\nMulti-pass adaptation requires too much computation, delaying prediction, and modifying training requires re-training on all of the source data.\nWhile arguing for these strict limits on the latency and the loss, this work relaxes the restriction to not use source data by allowing \"light-weight\" information (that is, statistics, clusters, etc.).\nThis variant of test-time training is named sequential test-time training (sTTT).\n\nThe test-time anchored clustering (TTAC) method is proposed to address this setting.\nThe anchored clustering is a joint clustering of the source and target, where the class-wise source clusters are the anchors, and the predicted test clusters are optimized to match the anchors in mean and variance by minimizing the KL divergence.\nBoth clusterings are computed as mixture-of-Gaussian distributions with one Gaussian per class.\nThe updates to the target clusters are regularized by exponential moving average (EMA) and stabilized by filtering out noisy predictions according to a queue of predictions on past test data (pseudo-label filtering).\nLastly, filtered out samples can still contribute to an update of the cluster statistics by global feature alignment.\n\nExperiments show that TTAC rivals or improves on test-time adaptation and source-free adaptation baselines.\nA variety of settings (one-pass/multi-pass and same/different training) and datasets (CIFAR-10/100-C, CIFAR-10.1, VisDA-C, and ModelNet-40-C) are evaluated.", "questions": "Questions\n\n- Why is ImageNet-C not evaluated? How does TTAC perform on it? It is a gold standard benchmark, and results on CIFAR-10/100-C do not always reflect results at larger scale.\n- How many updates are made when updating the network on the test input and sampling queue? Is there a single update per batch, as in TENT, or multiple updates like in TTT or MEMO?\n- How much computation is needed for TTAC? Could you measure it in number of forward passes/backward passes or wallclock time per test sample? In particular, what is the overhead of the queueing?\n\nOther Feedback\n\n- Figure 1 is complex! To make it more accessible and informative, consider taking certain points and moving them into their own figure. For instance, PLF could perhaps be explained on its own.\n- Note that SHOT has a journal edition in PAMI'21 with improved results. It is preferable to compare to the latest and strongest edition of the method.\n- Here are a few typos noticed during review: \"it is inevitable to access\" on line 58 could be \"still requires access to\".\n  On line 125 there is a missing period after \"alone\". On line 241 \"Ours sTTT\" should be \"Our sTTT\". There are more.\n", "limitations": "- There is no explicit discussion of limitations.\n- The computational overhead of queuing data for statistics and parameter updates needs acknowledgement and ideally measurement. TTAC is more expensive than BN or TENT.", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\n- This work helps explore a newer setting by emphasizing certain dimensions of the test-time adaptation problem, in particular (1) the use of source data, (2) online or one-pass vs. offline or multi-pass updates, and (3) altering training.\n  As a newer setting, these choices need to be interrogated. The choice of sTTA in this work is different than some but not all prior work (in particular, BUFR [A] works in the same setting).\n- The experiments explore a variety of settings (one-pass/multi-pass and same/different training) and datasets (CIFAR-10/100-C, CIFAR-10.1, VisDA-C, and ModelNet-40).\n  Across these settings and datasets TTAC does as well or better than common baselines for test-time adaptation and source-free adaptation.\n- The proposed TTAC method keeps improving more with additional data during testing compared to BN or T3A, which plateau at 20% of the test set.\n- The use of source data does not alter training. It still requires the full source data for fitting the anchored clusters of TTAC offline, but fitting these clusters is agnostic to architecture, training loss, etc.\n- Without source data, in the source-free adaptation setting, a variant of TTAC still works (Table 3) although its accuracy is close to that of existing methods (SHOT, TENT).\n- The ablation study covers each component of the method and every setting considered.\n\nWeaknesses\n\n- sTTT is close to existing notions of test-time training, test-time adaptation, and source-free adaptation.\n  sTTT underlines one-pass adaptation and minor use of source data, but test-time adaptation already focuses on one-pass adaptation _without_ source data, and several source-free methods report one-pass and multi-pass results (SHOT and BUFR, for example).\n  A missing related paper, BUFR, has already argued for minimal use of source data---in their case, the collection of feature histograms---to improve adaptation peformance with results in the setting this work calls sTTT (see one-pass results).\n- TTAC lacks technical novelty, as it applies several known components from source-free and test-time adaptation without a particularly strong point of innovation.\n  - exponential moving average estimation: EATA [B], BN [C]\n  - pseudo-label filtering: SHOT [18] has a prototypical clustering and pseudo-labeling filtering step that serves the same purpose as PLF\n  - global feature alignment: TTT+ [19], as acknowledged in this work\n  - queuing data to stabilize estimates: TTT+ [19], as acknowledged in this work\n  - incremental mean and variance updates are well-known (Equation 8)\n- The use of a queue for past data increases the computation for inference and adaptation. Algorithm 1 suggests the use of two devices, one for prediction and one for adaptation, but this is double the resources of fully test-time methods like BN or TENT.\n- The experiments do not always align with the motivation and claims. In the fully source-free setting on CIFAR-10-C, the accuracy of TTAC is comparable to other methods like SHOT and TENT. Although sTTT emphasizes one-pass adaptation, Table 4 focuses on multi-pass adaptation, and results on VisDA-C in one pass are not reported.\n  Although there is an emphasis on practicality and efficiency, results are not reported on ImageNet-C, which should be possible for an efficient method. It is potentially worrisome that most results are reported on the smallest dataset, CIFAR-10-C, and even on CIFAR-100-C the gap with existing methods shrinks (Table 1).\n- (Minor) The writing needs editing throughout for word choice and grammar. (See \"Other Feedback\" in the next section for typos spotted.)\n\n\n[A] BUFR: Source-Free Adaptation to Measurement Shift via Bottom-Up Feature Restoration. Eastwood et al. ICLR'22 and arXiv'21. \n\n[B] EATA: Efficient Test-Time Model Adaptation without Forgetting. Niu et al. ICML'22 and arXiv'22 (~2 months before the NeurIPS deadline). \n\n[C] BN: Improving robustness against common corruptions by covariate shift adaptation. Schneider et al. Neurips'20.\n\n**Update**: I have raised my score given the benchmarking on ImageNet-C and profiling of computation time.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Qn_IRh3q8IN", "reviewer": "Reviewer_Gh1A", "summary": "The paper proposes a clustering-based method for sequential test-time training, where the test data under distribution shifts arrives in a stream manner. The core idea of the proposed method is to match the cluster statistics between the source and target domain. To obtain a robust estimate of the cluster statistics, the paper introduces a pseudo label filtering strategy as well as global feature alignment. The empirical results are better than several previous methods on multiple datasets. ", "questions": "* How sensitive is the proposed method to the threshold of pseudo label filtering?\n", "limitations": "The authors did not clearly discuss limitations of their method.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n* The categorization of test-time algorithms proposed in the paper is neat and convincing. Indeed, the evaluation protocol in test-time training and source-free adaptation has been sometimes inconsistent. The paper does a great job to streamline the evaluation protocols. \n* The proposed anchored clustering also looks technically sound. While clustering, pseudo labelling and feature alignment have been separately explored in recent methods, the paper nicely connects them in a principled way. I also enjoy reading the detailed modifications of these components, e.g., eq 8.\n* The empirical evaluation of the method looks complete and rigorous. It’s great to see that the proposed TTAC is compatible with other methods like SHOT and TTT++. \n\nWeaknesses\n* Many prior test-time algorithms work only under a narrow set of distribution shifts. I suspect the proposed method may be subject to some particular settings too. It would be good if the authors could clearly discuss limitations and failure cases.\n* In particular, the proposed method introduces additional hyperparameters for peusde label filtering. It would be good to know how these parameters affect the performance.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "JgOo7VsUfzL", "reviewer": "Reviewer_pHz1", "summary": "In spite of the numerous efforts put into test-time training (TTT), the experimental settings are unclear, resulting in unfair comparisons.\nThis paper revisits TTT assumptions and classifies TTT protocols. This paper aims to tackle on the most realistic and challenging TTT protocol (one-pass test time training with no modifications to training objective). Motivated by the confused evaluation protocols for TTT, the authors propose test-time anchored clustering (TTAC) approach to align target domain features to the source ones in sequential test-time training. On five TTT datasets, TTAC consistently beats state-of-the-art approaches.", "questions": "- Sensitivity analysis for key hyperparameters such as thresholds used for pseudo label filtering related was not presented. I would like to know the trends related to key hyperparameters.", "limitations": "There is no description of the limitations or potential negative social impacts of the proposed method.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n- This paper is well-written and the whole part including motivation, method and experiment are easy to understand.\n- Clear and detailed descriptions of the proposed method, which provide enough information to reproduce.\n- This paper presents a categorization of TTT protocols based on two key factors so that comparisons of TTT methods within each category could be fair.\n- This paper shows the experimental results for various experimental protocols for each method, which can serve as a baseline for subsequent TTT-related studies.\n\nWeakness:\n- This paper is not the first attempt to use statistical information between the source domain and the target domain to alleviate domain discrepancy. [1] was not mentioned even though it is very relevant to this paper in that it reduces the difference between the centroid (mean) of the source domain and the centroid (mean) of the target domain via pseudo labels or uses a moving average scheme. Although this paper uses covariance in addition to mean, novelty is not enough.\n- On line 125, there must be a space between alone and we.\n\n\n[1] Xie, Shaoan, et al. \"Learning semantic representations for unsupervised domain adaptation.\" ICML 2018.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}]

Author rebuttal:

Decision: Accept