TTT++: When Does Self-Supervised Test-Time Training Fail or Thrive?
Authors: Yuejiang Liu
Parth Kothari
Bastien Germain van Delft
Baptiste Bellot-Gurlet
Taylor Mordan
Alexandre Alahi
Venue: NeurIPS 2021 Poster (2021)
Keywords: Test-time Adaptation
Source-free Domain Adaptation
Self-supervised Learning
Feature Alignment
Robustness
Generalization

Abstract: Test-time training (TTT) through self-supervised learning (SSL) is an emerging paradigm to tackle distributional shifts. Despite encouraging results, it remains unclear when this approach thrives or fails. In this work, we first provide an in-depth look at its limitations and show that TTT can possibly deteriorate, instead of improving, the test-time performance in the presence of severe distribution shifts. To address this issue, we introduce a test-time feature alignment strategy utilizing offline feature summarization and online moment matching, which regularizes adaptation without revisiting training data. We further scale this strategy in the online setting through batch-queue decoupling to enable robust moment estimates even with limited batch size. Given aligned feature distributions, we then shed light on the strong potential of TTT by theoretically analyzing its performance post adaptation. This analysis motivates our use of more informative self-supervision in the form of contrastive learning for visual recognition problems. We empirically demonstrate that our modified version of test-time training, termed TTT++, outperforms state-of-the-art methods by significant margins on several benchmarks. Our result indicates that storing and exploiting extra information, in addition to model parameters, can be a promising direction towards robust test-time adaptation.

TTT+:: When Does Self-Supervised Test-Time Training Fail or Thrive?
Yuejiang Liu
Parth Kothari
Bastien van Delft
Baptiste Bellot-Gurlet
Taylor Mordan
Alexandre Alahi
École Polytechnique Fédérale de Lausanne (EPFL)
{firstname.. Siirstaame.llsttaaeeepppppe lastnane
Abstract
Test-time training (TTT) through self-supervised learning (SSL) is an emerging paradigm to tackle distributional shifts. Despite encouraging results, remains unclear when this approach thrives or fails. In this work, we first provide an in- depth look its limitations and show that TTT can possibly deteriorate, instead improving. the test-time performance in the presence of severe distribution shifts. To address this issue, we introduce a test-time feature alignment strategy utilizing offline feature summarization and online moment matching, which regularizes adaptation without revisiting training data. We further scale this strategy the online setting through batch-queue decoupling enable robust moment estimates even with limited batch size. Given aligned feature distributions, we then shed light on the strong potential of TTT by theoretically analyzing performance post adaptation. This analysis motivates our use of more informative self-supervision in the form of contrastive learning for visual recognition problems. We empirically demonstrate that our modified version of test-time training, termed TTT++, outper- forms state-of- the-art methods by significant margins on several benchmarks. Our result indicates that storing and exploiting extra information, in addition to model parameters, can be promising direction towards robust test-time adaptation. Our code is available at httphttt: thub. com/ wptttttfl/ttttppppppp plus
Introduction
Machine learning models often struggle to generalize under distribution shifts. Even a perceptually mild shift between training and test data, e.g., JPEG compression, may cause severe prediction errors [ One popular family of methods to address this challenge is learn an invariant representation across domains by making use of labelled training data and unlabelled test data simultancously (2/5). However, revisiting training data at test time can impractical due to increasing privacy concerns, inflating sizes of datasets as well as many other real-world constraints. This shortcoming prompts more challenging yet appealing test-time adaptation paradigm: given a trained model, how can we adapt it from one domain to another on the fly, without access to training data and human annotations?
One promising approach towards this goal is test-time training (TTT) through self-supervision (6). The key idea of TTT is simple and straightforward: train the model on two tasks, namely a main task and supf-supe learning (SSL) task, and update the model based only on the SSL task at test time. This technique implemented with self-supervised rotation prediction has shown encouraging results for improving the robustness of image classifiers under variety of distributional shifts. Yet. its empirical performance is still inferior to other families of test-time algorithms 7 g
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

In this paper, we first take an in-depth look at TTT with emphasis on its limitations, Our analysis starts with a basic question: can TTT always mitigate the effects of distributional shifts? Through an illustrative problem, we show that the TTT framework can lead to surprising failures, deteriorating the test accuracy rather than improving it. This problem is largely attributed the unconstrained update from the SSL task that interfere with the main task. To address this issue, we introduce test-time feature alignment strategy by means of offline feature summarization and online moment matching: once training completes, we compute the mean and covariance matrix of training features and store them as part of the model, referred to as offline feature summarization; at test time, we encourage the test feature distribution to be close to the training one by matching the moments estimated online with those pre- pre-compueed offline, a process referred to as online moment matching.
One practical challenge for online feature alignment lies in scaling the strategy to problems with a large number of classes, as obtaining robust estimate of moments often requires at least a handful of samples per class. To mitigate this issue, we draw inspiration from recent literature [9] and decouple the sample size from the batch size for moment estimates. Specifically, we maintain a large dynamic queue of encoded features and progressively update it in mini-batch manner. This modification enables effective feature alignment even with limited batch size, greatly improving its viability in the online test-time setting.
Finally, we shed light on the strong potential of TTT through a theoretical analysis of the test accuracy after adaptation. In particular, we derive a lower bound of the test accuracy on the main task and show that it is expected to grow rapidly when the SSL task gets closer to the main task. These findings motivate our integration of contrastive representation learning 12 as strong instance of SSL,, into the TTT framework in visual recognition problems.
By combining the three proposed components, we devise an improved version of test-time training, termed T Experimental results show that TTT++ significantly outperforms other recent methods by significant margins on various robustness benchmarks. Our results suggest that exploiting extra information, including both task-specific information in the form of strong self-supervision and domain- domain-specific information in the form of feature summarization, can be a promising direction to enhance the effectiveness of test-time adaptation.
2 Background
2.1 Related Work
Test-time Adaptation. Adapting machine learning models based on test samples has garnered
growing interests in both generative problems such as super-resolution [13], image synthesis [T4]
and image manipulation [15], and discriminative problems like image classification [16]. Our work
is focused on the latter one in the presence of distributional shifts. Several recent works [16] [7] have shown the advantage of adapting the learned classifier to new test domains in the unsupervised manner, without revisiting the source data. One simplest form is to replace the batch-norm statistics estimated the training set with those on test examples [17.. Another line of work proposed to adapt the model parameters by exploiting the predicted labels on test examples, such as entropy minimization [ and pseudo-labeling (7]. While these methods yield promising results on some benchmarks, they are inherently restricted to classification problems and often vulnerable under large
distribution shifts (18).
More closely related to ours, 6 proposed test-time training through self-supervised learning, e.g., predicting the type of image rotations. This approach does not involve any assumptions about the output for the main task and is therefore more generic. It has been successfully applied to a variety of problems, such instance tracking [19] and reinforcement learning (20). Nevertheless, it was shown empirically inferior to other test-time algorithms [8]. Our work provides an in-depth analysis of its limitations, introduces simple yet effective remedies, and consolidates more theoretical grounds.
Feature Alignment. Aligning the distributions of training and test samples in the feature space is commonly used for domain adaptation. Previous feature aligmment methods fall into two main
categories: minimizing a divergence measure, such as MMD 21]. Coral 22 and CMD (23], or encouraging the domain confusion through adversarial training 5 24 However, most of these methods rely on the co-existence of source and target data, and thus cannot be readily applied to the test-time setting where source data is not available. Our work revisits the critical role of feature

---

Main
Main
Main
SSL
SSL
SSL
(a) training features
(b) test features before TTT
(c) test features after TTT
Figure I: Illustration of failure case where TTT hurts robustness under distributional shifts. The predictive model reaches high accuracy on both the main classification task, i.e., separating red blue data points, and the auxiliary self-supervised task, ie., separating circles and crosses, in the training domain. (b) Given a large distributional shift, test samples are encoded into new subspace, resulting limited accuracy on both tasks. (c) Without any constraints the feature distribution, TTT may result in updated encoder severely overfitting the SSL task, and consequently deteriorate the accuracy on the main task, opposed to improving
alignment for test-time training and proposes simple and practical strategy that enables online feature alignment, even with a limited batch size.
Sepe-Suue Learning. Supe-supe learning is powerful paradigm to learn rich repre- sentations from unlabeled samples. Stunning progress has been made in recent years by designing informative self-supervised tasks [11 25 28 and stabilizing the training process (9 29). Prior
works are mainly focused on unsupervised pre-training, whereas our work looks into the importance of incorporating strong self-supervised learning methods for test-time adaptation.
2.2 Preliminary: Test-Time Training
Test-time training (TTT) [6] is general framework for adapting neural network models to test distribution based on unlabeled samples. Different from the conventional approach that trains the model only the task of interest, TTT considers tasks: main task and auxiliary SSL task. The model is trained on both tasks simultaneously with a multi-task architecture composed one shared encoder g and separate heads 7. and πT, respectively. Given labeled training dataset D = { y:) (e..... NJ. the model is trained to minimize two losses jointly:
1
Ltrain (D:g, Tmas Te)
(x 9eig, Tm) (C,
where À is hyper-parameter balance the two tasks.
In the presence of distributional shifts, the learned model often struggles to directly generalize to a new test set D' {E:))E( The core idea of TTT is to fine-tune the encoder based on the self-supervised task with the test examples,
1 N' N Σ c,(Ri)))
Crrr(D'sg')
with the hope that the updated model n. o g' yields improved results the main task.
TTT instantiated with a self-supervised rotation prediction task has been demonstrated effective for improving the robustness of image classifiers under common distributional shifts (6]. However, it was shown inferior other families of test-time adaptation methods 8 We will next look into its strengths and limitations, and propose improved version for better adaptation performance.
3 When Does Test-Time Training Fail?
In this section, we first throw light on caveat of test-time training under large distributional shifts, and subsequently introduce practical solutions tailored for the test-time setting.

---

Stage Training
Stage IlI: Test-time
m
Lm
S
7t5
L
Batch-queue Decoupling
Stage Il: Summarization
4
Ugy Zg
Ugr
Our modified version of test-time training (TTT+)). Our method consists of three stages: model training, offline feature summarization, and online test-time adaptation. (i) During training, the model is optimized for the main task and auxiliary contrastive self-supervised task jointly. (ii) Once training completes. summarize the feature distributions after the encoder and the self-supervised head in the form of first and second-order moments. (iii) At test time, we adapt encoder through online feature alignment (Sec.3.22 and sulf-super learning (Sec. 4.2). In case of limited batch size, we maintain a large dynamic queue of feature vectors for robust moment estimates (Sec. 3.3
Figure 2:
3.1 Illustrative Example of Failures
One implicit assumption behind TTT is that the encoder update based on SSL can counter the effect of the underlying distributional shift on the main task. This assumption, however, is likely broken under large shifts and results in unexpected adaptation failures. To illustrate this limitation, introduce a simple toy problem in Figure where the main task and the SSL task are defined as classifying colors and symbols of encoded features in the 2-dimensional latent space. We consider an ideal scenario where the two tasks are highly correlated such that a well-trained model can attain high accuracies on both of them in the training domain. Nevertheless, in the presence of significant distributional shift, the model may still suffer from substantial prediction errors in the test domain.
In this scenario, while TTT may restore the discriminative power of the learned representation for the main task to a certain degree, the unconstrained self-supervised adaptation may yield severe overfitting to the auxiliary SSL task. As consequence, the performance the main task deteriorate as opposed to improving. This phenomenon is not restricted to our illustration and also occurs in practice, as shown in Section 5.1
3.2 Online Feature Alignment
As illustrated in the toy example above, simply applying self-supervised adaptation at test time can lead to arbitrarily poor results. To address this issue, introduce an online feature alignment strategy to ensure robust adaptation at test time. The core idea of our strategy is to impose a constraint over the feature space during TTT such that the feature distribution of test examples remains close to that in the training domain. While some feature alignment techniques such as MMD 2 and adversarial training 24 have been widely used for domain adaptation, they often rely on sampling from training and test domains concurrently, which is impractical in the test-time setting. We, therefore, turn to classical divergence measures that can be estimated independently for each distribution. More specifically, we use the square distance of the first and second moments between two feature distributions, inspired by DDC 30 and Coral summarize the distribution of training features in a compact format and store it as part of the model, eliminating the need to revisit the training data during test-time adaptation.
to approximate the domain discrepancy. This design choice allows to
Concretely, once training completes, we perform an offline feature summarization step that char- acterizes the distribution of feature vectors in the training domain Z = (=f xf through the empirical mean 14, 1/ D 2; and covariance matrix Σg = ~ (ZTZ (IT Z)). The former essentially equivalent to the channel-wise batch normalization statistics while the latter more informative yet light- for computation and storage. At test time, regularize the

---

self-supervised adaptation by minimizing the distance between the feature statistics estimated from mini-batch of test samples, i.e., ين= and Σn and the pre-stored quantities about the training domain:
f.a 11 1L2122 + || -E/I
(3)
where |l-l12 is the Fuclidean norm and |-111 is the Frobenius norm.
The basic form of online moment matching can be limiting in that the low-order statistics may be insufficient to fully capture complex distributions in high dimensions, e.g., 2048 for the standard ResNet-50. To alleviate this issue, we align the feature distributions at both the output of the encoder and the output of the self-supervised head, which are of lower dimensions, e.g., 128 in the case of contrastive learning described in See. 4.2 Our final objective at test time is weighted combination the self-supervised loss C,, the feature alignment loss at the encoder f,ss and that at the self supervise head f
CITT++ = L, + X,f,,= + Xg
(4)
where 0 and A, are parameers controlling the emphasis on each term.
3.3 Online Dynamic Queue
One practical challenge for online feature alignment lies in scaling the strategy to problems having large numbers of classes. Intuitively, good estimate of moments of the entire distribution needs at least a handful of samples per class. As consequence, the demand for sample size grows linearly with the number of classes, for instance, over ~1000 samples are required in the case of CIFAR- 100. However, the computational resources during deployment are often limited to accommodate such large batch size in the test-time setting.
To overcome this challenge, we draw inspiration from recent literature 9 and maintain a dynamic queue of encoded features decouple the batch size from the sample size for moment estimates. More specifically, we construct a dynamic queue that contains a few batches of feature vectors encoded test time. We progressively update the queue by appending the latest mini-batch and popping out the oldest one, as illustrated in Figure 2 This batch-queue decoupling allows us to collect large and consistent pool of samples for online moment matching, even with very limited batch size.
By integrating the online moment matching and batch batch-queue decoupling, into account both the discriminative power and the marginal distribution of the updated representations, enabling more robust adaptation under various settings. It is worth noting that the current moment matching strategy is just a particular instance of the online feature alignment scheme. It can be naturally extended to incorporate higher-order statisties [3] 31 to bring further performance gain at the cost of larger space and computational complexities.
timt-time algorithm takes
When Does Test-Time Training Thrive?
Given properly aligned feature distribution, next look into the potential of test-time training given strong SSL tasks. We first derive a lower bound of the test accuracy in general scenarios and then analyze it in specific setup where the performance on the main task can be directly estimated. These analyses motivate our use more informative self-supervised learning for test-time training.
4.1 Theoretical Results
We consider a training set comprised of samples drawn from the joint distribution IP X,,... where X, Y., and Y, are random variables corresponding the training samples. the main task labels and the self-supervised labels respectively. Similarly, the test set consists of samples drawn from the joint distribution P Px Y!. In the presence of distribution shift, P P..y,..Y. and PxrY.. Y are not identical. However, we make the following assumption about label distribution in our analysis.
Assumption 1. The training and test labels are equal in distribution, Ym. Ynn Y, d Y! and (Ym, YY) 1 (Y%, Y?).
In addition, we restrict our analysis to the scenarios where both two tasks can be solved perfectly during training.
5

---

Assumption 2. There exist an encoder and classifiers xm and such that ((Tm( = Ym)
1 and ((T))((X)) Y,)
During TTT, the shared encoder g is updated to g' such that the self-supervised head iTs fits the test data. Given our proposed online feature aligmment in Equation 4 we assumee the encoded features in the training and test domains, i.e., Z and Z', have the following property.
Assumption 3. The marginal feature distribution and the conditional feature distributions at test time are aligned with their counterparts during training, that is, _ 7' and (Z Y, = k:) - (Z' Y =) for all classes k in the SSL task.
Under these assumptions, we consider the outcome of test-time training in the worst-case scenario. Theorem 1. The prediction accuracy on the main task is lower bounded:
P( (Z') = Ymm 2 Σ P(Y, = V/) max {0.2
y-) - 0.5 } (5)
0.2 max P(Ym = Ym Yy, = -
Vx
Proof. Please refer to Section
A.I in the supplementary material.
Theorem D Thighlights the importance of the relation between the main task and the SSL task for the test accuracy after adaptation. More specifically, the SSL task needs to be informative with respect to the main task to guarantee the performance, i.e., knowing the SSL class ye makes main class yo highly probable, or equivalently PY = ym Y. = ya) is large, leading to a greater lower bound.
To further understand the impact of the task relation on test-time training, we next consider a particular setting, where the encoded features fully overfit the SSL task (e.g.. lengthy time training) and become independent of the main task label given the SSL label. Under this condition, the prediction accuracy on the main task can be directly estimated as follows.
Theorem 2. If Z' 11 Y YY, then the prediction accuracy on the main task is given by
P(Tm(Z)) Ymm
Σ y
P(Y. -Vo) Σ ym Y. = )
(6)
3m
Proof. Please refer Section A.2 in the supplementary material.
Intuitively, if the encoded features do not contain more information than the SSL labels about the main task, the accuracy of the main classifter n.. only depends the property of the SSL task. In particular, the square term on the right-hand side of @iaaaioon/emmpasizeeee the paramount importance of designing a closely related SSL task. When the two tasks diverge, the test accuracy drops quadratically fast, leading to ineffective adaptation.
4.2 Test-Time Training through Contrastive Learning
Our theoretical analysis above reveals the importance of incorporating an SSL task highly correlated with the main task for test-time training. One practical way to quantify the relation between two tasks is to measure the transferability of the representation learned from one task to another (32 Given the remarkable results contrastive methods for visual representation pre-training 9 33 34]
we hypothesize that they would also be suitable choices for test-time training. We thus replace the rotation prediction task with SimCLR [2] in the context of visual recognition. Given a mini-batch of B images, augment each image two views. We consider the two augmented views from the same original instance as a positive pair and treat the other pairs as negative ones. The feature vector 2 ((x;) of each image x; is projected self-supervised head. The projected hidden embeddings from a positive pair <h,, hy> are encouraged to be closer than those from the negative ones through the following loss,
diwee-dimensionll space h; nT (2;) through
exp(sim(hi /4, h2)/T)
C. - log Σ Light exp(sim tw)/T)
where r a temperature scaling parameter. The distance between projected embeddings measured by cosine similarity sim(u, u, v) = u" uFo/[[|u]]]|]]]]

---

Label Clessiiictiio
Adapt Accuracy
Adapt Accuracy
0.8
0.75
0.7
0.70
Feature PCA
0.65
0.6
TTT
0.60
TTT
0.5
0.55
Training
Test Accuracy
Label Agreement
Figure 3: Qualitative comparison of TTT 6 and TTT+ on the inter-twinning moons problem in the presence large translation and rotation shifts. The vamilla TTT drives the decision boundary further away from desired one due to severe feature misalign- ment, marked by the dashed arrow in the feature PCA visualization. In comparison, decision boundary the test domain effectively.
Figure 4: Quantitative comparison of TTT 6 and
TIT+ on the inter-twinning moons problem under 150 different setups. Standard deviations are visualized shaded regions. Our method (left) is particularly advan- tageous under large shifts, i.e., low test accuracy before adaptation, and (right) greatly benefits from higher cor- relation, ie.. larger label agreement, between the main and SSI. tasks.
method adapts the
5 Experiments
We empirically validate our proposed method in four scenarios: synthetic toy problem, common image corruptions, natural domain shifts, and sim-to-real transfer.
We consider the following baselines throughout our experiments:
Test: the model in the training domain directly evaluated on the test data without any adaptation;
Test-time normalization (BN) 35 updates the batch normalization statistics of the trained network according to the test data;
Test-time entropy minimization (TENT) S updates the batch normalization statistics of the trained network by minimizing the entropy of the model predictions on the data;
Source Hypothesis Transfer (SHOT) [36 freezes the classifier module and updates only the feature extraction module by exploiting the concepts of information maximization and self-supervised pseudo-labeling during testing;
Test-time training (TTT-R) [6] trains the network jointly on the main task and rotation-base baadd SSL task in the source domain; during test, TTT-R continues to train on the rotation-based based task in the target domain.
We also evaluate the following ablated versions of our method:
Test-time feature alignment (TFA) aligns the first-order and second-order statistics of the source and target distributions during testing (Section 3.2 and 333
Test-time contrastive learning (TTT-C) trains the network jointly on the main task and a contrastive learning (SSL) task in the source domain; during test, TTT-C continues update the encoder
based on contrastive learning in the target domain (Section 4.2
The full version of our method, improved test-time training (TT++), combines TFA and TTT-C
5.1 Synthetic Toy Problem
We first evaluate our method on the inter-twinning moons problem to predict the moon class of given data point and the SSL task is to predict on which side of the hyperplane (i.e., linear separator between the two moons) the data point lies The relation (label agreement) between the two tasks depends on the separation distance between the two moons. To solve both tasks simultaneously, shared encoder and two 2-layer MLPs as separate task heads. Each hidden layer contains neurons. The learned model attains over 99% accuracy both the main and SSL tasks in the training domain. We simulate a variety of distributional shifts through translation and rotation of all data points.
where the main task is
build small neural network that consists of 2-layer MLP the
Figure 3 shows the decision boundaries and encoded features from the vanilla TTT and our proposed TTT++ particular test case of large distributional shift. TTT not only fails to improve the classification accuracy but even pushes the decision boundary further away from desired one due to

---

Table 1: Average classification error (%) On
Test
50
Tent
CIFARIO-C/100-C 1 and CIFAR
40
Error (%)
40
TTT+
Method
C10-C
C100-C
C10.1
30-
Test
29.1
61.2
12.1
BN A4
15.7
43.3
14.1
20
TTT-R 6
14.3
40.4
11.0
10-
SHOT 36
14.7
38.1
11.1
TENT 8
12.6
36.3
13.4
brit
contr
defoc
elast
fog
frast
gauss
glass
impul
Jpeg
moin
Dixtll
shot
snow
zoom
TFA (Ours)
11.9
35.8
12.1
TTT-C (Ours)
10.7
36.9
9.7
TIT+ (Ours)
9.8
34.1
9.5
Figure 5: Classification error (%) CIFARIO-C M
the severe feature distribution mismatch, as evidenced in the PCA visualization. In comparison, our TTT++ yields substantial performance gain, boosting the test accuracy from 50% to 93%, thanks to the reduced feature distributional shift enforced by the proposed online moment matching.
Figure 4 summarizes the quantitative results of our methods well the vanilla counterpart under 150 different simulated setups. As shown on the left graph, when the domain shift only causes mild test errors the main task, both the original TTT and our TTT++ yield strong adaptation results. However, the effectiveness of TTT deteriorates quickly along with the growth of domain shift, as reflected the lower test accuracy. In comparison, our proposed TTT+ demonstrates clear advantages under large shifts, e.g. when the test accuracy before adaptation is around 0.5. We further examine the impact of the relation between the main and SSL tasks on the adaptation performance by varying the separation distance between the two moons. The simulation results confirm the high potential of tist-time training given improved SSL tasks, analyzed in Section 4.1
5.2 Common Image Corruption
We further assess the robustness of our method against common image corruptions. Following the evaluation protocol of previous work [ we train ResNet-50 (38] on CIFARIO/CIFAR and test it on the CIFARIO-C/CIFARII 100-C [ datasets, which contain 15 types of algorithmically generated corruptions, such as noise, blur and snow effects. We use batch size of 256 for test-time adaptation. In addition, we use dynamic queue containing 16 batches of feature vectors for online feature alignment on CIFAR100-C.
[39]
Figure shows the quantitative results under each type of image corruption. The average results on CIFARIO-C and CIFAR 100-C are reported Table Our TTT++ clearly outperforms the prior ffte-offtheeart test-time methods on CIFARIO 10-C and CIFAR 100-C. C. In particular, incorporating a strong ceffsuupevvisio task (TTT-C) already suffices perform on par or better than TENT. Adding test-time feature alignment (TFA) on top of that yields an additional ~ 8% relative reduction in terms of the test eиror.
5.3 Natural Domain Shift
We next demonstrate the efficacy of our method TTT+ to tackle natural distribution shifts. We again use the pre-trained ResNet-50 and test it on CIFARIO.1 [40), a recently collected test set subject to natural distributional shift. Despite its high perceptual similarity with the CIFARI 10 dataset, the CIFARIO.1 typically leads to drop of accuracy (4% to 10%) for a wide range of deep models (40).
Table I summarizes the results of various test-time algorithms on CIFAR10.1 Previous batch-norm- based methods perform poorly and even degrade model accuracy. This phenomenon tied their implicit assumption that different samples and spatial locations are shifted in a similar manner [18]. which is true for algorithmically generated image corruptions but does not hold on CIFAR-10.1 In contrast, TTT++ is more generic and yields stronger performance under the natural distribution shift.
5.4 Sim-to-Real Transfer
We finally validate the effectiveness of our method on the VisDA-C dataset 42 large-scale benchmark of synthetictto-real object classification. As shown in Table
challenging
2 prior methods

---

Table 2: Classification (%)
the large-scale VisDA-C dataset
42
Method
plane
beyel
horse
knife
meyel
person
plant
sktbrd
urin
truck
Per-class
Test
56.52
88.71
62.77
30.56
81.88
99.03
17.53
95.85
51.66
77.86
20.44
99.51
58.72
BN 4T
44,38
56.98
33.24
55.28
37.45
66.60
16.55
59.02
43.55
60.72
31.07
82.98
48. 12
TENT
13.43
77.98
48.15
21.72
82.45
12.37
35.78
76.41
98.93
42.73
SHOT
36
5.73
13.64
23.33
42.69
7.93
86,99
19.17
19.97
11,63
11.09
15.06
43.26
25.04
TFA (Ours)
28.25
32.03
33.67
64.77
20.49
56.63
22:52
36.30
24.84
35.20
25.31
64.24
39.58
TTT-C (Ours)
5.46
32.23
25,42
37.03
7.84
85,20
9.14
23.80
11.72
11.00
7.74
56.87
25.72
TTT++ (Ours)
4.13
26.20
21.60
31,70
7.43
83,30
7.83
21.10
7.03
7.73
6.91
$1.40
22.46
Table 3:
Classification error (%) results from online feature alignment with or without dynamic queue of feature vectors the CIFARI00-4 under level-5 fog coruuttion. Sample size Batch size # Batches. Given a fixed batch size, enlarging the queue size leads similar results as having a larger batch size.
w/o queue
w/ queue
Sample Size
64
128
256
64 x 2
64 x
64 x 8
64 x 16
Test Error
40.31
38.67
37.01
39.84
37.37
36.18
36.02
that are fairly competitive under image corruptions, such BN 41 and TENT 8 are not effective on VisDA-C. We conjecture that this is attributed to their strong restrictions over the adaptable parameters at test time. In contrast, our proposed method is more flexible, allows the model to update the entire encoder, and thus achieves compelling results on VisDA-C. Furthermore, the test-time feature alignment plays a crucial role in this synthetic-to-real domain adaptation problem, providing 13% performance boost on top of the TTT-C
5.5 Effect of Batch-Queue Decoupling
To verify the effects of batch-queue decoupling, we compare the results of online feature alignment with different sample sizes. Table summarizes the test errors on CIFAR 100 under the level-5 fog corruption. As expected, larger sample sizes generally lead lower classification errors. Interestingly, while the performance of using a dynamic queue is slightly worse than its counterpart of the same sample size from single large batch, enlarging the queue size always yields stronger results. For instance, given a small batch size of 64, using a dynamic queue maintaining 512 or 1024 feature vectors from 8 or 16 consecutive batches respectively is more advantageous than the vanilla moment matching based on batch size of 256 samples. This result corroborates the benefit of integrating dynamic queue into our proposed feature alignment framework, for enhancing the scalability in the online setting.
5.6 Design Choice for Moment Matching
As discussed in Section our proposed online feature alignment can be instantiated with different orders of moments and applied at various layers. To understand the effects of the detailed design choices, we empirically compare our proposed version against several ablated variants in Table 4 Irrespective of the layer choice, our proposed online feature alignment consistently results in reduced classification error. Nevertheless, moment matching applied to the self-supervised head alone leads to lower test error compared to applying it to the feature extractor output in 11 out of 15 types of corruption. We conjecture that the strong performance of the former one is attributed to the lower dimensionality of the feature vector, which allows for more accurate estimate of feature statistics. The best result comes from the online feature alignment at the outputs of both the encoder and the projection head, which validates our design choice in Section 3.2
We further validate the choice divergence measure through an ablation study. The online feature alignment using the second-order moment (covariance) leads to clearly better results than the one using the first-order moment (mean). also evident that the online feature alignment is most effective when both the mean and the covariance are taken into account.

---

Table Classification (%) CIFARIO-C 1 with different versions of online feature alignment. Taking into account both the second-order moments at two different layers is betler than the other counterparts in terms of the robustness against most types of image corruptions.
TFA
brit
contr
defoc
clast
fog.
frost
gauss
glass
impul
jpeg
motn
pixel
shot
snow
200m
Cya
7,96
7.57
12.54
14.62
21.05
21.4
12.68
11.92
10.7
12.74
wio Lna
7,85
7.84
9.18
16.51
14.33
11.99
13.79
20.08
20.17
12.42
12.02
10.5
12.78
13.28
7.44
w/o E
7.49
7.56
9.62
18.62
19.22
12.72
16.02
25.07
25.17
13.43
13.63
11.22
15.04
15.11
7.77
wlo j
7.43
7.37
8.90
15.92
12.98
11.57
13.46
19.27
18.95
11.87
11.11
9.97
12.81
11.76
7.04
Full
7.44
7.40
8.89
15.73
12.82
11.49
12.94
18.46
19.13
11,66
10.77
9.93
12.67
11.73
7.03
Conclusion and Discussions
In this work, we conduct an in-depth analysis of the limitations and potential of self-supervised test-time training. We draw attention to the risk of feature distribution mismatch, which is critical but largely overlooked in recent test-time algorithms. We shed light on the strong potential of this approach by analyzing the growth of test accuracy given improved SSL tasks. These analyses inspire three proposed modifications, namely online feature alignment, batch-queue decoupling and con- trastive test-time training, which yield state-of-ther results on multiple robustness benchmarks. Our results suggest the advantages of bringing additional task-specific and domain-specific information in compact format for test-time adaptation. We hope these findings will motivate researchers and practitioners to rethink what should be stored, in addition to weight parameters, for the robust deployment of machine learning models.
Limitations. In this work, we restrain feature summarization to first and second-order moments. Yet, the low-order statistics may be insufficient to characterize the complex distribution of high- dimensional features. Developing more advanced summarization methods tailored for test-time adaptation is interesting avenue for future work. In addition, there may exist a considerable gap between our theoretical analysis and the empirical results, when the stated assumptions do not hold. For instance, neither the classification heads nor the feature alignment is perfect in practice. More theoretical guarantees can be valuable for the practical use of test-time adaptation in safety-critical scenarios.
Open Questions. In our experiments, we only consider the standard ResNet-50 as the backbone architecture and share the whole feature extractor between the main task and self-supervised task. Yet, recent literature (30 43 has shown that different layers capture different levels of semantic granularity. The impact of architectural design on test-time training remains an open question. Furthermore, while we empirically compare our proposed method against other families of test-time adaptation algorithms, these techniques exploit different supervisory signals extracted from unlabeled data, which can complementary each other. Blending these techniques into unified framework is another interesting direction to explore in the future.
Societal Impact. Our work aims at expanding the current horizon of machine learning algorithms for test-time adaptation. For applications where humans' lives are at risk, such as autonomous driving, trust, safety, robustness all mandatory keywords. The field has made amazing progress when the training and testing environments are highly similar. What happens when a machine gets deployed in environment? We, humans, have innate capability for handling such shifts. We believe that machines should have the same capability as humans. Indeed, there is a long way to go. Nevertheless, hope that work will foster more research in analyzing and devising algorithms for robust and effective adaptation at test time.
Acknowledgements
This work was supported by the Swiss National Science Foundation under the Grant 200021- 199326, Honda R&D Co. Ltd, EPFL Open Science fund and Valeo. We thank Sudeep Salgia, Tao Lin, Lingjun Meng, Yifan Sun for helpful inputs to our early drafts and anonymous reviewers for valuable comments.
10

---

References
[1] Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. Proceedings of the International Conference Learning Representations, 2019.
[2] Mingsheng Long, Yue Cao, Jianmin Wang, and Michael Jordan. Learning Transferable Features with Deep Adaptation Networks. In International Conference Machine Learning, pages 97-105. PMLR, June 2015. ISSN: 1938-7228.
[3] Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I. Jordan. Deep Transfer Learning with Joint Adaptation Networks. In Proceedings of the 34th International Conference Machine Learning, pages 2208-2217. PMLR, July 2017. ISSN: 2640-3498.
[4] Yaroslav Ganin, Evgeniya Ustinova, Hama Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. Domain-adversarial training of neural networks. The Journal of Machine Learning Research, 17(1):206-20300, January 2016.
[5] Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. Adversarial Discriminative Domain Adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 7167-7176, 2017.
[6] Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei Efros, and Moritz Hardt. Test-Time Training with Self- Superviserv for Generalization under Distribution Shifts. In Proceedings of the 37th International Conference on Machine Learning, pages
PMLR, November
2020. ISSN: 2640-3495
[7] Jian Liang, Dapeng Hu, and Jiashi Feng. Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation. In International Conference on Machine Learning (ICML), pages 6028-6039, July 13-18 2020.
[8] Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully Test Time Adaptation by Entropy Minimization. In International Conference Learning Representations, September 2020.
[9] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum Contrast for Unsupervised Visual Representation Learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages
2020.
[10] Raia Hadsell, Sumit Chopra, and Yann LeCun. Dimensionality Reduction by Learning an Invariant Mapping. In 2006 IEEE Computer Society Conference Computer Vision and Pattern Recognition Volume (CVPR'06), volume 2, pages 1735-1742, New York, NY, USA,
2006. IEEE.
[11] Zhirong Wu, Yuanjun Xiong, Stella Yu, and Dahua Lin. Unsupervised Feature Learning via Non-parametric Instance Discrimination. In 2018 IEEE/CVE Conference on Computer Vision and Pattern Recognition, pages 3733-3742, June 2018. ISSN: 2575-7075.
[12] Ting Chen, Simon Kornblith. Mohammad Norouzi, and Geoffrey Hinton. A Simple Framework for Contrastive Learning of Visual Representations. In Proceedings of the 37th International Conference on Machine Learning, pages 1597-1607. PMLR, November 2020. ISSN: 2640- 3498.
[13] Assaf' Shocher, Nadav Cohen, and Michal Irani. Zero-Shot Super-Resolution Using Deep Internal Learning. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3118 1118-116, June 2018. ISSN: 2575-7075.
[14] Yuejiang I. Liu, Parth Kothari, and Alexandre Alahi. Collaborative Sampling in Generative Adver- sarial Networks. Proceedings of the AAAI Conference on Artificial Intelligence, 34(04):4948- 4956, April 2020. Number: 04.
[15] David Bau, Hendrik Strobelt, William Peebles, Jonas Wulff, Bolei Zhou, Jun-Yan Zhu, and An- tonio Torralba. Semantic photo manipulation with a generative image prior, ACM Transactions on Graphics, 38:4):59:1159111 July 2019.

---

[16] Jogendra Nath Kundu, Naveen Venkat, Rahul M. V. and R. Venkatesh Babu. Universal Source- Free Domain Adaptation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4544 4553, 2020.
[17] Steffen Schneider, Evgenia Rusak, Luisa Eck, Oliver Bringmann, Wieland Brendel, and Matthias Bethge. Improving robustness against common corruptions by covariate shift adaptation. Advances Neural Information Processing Systems, 33, 2020.
[18] Collin Burns and Jacob Steinhardt. Limitations of Post-Hoc Feature Aligmment for Robustness. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pages 2525-2533, 2021.
[19] Yang Fu, Sifei Liu, Umar lqbal, Shalini De Mello, Humphrey Shi, and Jan Kautz. Learning Track Instances without Video Annotations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8680-8689, 2021.
[20] Nicklas Hansen, Rishabh Jangir, Yu Sun, Guillem Alenyå, Pieter Abbeel, Alexei A. Efros, Lerrel Pinto. and Xiaolong Wang. Self-Supervised Policy Adaptation during Deployment, In International Conference on Learning Representations, September 2020.
[21] Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Schölkopf, and Alexander Smola. A Kernel Two-Sample Test. Journal of Machine Learning Research, 13(25):723-773, 2012.
[22] Baochen Sun, Jiashi Feng, and Kate Saenko. Correlation Alignment for Unsupervised Domain Adaptation. In Gabriela Csurka, editor, Domain Adaptation in Computer Vision Applications, Advances in Computer Vision and Pattern Recognition, pages 153-171. Springer International Publishing, Cham, 2017.
[23] Werner Zellinger, Thomas Grubinger, Edwin Lughofer, Thomas Natschläger, and Susanne Saminger-Plate Central Moment Discrepancy (CMD) for Domain-Invarian Representation Learning. International Conference on Learning Representations, International Conference on Learning Representations, November 2016. 00283.
[24] Yaroslav Ganin and Victor Lempitsky. Unsupervised Domain Adaptation by Backpropagation. In Proceedings of the 32nd International Conference Machine Learning, pages 1180-111 PMLR, June 2015. ISSN: 1938-7228.
[25] Mehdi Noroozi and Paolo Favaro. Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles. In Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling, editors, European Conference Computer Vision, pages 69 84, Cham, 2016. Springer International Publishing.
[26] Mathilde Caron, Piotr Bojanowski, Armand Joulin, and Matthijs Douze. Deep Clustering for Unsupervised Learning of Visual Features. In Vittorio Ferrari, Martial Hebert, Cristian Sminchisescu. and Yair Weiss, editors, Computer Vision ECCV 2018, Lecture Notes in Computer Science, pages 139-156 156, Cham, 2018. Springer International Publishing.
[27] Yuejiang Liu, Qi Yan, and Alexandre Alahi. Social NCE: Contrastive Learning of Socially- Aware Motion Representations. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15118-15129, 2021.
[28] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski. and Armand Joulin. Emerging Properties in Self-Supervised Vision Transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9650-9660, 2021.
[29] Xinlei Chen and Kaiming He. Exploring Simple Siamese Representation Learning. In Pro- ceedings of the IEEF/CVF Conference on Computer Vision and Puttern Recognition, pages 15750-15758, 2021.
[30] Eric Tzeng, Judy Hoffman, Ning Zhang, Kate Saenko, and Trevor Darrell. Deep Domain Confusion: Maximizing for Domain Invariance. arXiv:1412.34P5 [es], December 2014. arXiv: 1412.3474.
12

---

[31] Chao Chen, Zhihang Fu, Zhihong Chen, Sheng Jin, Zhaowei Cheng, Xinyu Jin, and Xian- sheng Hua. HoMM: Higher-Order Moment Matching for Unsupervised Domain Adaptation. Proceedings of the AAAI Conference on Artificial Intelligence, 34(04):322233429 April 2020. Number: 04.
[32] Amir R. Zamir, Alexander Sax, William Shen, Leonidas J. Guibas, Jitendra Malik, and Silvio Savarese. Taskonomy: Disentangling Task Transfer Learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3712-3722, 2018
[33] Jean-Bastien GrilI, Florian Strub, Florent Altché, Corentin Tallec, Pierre H. Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Daniel Guo, Mohammad Gheshlaghi Azar, Bilal Piot, Koray Kavukcuoglu, Rémi Munos, and Michal Valko. Bootstrap your own latent: A new approach to self-supervised Learning. arXiv:2006.07733 [cs, staf], September 2020. 00623 arXiv: 2006.07733.
[34] Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsupervised Learning of Visual Features by Contrasting Cluster Assignments. arXiv:2006.09882 [cs], January 2021. 00403 arXiv: 2006.09882.
[35] Steffen Schneider, Evgenia Rusak, Luisa Eck, Oliver Bringmann, Wieland Brendel, and Matthias Bethge. Removing covariate shift improves robustness against common corruptions. CoRR, abs/2006.16971, 2020.
[36] Jian Liang, Dapeng Hu, and Jiashi Feng. Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation. In International Conference on Machine Learning, pages 6028-6039. PMLR, November 2020. ISSN: 2640-3498.
[37] Pascal Germain, Amaury Habrard, François Laviolette, and Emilie Morvant. A PAC-Bayesian Approach for Domain Adaptation with Specialization to Linear Classifiers. In Proceedings of the 30th International Conference on Machine Learning, pages 738-746. PMLR, May 2013.
ISSN: 1938-7228.
[38] Kaiming He, X. Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770-778, 2016.
[39] A. Krizhevsky. Learning multiple layers of features from tiny images, 2009.
[40] Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do ImageNet Classifiers Generalize to ImageNet? In Proceedings of the 36th International Conference on Machine Learning, pages 5389-5400. PMLR, May 2019. ISSN: 2640-3498.
[41] Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Francis R. Bach and David M. Blei, editors, Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015, volume 37 of JMLR Workshop and Conference P Proceedings, pages 448 456. JML..org. 2015.
[42] Xingchao Peng, Ben Usman, Neela Kaushik, Judy Hoffman, Dequan Wang, and Kate Saenko. Visda: The visual domain adaptation challenge. ArXiv, abs/1710.06924, 2017.
[43] Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson. How Transferable Are Features in Deep Neural Networks? In Proceedings of the 27th International Conference on Neural Information Processing Systems Volume NIPS' 14, pages 3320-3328, Cambridge, MA, USA, 2014. MIT Press.
[44] Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised Contrastive Learning, arXivv2004.11362 [cs, stat), April 2020. arXiv: 2004.11366
13


Reviews:
[{"review_id": "tFAcQwpWHLK", "reviewer": "Reviewer_qJr7", "summary": "This paper focus on improving the generalization of deep neural networks in the presence of distributional shift at test time i.e., unsupervised source-free network adaptation on data distribution that is different from training. They follow the footsteps of a prior work by Sun et al. [8] and leverage an auxiliary task at test time. However, this work points out that test time training with auxiliary tasks like rotation prediction need not necessarily encourage the network to improve its generalization on the main task. Thus, the authors propose to transfer the benefits of self-supervised learning (SSL) to the main task by introducing an online feature alignment strategy that utilizes offline feature summarization statistics collected on the original training (source) data and also choosing more suitable SSL task like contrastive learning instead of rotation prediction. Tthey denote the resulting approach as TTT++. Results and comparison to existing techniques are shown on popular datasets like cifar10-c, cifar100-c, cifar10.1 and VisDA-C.", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "## Originality ##\nThe online feature alignment strategy is inspired by distribution matching measures from the domain adaptation literature, however proposed firstly in this work for test time adaptation. Additionally, the authors employ the popular contrastive task as the SSL task. It is clearly distinguished from the previous works and sufficiently cited. In general, this work introduces concepts  from other fields into test-time adaptation, that are not novel in general. Originality is thus relatively small.\n\n## Quality ##\nTechnical details of this work are clear. They provide theoretical analysis about the relationship between SSL and main task. The method is tested on popular benchmarks. This is a complete piece of work. Authors are honest about strengths and weaknesses. \n\nThe authors motivate the online feature alignment strategy by stating that simply applying SSL based adaptation results in poor accuracy. However, results on cifar100-c and cifar10.1 show that such strategy included in TTT++ is almost comparable to network adaptation with contrastive based SSL task and slightly better for cifar10-c and little prominent on VisDA-C. The improvements from such alignment strategy on cifar10-c and VisDA-C, but not on cifar100-c raises a question whether this technique scales to dataset with larger number of classes (e.g. ImageNet). Overall, results suggest that the impact of the online feature alignment strategy is not as pronounced as strong as authors claim.\n\n## Clarity ##\nThe paper is nicely written, well organized and easy to read. \n\nComments: \n* Section 3.1 provides an illustrative example of a failure case. Though this section is easy to understand, it is not sufficient to reason about a failure case. A synthetic problem is provided in appendix. I suggest authors may consider moving parts of appendix to main paper to provide sufficient rationale on the argument.\n* Figure 4 illustrates the encoder feature distribution. Despite authors state that TTT+ displays improved per-class separation, it is not clear to infer this conclusion from the figure. It appears that SHOT and TTT++ show similar per-class feature distribution and feature alignment. I suggest the authors to consider a different illustration or representation to emphasize their claim from Figure 4, ideally provide quantitative numbers rather than only a visualization.\n\n## Significance ##\nThe results indicate that the proposed approach outperforms previous works on cifar10-c, cifar100-c and cifar10.1. However, the major improvements come from replacing the rotation prediction task from Sun et al. [8] with the contrastive learning and this choice of SSL task seems straightforward given its recent popularity.  The improvements from the online feature alignment are relatively small and removing it would not result in \"Test-Time Training Failing\"; this theme from Section 3 is actually not observed anywhere in the results.\n\nMoreover, the numbers reported from SHOT in Table 2 are not consistent with the results from the original paper Liang et al. [9]. The per-class error rate is reported as 17.1% on VisDA-C in Liang et al. [9], whereas it is reported as 25.04% in the Table 2 comparisons. Taking the error rate reported in original paper into account, TTT++ does not improve upon SHOT[9].\n\nIt is not clear how the hyperparameters are chosen for the test time adaptation. Are they chosen based on the performance on the test corruptions of the respective benchmark or the validation corruptions? Do the hyperparameters vary across the corruptions?\n\nI appreciate the results obtained with contrastive learning as an auxiliary task that outperforms the previous works. However, based on my above comments, I see this as a simple extension of Sun et al. [8] with small novelty. I consider this submission as marginally below acceptance threshold and would be willing to reconsider my score based on the author responses and other reviews.\n\n## Update after author response ##\nThe authors have addressed my comments and questions and I am raising my score to 6, that is: I see the paper slightly above the acceptance threshold. \nThe main limitations that prevent me from going to a score of 7 or higher is that (a) the proposed feature alignment strategy in its current form is limited to problems with few classes and (b) most (but not all) performance gains compared to prior work TTT come from using a stronger and well established self-supervised contrastive learning objective.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "I appreciate the authors for addressing the limitations and societal impact sufficiently.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "g1sgasY7W-j", "reviewer": "Reviewer_qJ8x", "summary": "This paper is about test-time adaptation (TTA), in particular about improving the recently published test-time training (TTT) approach. It provides an analysis about conditions that can lead to TTT failures, and overcome them with a method that relies on i) feature alignment and ii) more sophisticated self-supervised learning (SSL) objectives (when compared with the TTT's seminal work). The method is compared against other recent TTA approaches, showing good performance on standardized benchmarks.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This is a well written paper, concerning an important, recently introduced topic. The proposed method is properly located in the literature, and the ideas are well argumented and presented. Furthermore, the proposed method is very reasonable.\n\nTTA has the potential to revolutionize the deployment of machine learning systems in the real world, but only a bunch of algorithms exists as of today. Improving one of the leading algorithms in this niche topic (TTT, by Sun et al. [8]), this paper brings valuable contributions to the table; indeed, while effective, TTT relies on an SSL approach that is currently far from the state of the art: in this work, the importance of using more effective SSL objectives is shown, and some more methodological contributions are explored.\n\nTo improve the paper, I believe more ablation studies will allow to properly assess the efficacy of the proposed strategy. In particular:\n\n1) One of the powerful facet of TTA is the possibility to adapt on the fly to single test samples, or at least to a few samples. In this regard, I could not understand in which of the following two ways test-time adaptation is performed: A) adapt on all test samples via SGD (with the indicated batch size) and at the end evaluate performance on all test samples, or B) adapt on a batch (with the indicated batch size) and evaluate the model on that batch (if this is the case, the adaptation process starts from scratch at every step, of from the point reached by adapting to the previous batch?). I believe that the setting B) is the most interesting for TTA, because, as mentioned above, it would allow adapting on-the-fly to single samples. In my opinion, this work should address this point, by assessing how performance changes - also with respect to competing algorithms - as the batch size varies (in my understanding, in this work the batch size at test time is kept fixed - see Sec. C). For a reference, see the analyses performed in Schneider et al. [42].\n\n2) Related to point 1), I believe that some more ablation studies to understand the trade-off between the hyper-parameters in Eq.4 should be provided. Related to this, I found curious to see in Tables D.1 and D.2 that using the covariance matrix alone helps more than using the mean vectors alone: the covariance matrix alone only carries information about how feature vectors are spread in their space, not about their location. What would happen without the SSL objective (that is, by setting $\\lambda_s=0$ in Eq.4)?\n\n3) Minor - I did not find the analysis in Sec. 3.1. very enlightening, because the main task and the SSL task are very artificial. An example where it is shown that the proposed method avoids degenerate solutions would be significantly more helpful, in my opinion.\n\nNote: I am not able to properly assess the usefulness and the correctness of the theoretical results in Sec. 4.1 (as well as their proofs), since they are beyond my expertize.\n\nI believe this is a good paper, with some parts still lacking a bit of work. I look forward reading the Author response, discussing with other Reviewers and updating my score accordingly.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "/"}, "extra_text": {}}, {"review_id": "LAAU08QVx4g", "reviewer": "Reviewer_YjL7", "summary": "Authors propose a test time training method using contrastive loss based SSL (Self Supervised Learning) task and feature alignment to address the domain shift between training and test data. Authors give lower bound for the performance during test under some assumptions. ", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "\n1) Authors clearly describe the background of the work and their proposed method. Proposed method is a novel combination of well known techniques. The proposed method improves performance by 2.5% on VisDA-C dataset and around 23% for the CIFAR 10 dataset compared to the baseline. \n\n2) Can authors compare the results on the Imagenet-C dataset as previous papers in the literature have done [1]? If that can not be done, can authors justify why?\n\n3) It is not clear from paper how hyperparameters are chosen for Tent and SHOT baselines? It is possible that the baseline method's hyperparameters won't work for this data if the data processing is different. In that case, was there any grid search performed on baseline methods' hyperparameters?\n\n4) It would be good to have a graph of classification error as test time training processes to understand if performance is increasing gradually as training progresses similar to figure 2 from [1].\n\n5) Assumption that test feature distributions are aligned seems a bit strong. Based on what type of corruption applied on test data, the distribution can change. Can authors provide empirical data to back up this claim?\n\n6) It would be useful to show how the method performs on different domain shifts in test data. Does the method perform comparably l with Elastic Transform domain shift as it does with Fog?\n\n7) In equation 3, authors match the first and second moment of the data. Would it be possible to match all the moments of the distribution instead of only 2 moments like in [2]. \n\n8) Fundamentally the approach seems very similar to TTT where only the pretext task has changed from rotation to a new self supervised task. Is there any other fundamental change apart from first and second moment matching of the data? \n\n\n[1] Sun, Yu, et al. \"Test-time training with self-supervision for generalization under distribution shifts.\" International Conference on Machine Learning. PMLR, 2020.\n\n[2] Blanchard, Gilles, et al. \"Domain Generalization by Marginal Transfer Learning.\" J. Mach. Learn. Res. 22 (2021): 2-1.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "For limitations please check the main review. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The work proposed a test time training method to address the distribution shift between training and testing using online feature alignment and contrastive loss based SSL. Reviewers overall see the paper well written and are content with the empirical evaluation. The rebuttal was able to convince some reviewers to raise their scores. I do agree with reviewer qJr7 that the contrastive loss based SSL task does not specifically address test time training, but rather is a generic approach to improve representation learning/generalization, which is well established. Focusing more of the discussion around the online feature alignment might shed more light on the new contribution of the work. As suggested by another reviewer, on-the-fly adaptation might be the more interesting setup to further distinguish the TTT setup from other setups facing distribution shift, such as domain adaptation.  
Decision: Accept (Poster)