Test-time Correlation Alignment
Authors: Linjing You
Jiabao Lu
Xiayuan Huang
Venue: ICML 2025 poster (2025)
Keywords: Test-time adaptation
Correlation alignment
Primary area: general_machine_learning->transfer_multitask_and_metalearning
TL;DR: A theoretically supported TTA  paradigm that effectively address the efficiency and domain forgetting chanllenges  by aligning feature correlations.

Abstract: Deep neural networks often degrade under distribution shifts. Although domain adaptation offers a solution, privacy constraints often prevent access to source data, making Test-Time Adaptation (TTA)—which adapts using only unlabeled test data—increasingly attractive. However, current TTA methods still face practical challenges: (1) a primary focus on instance-wise alignment, overlooking CORrelation ALignment (CORAL) due to missing source correlations; (2) complex backpropagation operations for model updating, resulting in overhead computation and (3) domain forgetting. To address these challenges, we provide a theoretical analysis to investigate the feasibility of **T**est-time **C**orrelation **A**lignment (**TCA**), demonstrating that correlation alignment between high-certainty instances and test instances can enhance test performances with a theoretical guarantee. Based on this, we propose two simple yet effective algorithms: LinearTCA and LinearTCA+. LinearTCA applies a simple linear transformation to achieve both instance and correlation alignment without additional model updates, while LinearTCA+ serves as a plug-and-play module that can easily boost existing TTA methods. Extensive experiments validate our theoretical insights and show that TCA methods significantly outperforms baselines across various tasks, benchmarks and backbones. Notably, LinearTCA achieves higher accuracy with only 4\% GPU memory and 0.6\% computation time compared to the best TTA baseline. It also outperforms existing methods on CLIP over 1.86\%. Code: https://github.com/youlj109/TCA

Test-time Correlation Alignment
Linjing You
Jiabao Lu"
Xiayuan Huang"
Abstract
120
Comtettan Dictus
341
Deep neural networks often degrade under dis- tribution shifts. Although domain adaptation offers a solution, privacy constraints often pre- vent access to source data, making Test-Time Adaptation (TTA)-which adapts using only un- labeled test data-increasingly attractive. How- ever, current TTA methods still face practical challenges: (1) primary focus on instance- -wise alignment, overlooking CORrelation ALignment (CORAL) due to missing source correlations; (2) complex backpropagation operations for model updating, resulting in overhead computation and (3) domain forgetting. To address these chal- lenges, provide theoretical analysis to in- vestigate the feasibility of Test-time Correlation Aligmment (TCA), demonstrating that correlation alignment between high-certainty instances and test instances can enhance test performances with theoretical guarantee. Based on this, we pro- pose simple yet effective algorithms: Lin- earTCA and LinearTCA*. LinearTCA applies simple linear transformation to achieve both instance and correlation alignment without addi- tional model updates, while LinearTCA* serves as plug-am and-aaddaaa module that can easily boost exist- ing TTA methods. Extensive experiments validate our theoretical insights and show that TCA meth- ods significantly outperforms baselines across var- ious tasks, benchmarks and backbones. Notably, Linear TCA achieves higher accuracy with only 4% GPU memory and 0.6% computation time compared to the best TTA baseline. also out- performs existing methods on CLIP over 1.86%. Code: wtwitttu...o/oooujjllloo
200
Level
Love
Lowe?
Level
Lovel
Cocagtion Level
TIAMsbod
Sioucee Donate
Souree Domais (Bofenc)
Tegeet Domain
Suane Dorrain (A.ler)
(d
Figure 1. Illustration of key limitations in existing TTA methods. (a) Correlation distance increases with domain shifts. (b) Computa- tion time and peak GPU memory usage CIFAR- 10-C, showing high overhead of existing methods. (c) Source domain perfor- after test-time adaptation, revealing challenges in retaining source knowledge.
1. Introduction
Deep neural networks (DNNs) have significantly advanced numerous tasks recent years (LeCun et al., 2015; Jumper et al., 2021; Silver et al., 2016) when the training and test data are independent and identically distributed (i.i.d)). How- ever, the i.i.d. condition rarely holds in practice the data distributions are likely change (Fang et al., 2020; Wang & Deng, 2018). This phenomenon, known as the out-of-distribution (OOD) problem or distri- bution shift, has been extensively investigated within the context domain adaptation (DA) (You et al., 2019; Zhou et al., 2022; Liang et al., 2024). Among various DA methods, CORrelation ALigmment (CORAL) (Sun al., 2017: Sun Saenko, 2016; Cheng al., 2021a) has been proven to be an effective and "frustratingly simple" paradigm, which aligns the feature distributions of the source and target domains at a feature correlation level rather than merely aligning individual instances.
time and space
'Equal contribution 'Institute of Automation, Chinese Academy of Sciences "College of Science, Beijing Forestry University. Cor- respondence Xiayuan Huang <huangxiayuan @aaaaneefffff..c edd. Linjing You 2yyoulinjng2223@@a..m @ia..a............
Proceedings of the nd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
However, DA methods are practically difficult when pre-

---

Test-time Correlation Alignment
trained models are publicty available but the training data and training process remain inaccessible due privacy and resource restrictions (Liang al., 2024). To address such source inaccssiaeceesibeee domain shifts task at test time, test-time adaptation (TTA) (Gong et al., 2024; Su et al., 2024a;b; You et al., 2025) has emerged as a rapidly progressing research topic. Although some recent attempts have been made to handle this task, current TTA methods still face several limitations:
construct a pseudo-source correlation that approximates the true source correlation? (2) Can this enable effective ? We provide a theoretical analysis showing that align- ing correlations between high-certainty and test instances improves test-time performance with guarantees. Based on this, we propose two simple yet effective methods: Lin- earTCA and LinearTCA*. Specifically, we first compute the "pseudo-souree correlation" by using high-certainty instances. Then, Linear aligns correlation through sim- ple linear transformations of embeddings without model up- dates, resulting in minimal computation and keeping source domain knowledge. While LinearTCA* serves as a plug- and-play module that can easily boost existing TTA meth- ods.
Firstly, overlooking feature correlations: Most existing TTA methods focus on instance-wise alignment (Wang et al., 2023; Nguyen et al., 2023; Wang et al., 2020) that only cap- ture central of the instances while neglecting the correlations between features. For example, relationships between edge and texture features can vary significantly across domains. Let's consider simple test the CIFAR-10-C dataset (Hendryeks & Dietterich, 2019) to show the relationship between feature correlation and domain shift. As shown in Figure la, the correlation distance (see Section 2.2) of ResNet-18 (He et al., 2016) embedding are computed with an increasing corruption level from 1 to 5. It illustrates that as domain shifts increase, the changes in feature correlation also increase.
Main Findings and Contributions: (1) We introduce a novel and practical paradigm for TTA, termed Test-time Cor- relation Alignment (TCA). The construction of the pseudo- source correlation and the adaptation effectiveness are the- oretically guaranteed. (2) Based on our analysis, we de- velop two simple yet effective
Bathoos--LinerTTCee and LinearTCA* validate TCA's effectiveness and its plug- and-play potential with other TTA approaches. We) con- duct comprehensive experiments to validate our theoretical insights and compare performance across diverse bench- marks, backbones, and tasks, evaluating accuracy, efficiency, and resistance to forgetting. Results show that LCA achieves outstanding performance, while LinearTCA* ro- bustly boosts other TTA methods under various conditions. (4) Further in-depth experimental analysis reveals the effec- tive range of LinearTCA and provides valuable insights for future work.
Secondly, overhead computation: Current TTA methods often rely on computationally expensive backpropagation for each test sample to update models (Sun et al., 2020;
Wang et al., 2020; Goyal et al., 2022; Bartler et al., 2022). However, many applications are deployed on edge devices, such as smartphones and embedded systems (Niu et al., 2024), which typically lack the computational power and memory capacity required for such intensive calculations. As a result, backpropagation-basee based TTA methods are limited in their applicability on these edge devices. In Figure 1b, we illustrate the computation time and maximum GPU memory usage of different TTA methods on the CIFAR- 10-C dataset. Compared to the non-adaptive source model (ERM(Vapnik, 1999)), most TTA methods show a dramatic increase in both items.
2. Preliminary and Problem Statement
We briefly revisit TTA and CORAL in this section for the convenience of further analyses, and put detailed related work discussions into Appendix A due to page limits.
2.1. Test Time Adaptation (TTA)
Lastly, domain forgetting: Another drawback of backpropagation-baasee baappaggttio- TTA methods is that they often lead to model updating, which gradually loses the prediction abil- ity of the source or training domain (Niu et al., 2024; Zhang et al., 2023). As illustrated in Figure Ic, after adaptation on test domain, the performance of most methods declines when return to the source domain, indicating that existing TTA approaches struggle to retain knowledge of the source domain.
In the test-time adaptation (TTA) (Tan et al., 2024; Yuan et al., 2023) scenario. it has access only to unlabeled data from the test domain and pre-trained model from the source domain. Specifically, let D, = {(a,,, M represent the labeled source domain dataset, where (r", v.) is sampled i.i.d from the distribution Dd and n. is the num- ber of the total source instances. The model, trained on the source domain dataset and parameterized by 0, is de- noted as ha(+) = g(f(-)) X. + V., where f(-) is the backbone encoder and g(-) denotes the decoder head. Dur- ing testing, he(-) will perform well on in-distribution (ID) test instances drawn from DD., However, given set of out- of-distribution (OOD) test instances D. { ~ DD, and ID, + DD., the prediction performance of he(+) would
D.
To address the above challenges, applying the "effective and frustratingly simple" CORAL method to TTA appears intu- itive-but but the lack of source data makes it highly challeng- ing. We thus explore the fcasibility of Test-time Correlation Alignment (TCA) by posing key questions: (1)Can we
2

---

Test-time Correlation Alignment
decrease significantly. To this end, the goal of TTA is to adapt this model h-(-) to D, without access 2,.. For each instance xi Xg, let the output of encoder f()) and decoder g(-) be denoted as 2f = f f() E R and pi = ((2)) E IR, respectively, where d is the dimension of the embeddings and c is the number of classes in a classification task. When encountering an OOD test instance zi, existing TTA meth- ods (Wu et al., 2024; Sinha et al., 2023; Lee et al.. 2024: Yuan et al., 2023) typically minimize
effective and "frustratingly easy" to implement DA. Since TTA is subfield of DA, it is natural extension to apply CORAL within TTA frameworks. However, due to privacy and resource constraints in TTA, it is impossible to compute the source correlation. This limitation hinders the application of CORAL in such real-world scenarios, i.c. test-time correlation alignment (TCA),
unsupervised or self-supervised loss function to align the embedding =/ or prediction pl. thereby updating the model parameters 0:
3. Theoretical Studies
In this section, we conduct an in-depth theoretical analy- sis of TCA based on domain adaptation and learning the- ory. We focus on two key questions: (1) Can we construct a "pseudo-source correlation " to upproximate the original source correlation? (2) Can TCA based on this pseudo- source correlation enable effective TTA? Before discussing the main results, we first state some necessary assumptions and concepts. Missing proofs and detailed explanations are provided in Appendix B.
min Pl O): 2/ ~ ID, 0
where 0 [ 0 a proper subset of 0 involved in the update, such as the parameters of the batch normalization (BN) lay- ers (Schneider et al., 2020; Su al., 2024c) or all parameters. Generally, the TTA loss function [() can be formulated by nearest neighbor information (Zhang et al., 2023; Hardt & Sun, 2023; Jang et al., 2022), contrastive learning (Wang et al., 2023; Chen et al., 2022), entropy minimization (Wang
Definition 3.1. (Classification error and empirical error) Let H. be hypothesis class of VC-dimension dg. The error that an estimated hypothesis he E H disagrees with the groundtruth labeling function I X. + 3/. according to distribution Dd is defined as:
et al., 2020; Niu et al., 2022), etc.
2.2. Correlation Alignment (CORAL)
The aim of correlation alignment (CORAL) (Sun et al., 2017: Cheng et al., 2021a; Sun & Saenko, 2016; Sun et al., 2016; Das al., 2021: Rahman et al., 2020b) to min- imize the distance of the second-order statistics (covari- ance) between the source and test features. Specifically, let Z, (< E IR xd denotes the feature matrix from the source domain, and Z, = { E Ra x denotes the feature matrix from the test domain. CORAL computes the covariance matrices of the source features Z₂ and test features Z,, and aligns correlation by minimizing the Frobe- nius norm of their two covariance matrices. The covariance matrix is computed below:
e(ha, 1) = Ez [1) - [(x)
(4)
which we also refer to the error or risk e(ho). The empirical error of ho E H with respect to a labeled
dataset D, = {(xxr, ((=r",,)))]" v' ~ D, is defined as:
E(ho)
11 Σ (ho - v/l
1
(5)
Assumption 3.2. (Strong density condition) Given the parameters f(,,, ,nt,, ,,,, Cr.re >> 0, we assume that the distribution DD, and Dj are absolutely continuous with respect to the Lebesgue measure ^[-] in Euclidean space. Let B(x, r) {ao : ||e0- =1 S r} denote the closed ball centered at point x with radius r. We further assume that A x; D, and E (0,r,), the following conditions hold:
Σ n
--
1,
n
the correlation distance is then given by (Sun & Saenko, 2016):
d(Es, Ex) = 4d2 1
IE, E.
(3)
ALB n BBz,, r)] 2 cx(B(xr)))
(6)
where E. and E3 are the covariance matrices of the source and test domains, respectively, and 1 is a column vector with all elements equal to 1 to perform mean-subtraction. ||. II represents the Frobenius norm.
[ID. r) (\(((r,r)))
(7)
OD p² < ax < n+: /
OLD ax
< 4
(8)
2.3. Problem Statement
Existing TTA methods suffer from overlooking feature cor- relation, overhead computation and domain forgetting. Re- search and practice have demonstrated that CORAL both
The strong density condition is commonly used when ana- lyzing KNN classifiers (Audibert & Tsybakov, 2007: Cai & Wei, 2021). Recently, it has also been applied in the
3

---

we have Test-time Correlation Alignment
test-time adaptation (Zhang al., 2023). Intuitively, As- sumption 3.2 requires that the divergence between ID, and Dr is bounded. When e 1, for each 2; ~ DR the neigh- borhood ball ((x,,r) r) is completely contained within D,. In contrast, when Q₁- = 0, r)), and DD, are nearly disjoint. Assumption 3.3. (L-Lipschitz Continuity) Let ho(.) = v((()) be estimated hypothesis on H. We assume that there exists a constant L such that 21,22 E D. U Dr, the encoder f() satisfies the following condition:
where Tdz A(3 1)) is the volume of the dj dimension unit ball and dj is the dimension of in- put I. Furthermore, considering the true source correlation M; = EiZ, T Z.] and the pseudo-source correlation ΣS Z. Zz. where Z, and Z, are centered. With probability of at least min(1 exp (- (-(aw (a,"TT,,"" "/,, + log 11160 6), the cor- 20sp "asp"N,,"" Na,
relation distance ||E, Σell is bounded by:
|Σ -EallE S 18, Pala [[
[ - f(x2)1 & 21 - -x2ll
(9)
)+ 18, 1& Polle + A ) B
21|2,1/e(
[1 (12)
The assumption of L-LLipschitz continuity is frequently em- ployed in the analysis of a model's adaptation capabilities (Mansour et al., 2009). It implies that the change rate of f(-) does not exhibit extreme fluctuations and is bounded by the constant L at any point.
where Ve is the one-hot encoding of Pt. A = [lo(kr")]] + ke(ho((,)) + ke(ho represents the output error of the sampled instances, and B = log(2/6) is the sampling error. 2k
Assumption 3.4. (Taylor Approximation) Let h- g((()) be L-Lipschitz Continuous hypothesis on H. = (xx) and p g(2). We assume that there exists a constant r* such that V I1, E D. U Dr, if || - 22ll S r", P2 = ( can be approximated using the first-order Taylor expansion at 2 as follows:
Theorem 3.5 implies the followings: (1) In Eq. (12), the terms X., Z. and Jy(Z,) remain unchanged with the same source data. The primary factor influencing the correlation distance |E,,- Σ.Il is prediction uncertainty ||,, P:llr and output error of the sampled instances c(ho(X.)) (2) In- tuitively, previous studies (Gui et al., 2024; Niu et al., 2022; Yuan et al., 2024) empirically suggest that instances with higher output certainty have less output error. other words, with a smaller divergence between the prediction Pt and its one-hot encoding Y,, both uncertainty YY, Pt F and error (X,)) will decrease, resulting distance. (3) Therefore, a reasonable pseudo-source con- struction method is to select the k test instances with the smallest || - Pellr values (i.e. high-certainty test in- stances) and compute their correlation matrix as pseudo- source correlation.
P2 =1 + P (2) 2 + ol1 =2)) (10)
where P1 = )11), J (2)) is the Jacobian matrix of g evaluated at 21, and of 2ı - =2l represents the higher- order terms in the expansion.
It indicates that when the outputs 21 and Zg are close (i.e., their distance is within the radius r"), the decoder can be well-approximated by a linear function at 21.
smaller correlation
3.1. Correlation of high-certainty test instances approximates the source correlation
We characterize the divergence of correlation between the pseudo-soute source and the source correlation in the following Theorem 3.5.
3.2. Test-time correlation alignment reduces test classification error
be L-pischhitz continuous hypothesis on H. Ω = ULED, B(x,r* is the set of balls near the test data. We sample k source instances from D. S and l test instances from Dj to obtain [ Z. Pa] and [X,; Ze, Pi] by hg(:), respectively. Per Assumption 3.2, Assumption 3.3 and Assumption 3.4, with a probability of at least (ora Ha, "1,,=1n,-11" 20s4 Ray "In.
Theorem 3.5. Let hothhh((=
In this section, we establish the TTA error bounds of hypoth- esis he when minimizing the empirical error in the source data (Theorem 3.6) and examine the influence of using the pseudo-source correlation (Corollary 3.7), which further indicates factors that affect the performance of hg.
Theorem 3.6. Let H. be hypothesis class of VC- dimension d.y- If h E H minimizes the empirical error (a) D., and hū arg minneH Ex(h) is the opti- mal hypothesis on D,, with the assumption that all hy- potheses are L-Lipschitz continuous, then V8 E (0, 1), with probability with at least 5 the following in- equality holds:
1 exp
+ log k).
Pt - Pall + !o(kr*))l [2,(
12: - 12-12115
(11)
4

---

Test-time Correlation Alignment
Test data
Emshedings
Original
Prediction
((h) S (e(h))) + O(v
-e + | - Exll,) + C /114 1111
Ercodd
podd
Px
Ps
where C 2/ d. 4.10g(2n,)= + 27 and 7 Ma. Mt, E, and Σ, de- note the means and correlations of the source and test embeddings, respectively. We use O(-) to hide the constant dependence.
2n,
Prediction Certainty
minne (e. + ex (h(())))
inverTCA TCA Prediction
Transtormed Embesings
Linser
Seesdo-Source
Trunsfom
R2
For fixed Dg and Dr. G (h,)) and C' are constants, indicating that the primary factors affecting the performance of hg on the test data D, (i.c., (f)) are I| M4 ll: and ||_,- - E.ll ll By aligning correlations during testing, which means reduc- ing |Σ, - Σ₁ |/ we can effectively decrease the model's ss classification error on the test data. Combining Theorem 3.5 with Theorem 3.6, the following corollary provides a di- rect theoretical guarantee that TCA based on pseudo-source correlation can reduce the error bounds on test data.
Figure 2. ing testing, we first obtain original embeddings and predictions using the source model. Based the certainty of the original pre- dictions, we select subset embeddings to form a "pseudo-source domain". A linear transformation then applied to align the corre- lations of the original embeddings with those of the pseudo-source domain, ultimately producing the final predictions of LinearTCA. Notably, this process does not require updating any parameters of the original model.
The pipeline of proposed LinearTCA method. Dur-
Corollary 3.7. Let g. E.g and En denote the source, pseudo-source and test correlation, respectively. The- orem 3.5 establishes the error bound between En and Eg. while Theorem 3.6 demonstrates that reducing the difference between En and Egg can decrease classifica- tion error on the test data. By applying the triangle inequality, we have:
Remark. Section 3.1 answers the first question that the feature correlation of high-certainty test instances from the pre-trained model can approximate the feature correlation of the source domain. Section 3.2 provides a theoretical guarantee that conducting correlation alignment between pseudo-source correlation and test correlation during TTA can effectively reduce the test error bound. These theoretical findings are further validated in Section 5.2.
[E EElF EE, ES + -. EEllF S
12 E. + +I E (13)
4. The Test-time Correlation Alignment Algorithms
Therefore, Theorem 3.6 can be rewritten as:
((h) 5
As illustrated in Figure 2, building our theoretical find- ings, we propose two simple yet effective TCA methods: LinearTCA and LinearTCA*. We start with detailing the construction of the pseudo-source correlation, followed by the implementation of LinearTCA and LinearTCA*.
e<(hi) + O(v /lluu - - + IE, - Ex11-)+ + C
<((h)) + 0(111 mell + +2122 18p-Pello+ Pill +
11,g(2.)1e
+ [1 [1 - Pelle + +A 2 B E3 )2)1)) +C
4.1. Pseudo-Source
For each instance æi arrives in test time, we first get em- bedding 2t f(x) and prediction pf = g(2f). Per The- orem 3.5, compute its prediction uncertainty of
(14)
Corollary 3.7 indicates the followings: (1) Reducing the correlation distance between the test data and the pseudo- source, i.e., Ex 1.?. can reduce the test classification eгror. The pseudo-source correlation E. is computed by selecting k instances from the test data with minimal un- certainty, measured by I|Ve 1.. (2) Updating model parameters to decrease ||Y- P.ll?- can further reduce the test еттоr. (3) Additionally, minimizing the instance-wist distance l|f 14m 12 can also contribute to reducing the test error, which is consistent with previous studies (Niu et al., 2022; Wang et al., 2023; 2020).
O 1 where of onchot (argmazz))g) We then temporarily store the pair w4) in the Pseudo-Source bank M = M U wf). Subsequently, M is updated based on its element count and confidence. The update rule is as follows:
M [.M ((()) wa S we whih
(15)
else
where w represents k-th lowest uncertainty value in M. Finally, the Pheudo-Souree correlation can be calculated as
5

---

Test-time Correlation Alignment
follows:
Monoon 122 55
Averge Uncertiiny
Σ, 1
2 2g
1 1n, 2T 2 (n.))
(16)
3g
ng -1
îgg
34
where 2, = [zllet E MM and â. = M.
01
10
4.2. Methods
170
LinearTCA: During testing, given the embeddings Ze and Z. sampled from the test and pseudo-source domains, re- spectively, our objective is to minimize their correlation distance:
Rloq p-1.00
p=1100
180
Corrdutiio Distnes
the Scurco [X,- X.1
153
H4
||2:0- Σلg
LincarTCA
(17)
Corriation Distance the Pscudo Source [2: E,1 (b)
Figure 3. Experimental validation of theories. (a) Average uncer- tainty and correlation distance source domain of each group, groups with lower uncertainty exhibit smaller correlation distances. (b) Relationships between ACC, correlation distance the source, and correlation distance to the pseudo-sourge both ACC and [ Σ, are strongly linearly related |[ E., ||-
To achieve this alignment, we aim obtain suitable linear transformation W as follows:
WW E
min W
(18)
Setting WTEW = E. and applying eigenvalue decompo- sition, the closed-form solution for W can be derived
5. Experiments
W U,A
(19)
5.1. Experimental settings
Following previous studies, we evaluate the adaptation performance on two main tasks: domain generalization (PACS (Li et al., 2017), OfficeHome (Venkateswara et al., 2017), and Doma (Peng al., 2019) dataset) and image corruption ImageNet-C (Hendrycks & Dietterich, 2019)). What's more, we also evaluate our method on multimodal tasks based on CLIP (Radford et al., 2021). The comparison meth- ods include ba backpropagation fre (BN (Schneider et al., 2020), T3A (Iwasawa & Matsuo, 2021), AdaNPC (Zhang al., 2023)) and backpropagat. ionpoaaatt methods (TENT (Wang et al., 2020), PLC (Lee, 2013), EATA (Niu et al., 2022), SAR (Niu et al., 2023), TSD (Wang et al., 2023), TIPI (Nguyen et al., 2023), TEA (Yuan et al., 2024)). Backbone networks include ResNet-18 50 (He et al., 2016) and ViT-B/16 (Dosovitskiy, 2020). Addi- tionally, the evaluation encompasses multiple aspects, in- cluding accuracy, efficiency, and resistance to forgetting. For LinearTCA' report its results combined with the best baseline. Refer to Appendix D) for more implement information. For further experimental results and analysis, please see Appendix E.
where U, and U, represent the eigenvector matrices, A, and A, are the corresponding diagonal eigenvalue matrices, re- spectively. The transformed embeddings of the test domain can then be computed as:
10-C,CIIACCCCR-- 100--00000 and
zz = (Z₁ (c) W + +
(20)
where be and AS denote the mean embeddings of Z, and 7, respectively. As shown in Eq. (20), we also align the instance-wise shift |4o- μul by using åو. Finally, the predictions for the test domain after adaptation through LinearI'CA are:
g(Z,)
(21)
LinearTCA*: Since LincarTCA does not require param- eter updates the model, can serve as a plug-an and-play boosting module for TTA methods. Specifically, during TTA method optimizes the original model he hg via Eq. (1), we can obtain the resulting embeddings ZTTA and predictions PITA- By applying the LinearTCA on ZTTA and PITA with the same process from Eq. (15) to (21), the predictions of LinearTCA* are obtained. More details on these methods are provided in Appendix C.
5.2. Experimental validation of theories
'To enhance the robustness of the results, we recommend using torch's automatic gradient descent method to mitigate potential instabilities associated with eigenvalue decomposition. For the following experiments, implement this method with fixed learning rate of le-3.
For Theorem 3.5: Correlation of high-certainty test in- stances approximates the source correlation. We divide the test embeddings of CIFAR-10-C under ResNet-12 into 10 groups based on prediction uncertainty and caleu-
6

---

Test-time Correlation Alignment
Domain
Method
ResNer-18
ResNet-50
PACS
VIT-/166
AVG
ResNer-18
ResNet-90
OfficHHme
VIT-8/16
AVG
ResNet-18
ResNer-501
DomsinNel
VIT-/16
AVG
AVG
SOURCE
81.84
84.78
87.02
84.54
62.01
67.01
76.11
68.37
39.15
43.58
50.29
44.33
65.75
BN
82.65
$4.99
66.30
37.93
BP-Free
T3A
83.20
85.71
88.06
85 35.66
63.26
67.85
78.87
69.99
40.62
44.92
53.94
46.49
67.38
AdaNPC
83,48
86.12
19.11
86.24
62.88
67.05
77.26
69.07
40.50
45.17
53.28
46.32
67.21
TENT
58.07
84.98
86.09
67.67
76.95
43.97
68.65 48.48
PLC
K3.16
86.59
87.97
X5:91
62.22
66.44
76.51
68.39
37.96
41.63
47.29
42.30
66 53
EATA
83.30
81.68
86.60
84.86
62.49
67.01
76.98
68.83
11.65
46.89
51.40
47.65
67.11
BP-Based
SAR
12
76.66
TIPI
$7.39
88.01
87.98
87.79
63,25
68.36
77.09
69.57
36.05
44.08
39.70
39.94
63.77
TEA
87.19
89.75
87.37
87.77
63.43
68.56
76.15
69.338
39:43
43.48
48.41
43.78
66.98
TSD
87.83
83.43
66.
Ouns
LCC Line=TCA*
83.59
86.78
61
86.35
63.66
68.43
78.26
70.12
40.79
44.89
52.79
46.16
67.55
88.77
90.68
89.30
89.58
64.27
69.32
79.02
70.87
42.20
47.17
55.49
48.29
69.58
ImgCop
Method
R8
ResNei-50
CIFAR-10-C
VIT-8/16
AVG
ResNer-18
ResNet-50
CIFAR-TOO-C
VIT-8/16
ResNei-18
ResNer-50
ImageNes-C:
VIT-8/16
AVG
AVG
SOURCE
30.80
50,77
71,48
37.65
31.01
34.02
51.71
38.91
11.70
18.13
39.83
21.23
10.27
BN
73.70
72.24
48.38
48.41
27.59
32,06
T3A
54,87
74.21
32.52
14.56
18.05
24.13
AdaNPC:
$7.72
54.75
74.60
62. 36
29.70
32.27
53.21
38.39
11.95
15.62
36.78
21.44
40.73
TENT
75.21
72.33
71.48
73.01
50.82
50.12
52.72
51,22
35.39
41.32
48.01
41.57
55.27
PLC
73.72
72.34
71.46
72.51
48.35
48. 48.38
51.71
49.48
27. 59
32.06
38.74
32,80
51 $1.59
EATA
73.86
72.38
73.67
73.30
49.71
49
62.40
$4,00
19. 19
48.17
64.36
50:58
59.29
BP-Based
SAR
73.97
73.37
71.48
72.94
51.60
50.25
$4,29
52.05
38.55
46.30
57.94
47.66
57.55
TIPI
72.46
71.48
73.35
50.30
52.36
41.87
55.49
TEA
76.20
72.54
71:48
73.41
50.67
30.21
52.31
$1.06
32. 38
38.90
41.37
37.55
54.01
TSD
76.93
73.23
71.47
73.88
49,35
49.60
51,74
50.23
30.11
35,08
41,33
35.51
53.20
Ours
Linear ICA"
TCA
77.13
60.96
60.27
73.53
77.26
79.55
66 16
76.74
52.08
35.03
37.28
51.17
63.71
55,42
42.58
55.65
39.21
16.07
48.22
19.34
64.71
41.37
50.71
25.60
44.78
61.04
Table 1. Accuracy comparison of different TTA methods based on ResNet- 18/50 and ViT- B/1 hackhones. The upper part of the table corresponds to the domain generalization task, while the lower part corresponds to the image corruption task. The best results are highlighted in boldface, and the second ones are underlined. "_" indicates that ViT-B/15 6 does not include any BN layers.
late the correlation distance between each group and the original source. As shown in Figure 3a, groups with lower uncertainty exhibit smaller correlation distances, indicating closer approximation to the source correlation.
substantial improvements over the best best-performing base- line, with an increase of 4.95%. (2) Across datasets, LinearTCA shows robust improvement compared to the source model, with average gains of 1.79%, 1.75%, 1.78%, 8.48%, 3.67% and 4.51%, respectively. Particularly, on the Of f ceHome and Doma i neet. dataset, LinearTCA out- performs all baseline methods. However, on datasets such as CIFAR-10/ 100Aa--0000---0 1100-C and ImageNet-C, although Lin- earTCA yields ACC gains of 8.48%, 3.67% and 4.51% % over the source model, it falls short of some advanced methods. (3) Across backbones, LinearTCA also demon- strates robust improvements compared to the source model, especially with the ViT-B/1 6 backbone, surpassing the perfoss-performing baseline on most datasets. We provide a detailed analysis of these experimental results in Section 5.5 to further reveal the strengths and limitations of LCn.arTCA.
For Theorem 3.6 and Corollary 3.7: Test-time correlation alignment reduces test classification error. We iteratively op- timize W and record the correlation distances between test domain and pseudo-source domain, |Σ, -Ell. as well as the true distances between test domain and source domain, [M E.Il. and ACC. As shown in Figure 3b, under linear fit (R² = 0.97), |E, - E.1 lI is strongly positively related to [>, E. (Spearman correlation coefficient = 1). Under R² = 0.96, it is strongly negatively related to ACC (Spear- man correlation coefficient -1). This further validates that pseudo-source correlation alignment promotes alignment with the original source. Additionally, pseudo-source corre- lation alignment effectively reduces test classification error, thus improving the model's domain adaptation capability.
Tym
Mitted
MareviMM
2010
3N
25
-45.7
-2.5
+4.00
+2.00
1350
10.07
-1.50
5.3. Comparison with TTA Methods
+0034.14
96
4511545
-504
-4.51
-25.6
43339
+11177.6
87 Band
$48
+3602.5 02
-5300
+003175
-11.5
42531
-5006
+3855
Accuracy. Table 1 presents ACC comparisons between TCA methods and state-of-the-art TTA approaches across various benchmarks, backbones, and tasks. (1) As a plug- and-play module, LinearTCA* consistently enhances per- formance across all datasets and backbones, achieving a new state-of-thee-rt. Notably, on the CIFAR-10-C dataset with the ViT-B/16 backbone, LinearTCA* shows
150
-15311
+038810
+5189
+13/7
-3.00
4130.0
11F
.35260
-1992
+175273
-9975
+015)
Table 2. Maximum GPU memory usage and running time of dif- ferent TTA methods on CIFAR-
Efficiency, We evaluate each method's efficiency in terms of peak GPU memory usage and total runtime. Table 2 re- ports results on the CIFAR-10- 110-C dataset across different
7

---

Test-time Correlation Alignment
Type
Mataed
RMCS
10
Obfeciene
CHRAR-I--
CIRAR-IMGC
NT.5
AVG
Method
PACS
AVG
B3
963060 641
6255
6209 (22.380
73.55 (-1558
A
C
P
S
Free
TEA
52.561-130
CLIP
97.44
97.38
99.58
86.06
95.12
41:66:11000 300
(7.35
07.35
87.85-110
TENTY
97,54+0.02
97.37=00.04
99. 99.58=0.00
86.37++0.05
95.22
LlverTCA wil
99.35 00.00
94.40
(8.00
92.36 00.00
70.29 (8:09)
(8.12
TPT:
95. 05:10:4:41
91.42=0.22
98, 98,56=0.40
87 87.23-+0.06
93,08
TENT
EATA
PLC
05.12 254
26744-2013
13333-1000 021
00:33-100 (7
03.66-0 4.54
90.36.2.00 10
50.3422133 153
08.52 (-137)
39 08 (.5091)
73 (-1603
WATT-PI
CLIPArTTT
97.64+0.02
97.49110.08
91.37=00.02
97.47.10.04
99, 99.58=0.00
99,58
89.73.10.16
86.79+40.04
95.35
96.07
BP-SSaell
TEA
T5D
00.33 1/ 131
66 1610044500
05.30
03.30-1100 100
40.0.-1700 76
67.79
08 11.45
08.77
45 13776133 (3.90
LinearTCA
WATT-SF
97.66:10.088
97.80
97.51-1.02
99.58-00.00
89.56:10.14
92.32
97.36
96.08
TIP)
58. 15 6- 200
00.70
02:79-1100 40
76.95 1:1200
Lin
97.87100066
99.20:10.02
9.094
92.36:11.06
97.34
190334330
93.55:4.75
75
61.88
(1.34)
Method
OffecHooe
AVG
Table The accuracy of different TTA methods when retur-- ing to the source domain after adaptation. "BP-Free" indicates backpropagation--ii TTA methods, while "BP-Based" denotes
A
R
CLIPt
79.30
65.15
87.34
89.31
TENTY
79.26.10.14
65.64.10.05
87.49__0.02
89.50.10.04
80.47
TPT:
819740.17
67.01=0.21
89.00=0.06
89.66:20.06
backrroagationn-dpenne ones.
CLIPACTTf
79.344:005
63.69=0.11
87 87.35=0.07
89.29:40.03
80.42
WATT-Pf
80.374-0.25
68.59--0133
15-15-0007
90. 98-18-+0.03
81.82
WATT-SF
80.4340.09
68.26+ 11
88 18.02=00.08
90. 14:00.066
81.71
LinearTCA
85.55
68.70
90.26
90.58
83.77
backbones. TCA consistently achieves the lowest memory and time cost. For memory, since we record peak mem- ory consumption, LinearTCA exhibits minimal inddpen- dent memory usage (as shown in Table 3) and thus does not impose additional memory constraints on the device.
LinearTCA'
8..62
69.25 10.1
90.29 0.011
90.42.10.1
83.90
VLCS
Method
AVG
V
CLIP
99.43
67.75
71.74
84,90
80,96
TENTY
99.43-10.00
67.31
71.57=0015
85.10:1.111
80.85
TPT:
97.62:10.12
49.77:10.03
0..56.0086
7117110.70
72.53
In contrast, other meth- ods are embedded within the model's forward and backward propagation processes, significantly increasing peak memory
CLIPArTTT
99.43:1:.000
67.74=0.10
71.67=0.01
84.73:20008
80.89
Meded
Heshall 118
Jesbapo VisionS
AVD
WATT-Pi
99.36.10.00
67.55.1.39
74.75-0007
82.5310.10
81.05
WATT-Sf
99.36::0.00
68.59=0.25
75.16=0.12
83.24:0005
81.59
Linea/TCA
99.86
73.98
78.47
84,41
84.18
Table 3. Independent maximum GPU memory usage of Lin- earTCA on CIFAR-1O-C.
LinearTCA'
99.58
74.39-0.11
79.44-0.22
84.0640.14
Table S. The accuracy comparison of different methods on PACS, OfficeHomee, and VLCS datasets using CLIP-ViT-B/16 t: num- bers are from WATT (Osowiechi et al., 2024). The best results are highlighted in boldface, and the second ones are underlined.
usage (e.g.. TEA uses 15x the memory of the source model). For runtime, with a ViT-B/16 6 backbone, LinearTCA requires only 0.6% of AdaNPC's time. These results highlight LinearTCA's high efficiency, making it well-suited for resource-conssraiied edge deployment.
both variants perform similarly, suggesting that even simple correlation alignment can notably enhance performance on popular models like CLIP. This underscores its effectiveness as a versatile plug-and-play module for improving diverse adaptation methods
Forgetting resistance. Table 4 shows the change in ac- curacy when each method (using ResNet- 18)) returns to the source domain after adaptation. "LineerTCCA w/o W refers to the variant without the linear transformation, which is equivalent to the original source model and thus retains full source knowledge. Despite applying W. LinearTCA demonstrates much stronger resistance to forgetting than other methods--speeciall on PACS, where it even improves source performance, showing positive backward transfer. Moreover, LinearTCA* further enhances the forgetting ro- bustness of existing TTA methods.
5.5. Analysis
Effective range of LinearTCA. As discussed in Section 5.3, although LinearTCA* significantly improves all TTA meth- ods, LinearTCA only achieves SOTA performance on part of datasets and backbones. The reasons may be: 1) Al- though the highest-certainty embeddings are selected as pseudo-source domains, if these embeddings still exhibit substantial differences from the true source domain (or if the backbone's ' feature extraction capacity is insufficient, e.g.. ResNet-18 8 vs. ViT-B/16) the performance ceiling of LinearTCA is limited. In contrast, other TTA methods up- date the model, thereby raising this ceiling and facilitating easier correlation alignment for LinearTCA*. 2) We only use a linear transformation W for alignment, which may work well for simple shifts; however, the true distribution shifts may not conform to linear transformations but ex- hibit complex nonlinear relationships. We design demo experiment to validate this hypothesis. In Figure 4a and b, the test domain shifts are linear and nonlinear, respec- tively. As shown, the transformed embeddings in Figure 4a align well with the original distribution, while the perfor-
5.4. Performance on Closed-Source Foundation Models
To validate TCA's effectiveness on closed-source founda- tion models, we conduct experiments with CLIP (Radford et al., 2021) on PACS, OfficeHome, and VLCS datasets, fol- lowing the experimental setup in WATT (Osowiechi al., 2024). As shown in Table TCA achieving performance improvements of 1.28%, 2.08%, and 2.85% on the three datasets respectively. The superior results stem from our method's explicit alignment of embedding distributions with the source domain, which proves particularly effective for multi- modal models like CLIP that compute image-text sim- ilarity directly. While LinearTCA* holds a slight advantage,
8

---

Test-time Correlation Alignment
Lincar Noulineur Shitt
Badbone
Metiol
PACS
AVG
OficcHone
c:
3
C
K
Seuace
77.39
56.58
02
62.01
LiteuTCA
11.89
88.30
87:07
06.10
61100
88
99.15
41 16
71.30
11.29
6361
TCA
96.18
83.67
50.51
96
58.69
72.04
12.03
TCAbs
$1.59
$1.48
96.05
77.51
54.15
59.54
51.6
72.50
13.41
6435
Lites:TCA (6)
$5.98
$5:37
96.34
83
30
99.15
10.09
52.99
72.55
13.87
64.6
Table 7. Upper performance bound for TCA. TCA(a): Fine- tuning directly the target distribution. LinearTCA(b) and
devel
30
LincarTCA (b)): Applying LincarTCA and LinearTCA' with real source distributions.
Accurayy (Ng.
Mitiel
XVG
Dicruatoneme
3543
3414
3654
3630
2415
3437
3433
3415
2413
343
3627
3433
18:01
4376
4111
56.54
56.11
57.35
58.55
57.11
5732
17.99
46.36
URCSTCA
5RA1
58.17
58.7
8894
9886
13.03
23.35
23.5
5936
Table 8. Accuracy comparisons of different TTA methods the Art domain of OfficeHome dataset with varying batch sizes based ResNet-18. The best results are highlighted in boldface, and the second ones are underlined.
Number of Poeedo--Soreeeee Sonrce Instanoes k.
Numba of Sampling Teat
10
Figure 4. Analysis of TCA. (a) When the test domain (yellow) undergoes nearly linear shift from the source domain (blue). after adaptation by LinearTCA, the transformed lest domain (red) well-alignes with the source. (b) In the case of nonlinear shift, although partial alignment is achieved, it is still insufficient. (c) and (d) Ablation study examining the effect of pseudo-source domain size and test domain size.
inal LinearTCA in most domains. On OfficeHome, even the simpler LinearTCA with real source data (b) surpasses fine-tuning (a), highlighting the importance of source distri- bution and the effectiveness of approximating it in TCA.
Batbbon
Metiod
PACS
AVG
Oliatias:
AVG
Performance under difference batch sizes. To study the impact of batch size, we evaluate TCA's performance and pseudo-source estimation error under varying batch sizes in Table 8. Even with batch size 1, LinearTCA outperforms the source model by 2.16%, and LinearTCA* consistently improves over TEA across all settings. This robustness stems from TCA's incremental estimation of test-domain covariance, which converges over time. While small batch sizes mainly affect early predictions, their influence dimin- ishes as more data is seen. Moreover, the pseudo-souree estimation error remains unaffected by batch size, since it relies on small set of high-confidence samples (Figure 4c) and benefits from the same incremental computation.
73.17
77:3 34
05:00
$1:00
5::15
4810
11.34
13.31
(2:)
S001
91.22
$3.99
11.7%
13:65
9:31
13.79
Ra
13.34
H1.73
95.95
11.5
9151
1140
12.6
3110
11.0
Table 6. Extending Chear/CAAL.inea/T/ based transformations with two (MLP-2) and three (MLP-3) layers The best results are highlighted in boldface, and the second ones are underlined.
by introducing MIP-
mance in Figure 4b shows partial alignment which is still insufficient. We further explore the utilization of nonlinear architecture (MLP) for calculating transformation W. As shown in Table 6, incorporating nonlinear activations with deeper architectures leads to further improvements.
6. Conclusion and Future Work
Ablation study. Our method involves only one hyperparam- eter- the number of pseudo-soures source embeddings k. Since the total number of test samples is often unknown in prac- tice, we also sample 2o2 embeddings from the test set to study its impact. As shown in Figure 4c,d, LinearTCA achieves the best accuracy on Off cecceeee when k = 10 and hn2 = 2400. Importantly, it consistently outperforms the source model across a wide range of l and hi, demonstrat- ing strong practical applicability.
In this paper, we introduce the Test-time Correlation Align- ment (TCA) to address the chanllenges in Test-Time Adap- tation (TTA), such as overlooking feature correlation, over- head computation and domain forgetting. TCA is novel paradigm that enhances test-time adaptation (TTA) by align- ing the correlation of high-certainty instances and test in- stances and is demonstrated with a theoretical guarantee. Extensive experiments validate our theoretical insights and show that TCA methods significantly outperforms baselines accuracy, efficiency, and forgetting resistance across var- ious tasks, benchmarks and backbones.
Upper performance bound for TCA. To assess the upper bound of TCA, we conduct two additional experiments in Table 7: (a) fine-tuning directly on the target domain; (b) applying LinearTCA and LinearTCA* with real source dis- tributions. Compared to the original LinearTCA*, approach (b) further improves performance, by 0.38% on PACS and 1.03% on OfficeHome, Both (a) and (b) outperform the orig-
Future work may incorporate more nonlinear transforma- tions for more effective TCA. Additionally, with the inter- esting "positive backward transfer" phenomenon in Table 4, we will further investigate the underlying mechanism.
9

---

Test-time Correlation Alignment
Impact Statement
Goyal, S., Sun, M., Raghunathan, A., and Kolter, J. Z. Test time adaptation via conjugate pseudo-labels. Advances in Neural Information Processing Systems, 35:6204-6218, 2022.
This paper presents work whose goal to advance the field of Machine Learning. There are many potential societal consequences of our work, none which feel must be specifically highlighted here.
Gretton, A., Borgwardt, K., Rasch, M., Schölkopf, B., and Smola, A. A kernel method for the two-sample-pae proplee-roolem.. Advances in neural information processing systems, 19, 2006.
References
Arjovsky, M., Chintala, S., and Bottou, L. Wasserstein gen- erative adversarial networks. In International conference on machine learning, pp. 214-223. PMLR, 2017.
Gui, S., Li, X., and Ji, S. Active test-time adaptation: Theoretical analyses and an algorithm. arXiv preprint arXiv:2404.00 050044 2024.
Audibert, J.-Y. and Tsybakov, A. B. Fast learning rates for plug-in classifiers. 2007.
Hardt, M. and Sun,
Test-time training on nearest neighbors for large language models. arXiv preprint arXiv:23000 18466, 2023.
Bartler, A., Bühler, A., Wiewel, F., Döbler, M., and Yang, B. Mt3: Meta test-time training for self-supervised test- time adaption. In International Conference on Artificial Intelligence and Statistics, pp. 3080-3090. PMLR, 2022.
He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learn- ing for image recognition. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 770-778 778, 2016.
Cai, T. T. and Wei, H. Transfer learning for nonparametric classification: Minimax rate and adaptive classifier. 2021.
Hendrycks, D. and Dietterich, T. G. Benchmarking neural network robustness to common corruptions and pertur- bations. In 7th International Conference on Learning Representations. ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019.
Chen, D., Wang, D., Darrell, T., and Ebrahimi, S. Con- trastive test-time adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 295-305, 2022.
Iwasawa, Y. and Matsuo, Y. Test-time classifier adjust- ment module for model-agnostic domain generalization. In Proceedings of the 35th International Conference on Neural Information Processing Systems, NIPS '21, Red Hook, NY, USA, 2021. Curran Associates Inc. ISBN 978171388 181713845393.
Cheng, Z., Chen, C., Chen, Z., Fang, K., and Jin, X. Robust and high-order correlation alignment for unsupervised domain adaptation. Neural Computing and Applications, 33:6891-6903, 2021a.
Cheng, Z., Chen, C., Chen, Z., Fang, K., and Jin, X. Robust and high-order correlation alignment for unsupervised domain adaptation. Neural Computing and Applications, 33:6891-6903, 2021b.
Jang, M., Chung, S.-Y., and Chung, H. W. Test-time adapta- tion via self-training with nearest neighbor information. arXiv preprint arXiv:2207. 10792, 2022.
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Zidek, A., Potapenko, A., et al. Highly accurate protein structure prediction with alphafold. nature, 596(7873):583-58) 2021.
Das, T., Bruintjes, R.-J., Lengyel, A., van Gemert, J., and Beery, S. Domain adaptation for rare classes augmented with synthetic samples. arXiv preprint 10. 12216, 2021.
Dosovitskiy, A. An image is worth 16x16 words: Trans- formers for image recognition at scale. arXiv preprint arXiv:2010. 1929, 2020.
LeCun, Y., Bengio, Y., and Hinton, G. Deep learning. nature, 521(7553):436 444, 2015.
Lee, D.-H. Pseudo-label : The simple and efficient semi- supervised learning method for deep neural networks. ICMI. 2013 Workshop Challenges in Representation Learning (WREPL), 07 2013.
Fang. T., Lu, N., Niu, G., and Sugiyama, M. Rethinking importance weighting for deep learning under distribution shift. Advances in neural information processing systems, 33:1 1990- 2020.
Lee, J., Jung, D., Lee, S., Park, J., Shin, J., Hwang, U., and Yoon, S. Entropy is not enough for test-time adap- tation: From the perspective of disentangled factors. In The Twelfth International Conference on Learning Repre- sentations, 2024.
Gong, T., Kim, Y., Lee, T., Chottananurak, S., and Lee, S.-J. Sotta: Robust test-time adaptation on noisy data streams. Advances in Neural Information Processing Systems, 36, 2024.
10

---

Test-time Correlation Alignment
Li, D., Yang, Y., Song, Y.-z., and Hospedales, T. M. Deeper, broader and artier domain generalization. In Proceedings of the IEEE international conference on computer vision, pp.
Rahman, M. M., Fookes, C.,, Baktashmotlagh, M., and Srid-
haran, S. Correlation-aware adversarial domain adapta- tion and generalization. Pattern Recognition, 100: 100:107124, 2020a.
2017.
Liang, J., He, R., and Tan, T. A comprehensive survey test-time adaptation under distribution shifts. Interna- tional Journal of Computer Vision, pp. 1-34, 2024.
Rahman, M. M., Fookes, C., Baktasshmotlaghhh M., and Srid-
haran, S. Correlation-aware adversarial domain adapta- tion and generalization. Pattern Recognition, 100: 107 124, 2020b.
Mansour, Y., Mohri, M., and Rostamizadeh, A. Domain adaptation: Learning bounds and algorithms. arXiv preprint ar arXiv:0902.3430, 2009.
Schneider, S., Rusak, E., Eck, L., Bringmann, O., Bren- del, W., and Bethge, M. Improving robustness against common corruptions by covariate shift adaptation. Ad- vances in neural information processing systems, 33: 11539-11551, 2020.
Nguyen, A. T., Nguyen-Tang, T., Lim, S.-N., and Torr, P. H. Tipi: Test time adaptation with transformation invariance. In Proceedings of the IEEF/CVE Conference on Computer Vision and Pattern Recognition, pp. 24162- 24171, 2023.
Shen, J., Qu, Y., Zhang, W., and Yu, Y. Wasserstein distance guided representation learning for domain adaptation. In Proceedings of the AAAI conference on artificial intelli- gence, volume 32, 2018.
Niu, S., Wu, J., Zhang, Y., Chen, Y., Zheng, S., Zhao, P., and Tan, M. Efficient test-time model adaptation with- out forgetting. In International conference on machine learning, pp. 16905, PMLR, 2022.
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., Van Den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam. V., Lanctot, M., et al. Mastering the game of go with deep neural networks and tree search. nature, 529(7587)4444-48, 2016.
Niu, S., Wu, J., Zhang, Y., Wen, 7.,, Chen, Y., Zhao, P., and Tan, M. Towards stable test-time adaptation in dynamic wild world. In Internetional Conference on Learning Representations, 20223.
Sinha, S., Gehler, P., Locatello, F., and Schiele, Test: Test- time self-training under distribution shift. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 2759-27666 January 2023.
Niu, S., Miao. C., Chen, G., Wu, P., and Zhao, P. Test- time model adaptation with only forward passes. arXiv preprint ar Xiv:2404.016500, 2024.
Osowiechi, D., Noori, M., Hakim, G. A. V., Yazdanpanah, M., Bahri, A., Cheraghalikhani, M., Dastani, S., Beizace, F., Ayed, I. B., and Desrosiers, C. Watt: Weight average test time adaptation of clip. In Advances in Neural Infor- mation Processing Systems, volume 37, pp. 2024.
Su, Y., Xu, X., and Jia, K. Towards real-world test-time adaptation: Tri-net self-training with balanced normaliza- tion. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 15126-15135, 2024a.
Su, Y., Xu, X., Li, T., and Jia, K. Revisiting realistic test- time training: Sequential inference and adaptation by an- chored clustering regularized self-training. IEEE Trans- actions on Pattern Analysis and Machine Intelligence, 2024b.
Peng. X., Bai, Q. Xia, X., Huang. Z., Saenko, K., and Wang, B. Moment matching for multi-source domain adaptation. In 2019 IEEE/CVF International Conference Computer Vision (ICCV), pp. 1406-1415, 2019. doi: 10.1 22110//CCV.2211..00011
Su, Z., Guo, J., Yao, K., Yang, X., Wang, Q., and Huang, K. Unraveling batch normalization for realistic test-time adaptation. In Proceedings of the AAAI Conference Ar- tificial Intelligence, volume 38, pp. 15136-15144, 2024c.
Qian, Q., Qin, Y., Luo, J., Wang, Y., and Wu, Deep dis- criminative transfer learning network for cross-machine fault diagnosis. Mechanical Systems and Signal Process- ing, 186: 109884, 2023.
Sun, B. and Saenko, K. Deep coral: Correlation alignment for deep domain adaptation. In Computer Vision-ECCY 2016 Workshops: Amsterdam, The Netherlands, October 8- 10 and 15-16, 2016, Proceedings, Part III 14, pp. 443- 450. Springer, 2016.
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, I. Learning transferable visual models from natural language supervision. In Proceedings of the 38th International Conference on Ma- chine Learning, volume 139 Proceedings of Machine Learning Research, pp. 7768-8763.. PMLR, 2021.
Sun, B., Feng. J., and Saenko, K. Return of frustratingly easy domain adaptation. In Proceedings of the AAAI conference on artificial intelligence, volume 30, 2016.

---

Test-time Correlation Alignment
Sun, B., Feng, J., and Saenko, K. Correlation alignment for unsupervised domain adaptation. Domain adaptation in computer vision applications, pp. 153-171, 2017.
Yuan, L.,, Xie, B., and Li, S. Robust test-time adaptation in dynamic scenarios. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 11922-15932, 2023.
Sun, Y., Wang, X., Liu, Z., Miller, J., Efros, A., and Hardt, M. Test-time time training with self superviinn for generaliza- tion under distribution shifts. In International conference machine learning, pp. 9229-9248 PMLR, 2020.
Yuan, Y., Xu, B., Hou, L., Sun, F., Shen, H., and Cheng, X. Tea: Test-time energy adaptation. In Proceedings the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 23901 19901-23911, 2024.
Tan, Y., Chen, C., Zhuang. W., Dong, X., Lyu, l.., and Long, G. Is heterogeneity notorious? taming heterogeneity to handle test-time shift in federated learning. Advances in Neural Information Processing Systems, 36, 2024.
Zeng. L., Han, J., Du, L.,, and Ding, W. Rethinking precision of pseudo label: Test-time adaptation via complementary learning. Pattern Recognition Letters. 177:96-102, 2024.
Vapnik, V. N. overview of statistical learning theory. IEEE transactions on neural networks, 10(5):988 999, 1999.
Zhang, Y., Wang, X., Jin, K., Yuan, K., Zhang, Z., Wang. L.., Jin, R., and Tan, Adanpe: Exploring non- non-parametric classifier for test-time adaptation. In International Con- ference on Machine Learning, pp. 41647-41676. PMLR, 2023.
Venkateswara, H., Eusebio, J., Chakraborty, S., and Pan- chanathan, S. Deep hashing network for unsupervised domain adaptation. In Proceedings of the IEFE con- ference on computer vision and pattern recognition, pp. 5018-5021 5027, 2017.
Zhou, K., Liu, Z., Qiao, Y., Xiang, T., and Loy, C. C. Do- main generalization: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(4):4336 4415, 2022.
Wang, D., Shelhamer, E., Liu, S., Olshausen, B., and Darrell, T. Tent: Fully test-time adaptation by entropy minimiza- tion. arXiv preprint arXiv:2006. 10726, 2020.
Wang, M. and Deng, W. Deep visual domain adaptation: A survey. Neurocomputing, 155:135-153, 2018.
Wang, S., Zhang, D., Yan, Z., Zhang. J., and Li, R. Feature alignment and uniformity for test time adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20050-20060, 2023.
Wang, W. Ma, L., Chen, M., and Du, Q. Joint correla- tion alignment-basee based graph neural network for domain adaptation of multitemporal hyperspectral remote sensing images. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 1111773-184, 2021.
Wu, Y., Chi, Z., Wang. Y., Plataniotis, K. N., and Feng, S. Test-time domain adaptation by learning domain aware batch normalization. In Proceedings of the AAAI Con- ference on Artificial Intelligence, volume 38, pp. 15961- 15969, 2024.
You, K., Long, M., Cao, Z.,, Wang. J., and Jordan, M. I. Universal domain adaptation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 2720-2729, 2019.
You, L., Lu, J., and Huang, X. Probabilistic contrastive test-time adaptation. In ICASSP 2025 2025 IEEE In- ternational Conference on Acoustics, Speech and Sig- nal Processing (ICASSP). pp. 1-5, 2025. doi: 10.1 10.1 ICASSP49660.20D 2025. 10890260.
12

---

Test-time Correlation Alignment
Test-time Correlation Alignment
Appendix
The structure of Appendix is as follows:
Appendix contains the extended related work.
Appendix B contains all missing proofs in the main manuscript.
Appendix C details the proposed methods LinearTCA and LinearTCA*.
Appendix details the dataset and implementation.
Appendix contains additional experimental results.
A. Extended Related Work
A.1. Correlation Alignment
Correlation alignment is crucial technique in unsupervised domain adaptation (UDA) designed address domain shift problems. In real-world scenarios, significant domain shifts often occur between training and test data, which can severely degrade the performance of conventional machine learning methods. To tackle this challenge, CORrelation ALignment (CORAL) (Cheng et al., 2021a) is introduced to align the feature-wise statistics of the source and target distributions through a linear transformation. Similar to CORAL, Maximum Mean Discrepancy (MMD) (Gretton et al., 2006) is another technique for mitigating domain gap by minimizing the mean discrepancy between different domains. Unlike CORAL, which focuses on feature- wise correlations, MMD match the instance-wis wise statisties of the domain distribution.
Correlation Alignment has been extended and applied in several innovative ways. DeepCORAL (Sun & Saenko, 2016) extends CORAL to deep neural networks by employing differentiable Correlation Alignment loss function. This enables end-to-end domain adaptation and facilitates more effective nonlinear transformations, thereby enhancing generalization performance on unsupervised target domains. DeerCORAL (Das et al., 2021) leverages CORAL loss in combination with synthetic data to address long-tailed distributions real-world scenarios. High-order CORAL (Cheng et al., 2021b), which inspired by MMD and CORAL, utilizes third-order correlation to capture more detailed statistical information and effectively characterize complex, non-Gaussian distributions. IJDA (Qian al., 2023) introduces a novel metric that combines MMD and CORAL to improve distribution alignment and enhance domain confusion.
In addition to these advancements, recent studies have explored the integration of CORAL into more complex models and settings. For example, CAADG (Rahman et al., 2020a) presents a domain generalization framework that combines CORAL with adversarial learning to jointly adapt features and minimize the domain disparity. Moreover, JCGNN (Wang et al., 2021) integrates CORAL into Graph Neural Network (GNN) to generate the domain- domain-invariant features.
Although CORAL has achieved significant success in domain adaptation (DA), its application in test-time adaptation (TTA) constrained by privacy and resource limitations, which make infeasible compute the source correlation. This limitation significantly hampers the practicality of CORAL in more real-world scenarios, such test-time correlation alignment (TCA).
A.2. Test-Time Adaptation
In real-world scenarios, test data often undergoes natural variations corruptions, leading to distribution shifts between the training and testing domains. Recently, various Test-Time Adaptation (TTA) approaches have been proposed to adapt pre-trained models during testing. These methods can be broadly categorized into batch normalization calibration methods, pseudo-labeling methods, consistency training methods, and clustering blustering-based training methods (Liang et al., 2024). For further discussion, we classify them into two groups based on their dependence on backpropagation, as outlined in (Niu al., 2024).
13

---

Test-time Correlation Alignment
Backpropagation (BP)-Free TTA: This group includes batch normalization (BN) calibration methods (Wu et al., 2024; Schneider al., 2020) and certain pseudo-labeling methods (Zhang et al., 2023) that do not update model parameters. BN-based methods posit that the statistics in BN layers capture domain-specific knowledge. To mitigate the domain gap, these methods replace training BN statistics with updated statistics computed from the target domain. Some pseudo-labeling methods such as T3A (Iwasawa & Matsuo, 2021) utilize prototype similarity and AdaNPC (Zhang et al., 2023) utilize k-nearest neighbor (kNN) to refine predictions. Although BP-Free TTA methods are computationally efficient, their image corruption adaptation capabilities are often limited.
Backpropagation (BP)-Based TTA: This group encompasses certain pseudo-labeling methods (Zeng et al., 2024), consistency training methods (Sinha al., 2023), and clustering -aaed training methods (Lee et al., 2024). Some pseudo-labeling methods use filtering strategies, such as thresholding entropy-based approaches, to generate reliable pseudo-labels, thereby reducing the discrepancy between predicted and pseudo-labels. For instance. PLC (Lee, 2013) updates classifier layer parameters with certain pseudo-labels during adaptation. TSD (Wang et al., 2023) filters unreliable features or predictions with high entropy, as lower entropy correlates with higher accuracy, and applies a consistency filter to refine instances further. Consistency training methods aim to enhance the stability of network predictions or features by addressing variations in input data, such as noise perturbations, and changes model parameters. TIPI (Nguyen al., 2023), for example, simulates domain shifts via input transformations and employs regularizers to maintain model invariance. Clustering bustering-based training methods leverage clustering techniques to group target features, and reduce uncertainty in predictions and improving model robustness. TENT (Wang et al., 2020) minimizes prediction entropy on target data, while EATA (Niu et al., 2022) selects reliable instances to minimize entropy loss and applies Fisher regularizer. SAR (Niu et al., 2023) removes noisy instances with large gradients and encourages model weights to converge toward a flat minimum, enhancing robustness against residual noise. Generally, BP-Based TTA methods demonstrate superior domain adaptation capabílities compared to BP-Free methods, but they typically require multiple backward propagations for each test instance, leading computational inefficiencies.
Despite their strengths, both BP-Free and BP-Based TTA methods perform instance-wis alignment without considering feature correlation alignment. Our proposed method, TCA, is orthogonal to most existing TTA methods. It achieves both instance-wise and correlation alignment without backpropagation. TCA is a theoretically supported TTA paradigm that effectively addresses the challenges of efficiency and domain forgetting. By applying a simple linear transformation, TCA performs both instance and correlation alignment without requiring additional model updates. Moreover, can function as a hlu--nd--llas module to enhance the performance of existing TTA methods.
B. Proof of Theoretical Statement
B.1. Proof of Theorem 3.5
Here, we present Theorem 3.5 again for convenience.
Theorem 3.5 Let h+(-) s(f())) be an L-Lipschitz continuous hypothesis on H. Ω := UngD, B(I,r") is the set of balls near the test data. We sample i source instances from D, n S and l test instances from ID, to obtain [X,, Zg, P.] and [X,, Z,, Po] by h.) respectively. Per Assumption 3.2, Assumption 3.3 and Assumption 3.4, with probability of
at leastl exp (- (cr 5a, n2,-11 -1) + log kb), we have
2011 nd, r
Pr - P.ll + [|o(kr")]
1 7. <
(22)
[
where ,d, = XB((O, 1)) is the volume of the d, dimension unit ball and d, is the dimension of input Furthermore, considering the true source correlation Σ, = E|2," T' Z.] and the pseudo-source correlation Σ, Ž, 2, are centered. With a probability of at least min(1 exp
T Zg where Ż, and
(cop na, 1,, 20,4 Ray Inn
log k 8), the correlation
14

---

Test-time Correlation Alignment
distance |2, Ez1l bounded by:
[2, EollsS S
2 211. 14, 1y,-Palle P:lle A
Prlle + A B -
(33)
[1,((2)]]rr
where Y, is the one-hot encoding of Pt, A = ||o(kr))|] + ke(ha(X.)) + ke(ho(X.)) represents the output error of the sampled instances, and B log (2/6) is the sampling error.
2
We begin by proving Equation (22). According to Assumption 3.3 and Assumption 3.4, and under the additional assumption that Ze Zz d7s, where Vz, E Zx ||dz,|| S r", the function g(-) can be expressed using a Taylor series:
P. = -(z) g( P, P.+JJ(Z.)dZ + o(d
(24)
P = (,(2))d2 + o(d2.)
(25)
dz, P. - P, old7.s)
(26)
(z)
P. P. Jg( (.))
o dZ,
P. P.
o(dZ,)
P. - P. Jg( (zx)
o(kr*)
|dZ
(27)
(Z.)
[y(
(Zs)
F
F
Next, we examine the probability of the distance between 23 and 2, satisfying ||ze|| I r' under Assumption 3.2. Following the result from (Zhang et al., 2023), for any E Xx, and r < Tt, the probability distribution of x, falling within a ball B(I,,)) r) of radius r centered at is given by:
dD, (Ix) dx, 2 pt A(B(arr) n D2) 2 csp may ra
Da( B(xrr))
(28)
J
Let 1(x, B(x,,r)) r)) be an indicator function, where [(x,, B(e,, r)) independent and identically distributed Bernoulli random variables, representing the probability D2( B(I,,r)) Let S, (2,) Em= I[x,, B(x,,r) denotes the number of source instances W ~ Binomial(n.,. Caft rdr)
E D. that fall within B(x,,r). Then, (,, follows Binomial distribution. Let ). By applying Chernoff's s inequality, we obtain the probability that the number of source data points falling within B(I,, r) is less than m:
[E[[] - -m)²
exp (-law
(_ TA, NA, dd m)²
P(Sn(z)) < m) = P(W < m) S exp
(29)
2E(W]
2004 nT, dar
Let xe denote the i-th nearest data point d, within B(xe,r). The probability that the distance between v: and v. is less than is given by:
(-(cAt (CaH dd no m)²
(([ (m) x+ll 5) (x), 2 m) 2 11 exp
(30)
20e4s Ta, rell pel
For fixed Te, it suffices to find a single nearest neighbor x, that lies within the ball B(xe, r). and thus we set m 1. By applying the union bound, the desired probability can expressed as follows:
15

---

Test-time Correlation Alignment
n P(llo - xza1 rr)
N ((Sn(Er) 2 1)
nEx,,
=1 U ((Sn(I)) 1)
zeEx,
(cap Ta, 1 nu ns - 1)2
2 1 - 1eke
201f Te, Tei ns
(cafe Ta, (a, i 1122
1 exp
exp
+ log k
201 Ted Mar
(31)
Thus, with at least the probability exp (-
(ow may n + log k:). the distance satisfies ||dr S S nt.
20n may nn
Finally, under Assumption 3.3, let r = T, then:
=
|dzz lI F S L| F Sr
(32)
Combining the above equations, with at least the probability:
(-1 201 rdino + log
(-(aw
(GH Td, Tu n. 1)²
eee
we have:
P. Ps
o(kr'))
|Z. <
+
(33)
[Z,)
F
This completes the proof of Equation (22).
Next, we prove Equation (23). Let Σ denote the correlation matrix computed from & sampled source instances Z., and let Σ, denote the pseudo-source correlation matrix computed from is sampled test instances Z,. These matrices are computed as follows:
Σ 7.7 7,
(34)
ビ = (Z.
(2. dZ.) (2. + dZ.) Z Z, + dZ,
dZ, + (dZ.) (dZ.)" " dz,
(35)
The change the correlatione matrix is:
Σ. dz. dl, (dZ.) (dZ.)² dz.
(36)
Using the Frobenius norm, we obtain:
3Σ.- E < dZ
(dZ.) S 21 1l + |dZ., 11:
(37)
Additionally, since Σi is obtained from & source domain instances and contains statistical error relative to the true covariance matrix Σ, E[ By Hoeffding's inequality, we have:
(-
2k
POJE - EIEII 2 2exp
(38)
d²
16

---

Test-time Correlation Alignment
Here, d denotes the range of Eng, which is set to Let 2 exp
o, then:
log( /
(39)
2k
With a probability of least σ, we have:
log
- Eglls E < Vt
(40)
2k
By combining Equations (37) and (40), obtain:
log(2/8)
E - Ealle S [E E'
F +
||' E
+
+ ||d
(41)
2k:
We can further expand Equation (41) by applying Equation (33). However, since we cannot determine the true P, in Equation (33), we scale P: Ps|| r as follows:
P Polle = || - y, y, -
[||= + +1 1 PallF = P: - Y/llr + <(((x,)) +c(h(x.))
< I Yrll= 11', 1|,--11==
(42)
where is the true labels.
Finally, combining Equations (33), (41) and (42), we derive the following proposition: with least min(1
(eau n,, + log kb ,1 1 o):
exp 2014 may Mnn
[| P.ll (1Vi-p)l+ + [ P: (1i A) + B
[E, - Eelle s 21112.11r
(43)
(11g(2,)11
11
where Yú is the one-hot encoding of Pt, = = [|o(kr")]le c(h(Xx,)) c(h(X,)) represents the output generalization error, is the sampling error.
and 1og(2/8)
2k
B.2. Proof of Theorem 3.6
Here, we present Theorem 3.6 again for convenience.
Theorem 3.6 Let H. be a hypothesis class of VC-dimension d,, If H minimizes the empirical error é, (h) on D., and h; arg mingeR (h) is the optimal hypothesis TD,, with the assumption that all hypotheses are L-L-ipchitz continuous, then vó E (0, 1), with probability with at least 1 - å the following inequality holds:
(x())) S ce(ht) + 0
Voooollll ll. mell + ||2> )x11 +
2. + 27 and = minke (e + (h(()))) Mas 14h Σ, and E, denote the means 2n, and correlations of the source and test embeddings, respectively. We O(-) hide the constant dependence.
where C 2
To complete the proof, we begin by introducing some necessary definitions and assumptions.
17

---

Test-time Correlation Alignment
Definition B.1. (Wasserstein Distance (Arjovsky et al., 2017)). The p-th order Wasserstein distance between two distributions D, and D, is defined as:
W (D. D2)= ()))) inf
/f (
1/p
(44)
where [[,,, D:] is the set of all joint distributions on X, x Ag with marginal distributions D. and DD, and d(x,, Fu)) is the distance function between two instances Vg and Tu-
The Wasserstein distance can be intuitively understood in terms of the optimal transport problem, where d(z,, ) represents the unit cost of transporting mass from z, E D. to 2₁ E Dr and (x.,, 24) is the transport plan that satisfies the marginal constraints. According the Settttooovic--Rubiseeee theorem, the dual representation of the second-order Wasserstein distance can be written as:
W2([D,, D()
1/2
e/ 21) (
inf
(11ssMell mell2
sup
1fll.S1
tr((,, + - ((E//",,!!//)113)
(45)
where Ms and Me the means of (Ix) and f(z)), respectively, and |l sup 1/(x))_((x.) is the Lipschitz semi-norm, d(x,.0)) which measures the rate of change of the function f relative to the distance between 2, and 2t. In this paper, we use W2 as the default and omit the subscript For completeness, we present Theorem from (Shen et al., 2018) as follows:
Lemma B.2. (Theorem I in (Shen et al., 2018)) Let H be an L-Lipschitz continuous hypothesis class with VC-dimension du- Given two domain distributions, TD and D, let 7 minnente. + ex(h(())). The risk of hypothesis h on the test domain is then bounded by:
(h) +
2LW
(46)
From Definition B. and Lemma B.2, the difference between the true error on the training domain e, (h((t)) and the true the test domain (H(()) be obtained:
DD)sDD)) V llue mell3 tr(E, E, / & / Ilue 100--M11:: 1el13 +1E Eel1
(47)
(h) c.(fi)l s 7 + 2 Il4 - mell3 + |EE, E.11
(48)
use O to hide the constant dependence. Thus, have:
Ic, (c) c, (f)) S O 1el13 ||=, Σg 1
(49)
Then, provide upper bound the difference between the true error c. (h())) and the empirical emror (,(())) the source domain. We apply Lemma 7 of (Gui et al., 2024):
(()e(()) e, (f)) 2 e] & (2n.) exp(-
(50)
For any 8 (0, 1), set (2n,)a exp(-2n,ef we have:
d, log(2n.) - log
(51)
2n.
18

---

Test-time Correlation Alignment
Therefore, with probability least 6, have:
dy (og(2n.) log
(ca(h) - cc(h)l
(52)
n4
Combining Equations (49) and (52), let (t) arg minse (h)),, we obtain:
<(((tt)
<<a(h(() + OV II+- 1el13 + 150 - Ex11?
dh looloo((2)) (21))= - logo
<c((((()) +1 V
+ OV ||4s Mell || E11
2nx
dy (20g) log 5
<b((((0) +
+ + 0V /1p. - mc113 + +|>ا2 2,11
2n,
de log (2n,)
log 0
(c(()(())++ +
22
+ OV /111 Me 112 || Eell
2ng
dh (2gg)n))= - logg 8
(h(((()) + +//
27 + 20 1 /1lus 1e |E Ee11
2n,
(h)((t)) 0 ||4- Volm,. - Mel13 + | Ex1 +
(53)
which completes the proof.
C. Method Details
In this section, we describe the steps involved in the TCA algorithms used for test-time adaptation. The algorithm aligns feature correlations between the test and pseudo-source domains, without requiring access to the source domain data. The steps of the algorithm are outlined in Algorithm I.
Algorithm LinearTCA Algorithm
1: Input: Test instances X,, source model hg.
2: Output: Final predictions Py.
3: If use LinearTCA*: Update 9 by Equation (1)
4: Obtain embeddings and predictions:
Pr, 7, = (X((X))
5: Select high-certainty embeddings:
2. [Z((0] wt < w
6: Compute linear transformation matrix W:
| $i.
argminw
7: Apply transformation to embeddings:
2, = (Z, - H4) + fs
8: Generate final predictions:
P = g(2,)
19

---

Test-time Correlation Alignment
D. Experimental Details
D.I. Datasets
The datasets used in this work consist of a variety of domain -hiff challenges, enabling a comprehensive evaluation of test-time adaptation methods. The primary datasets employed include:
PACS: The PACS dataset comprises 9,991 images across 7 distinct classes: {dog, elephant, giraffe, guitar, horse, house, person). These images are drawn from four domains: {art, cartoons, photos, sketches ).
OfficeHome: This dataset contains images from different domains: fart, clipart, product, real-world), with a total of 15,500 images. It includes 65 object categories, and the challenge lies in the significant domain shifts between the different visual styles. OfficeHome is widely used for evaluating domain generalization and adaptation methods due to its large number of categories and diverse image sources.
The DomainNet dataset is sarge-scale dataset used in transfer learning, consisting 6 domains: fclipart, infograph, painting, quickdraw, real, and sketch). It consists of total of 586,575 images, with each domain containing 345 classes.
DemainNet:
CIFAR-10/00CC: CIFAR-10 and CIFAR- 100 are both foundational datasets in computer vision, containing 60,000 32x32 color images across 10 and 100 classes, respectively. The CIFAR- 1PFARR10/100 corruptions (e.g.. noise, blur, weather conditions) simulate real-world distribution shifts, making them highly relevant for evaluating robustness under adversarial conditions.
00C variants introduce additional
ImageNet-C: C: ImageNet-C is significantly larger compared to CIFARIO-C and CIFARIOO-C. This dataset contains 1,281,167 training images and 50,000 test images, categorized into 1,000 classes. Like CIFAR1O-C and CIFAR100-C, ImageNet-C also includes 15 types of corruptions.
D.2. Backbones
The choice of backbone models is critical for the performance of domain adaptation algorithms, they must efficiently extract features from images across various domains. For this work, we select the following backbone architectures:
ResNet-18/50. R8sNet-18 and ResNet-50 are used as backbone models in this study, where ResNet-18 offers a relatively lightweight model with fewer parameters, suitable for faster training and inference, while ResNet-50, with its deeper architecture, provides more expressive feature representation that may improve performance on complex datasets.
ViT-B/16:
The Vision Transformer (ViT) a more recent architecture that has demonstrated state-of-the-art perfor- mance in various vision tasks by treating images as sequences of patches. ViT-B/16 refers to a ViT model with base configuration and patch size of 16 pixels. VIT models are especially useful in scenarios where large-scale data and diverse domains are involved.
Contrastive Language- Image Pre-Training (CLIP), developed by OpenAl, is a cutting-edge multimodal model that bridges visual and textual domains through contrastive learning. CLIP employs dual encoders (ResNet/ViT for images and Transformer for text) project both modalities into shared semantic space. enabling zero-shot classification by matching image features with natural language prompts.
CLIP:
Both ResNet and ViT backbones are well-established in the literature and serve as strong candidates for evaluating domain adaptation techniques, with 18850 being more computationally efficient and ViT-B/16 being particularly effective in capturing complex relationships across domains. In this work, the zero-shot classification model CLIP is also included as a backbone to validate the effectiveness of our proposed methods on closed-source foundation models.
D.3. Implementation Details
Consistent with prior work (Wang et al., 2020; Niu et al., 2022; 2023; Nguyen et al., 2023; Yuan et al., 2024; Iwasawa Matsuo, 2021; Wang al., 2023; Zhang al., 2023), hyperparameter tuning in our experimental setup conducted
20

---

Test-time Correlation Alignment
across datasets. Specifically, in the Domain Generalization task, we first identify the optimal parameter set based on the highest aceuracy achieved the default domain (art paintings in PACS, art in OfficeHome and clipart in Domain These parameters are then applied to other domains to assess their performance. Specifically, we conduct search for the learning rate within the range {1e-7, 5e-7, le-6, 5e-6, le-5, 5e-5, le-4, 5e-4, le-3, 5e-3, le-2, 5e-2, le-1}. J. For methods that include an entropy filter component (e.g.. TSD), we explore the entropy filter hyperparameter in the set (1, 5, 10, 15, 20, 50, 100, 200, 300}. For AdaNPC, we explore the hyperparameter & (the number of nearest samples used for voting) over (5, 10, 15, 20, 30, 40, 50). For the LinearTCA method, we optimized the number of pseudo-source instances within the range (5. 10, 15, 20, 25, 30, 35, 40, 45, 50, 100. 200, 3003. For most datasets and backbones, smaller l values generally yield satisfactory results. For datasets with a substantial number of images per class, it is advisable experiment with larger k values. For the LinearTCA* method, we conducted an optimization of k values on the basis of other top-performing test-time adaptation method and its parameter settings.
For the Image Corruption task, we experiment with each TTA method using learning rates from {le-7, 5e-7, 1e-6, 5e-6, 1e-5, 5e-5, 1c-4, 5e-4, 1e-3, 5e-3, 1e-2, 5e-2, le-1} and the entropy filter hyperparameter in the set {1, 5, 10, 15, 20, 50, 100, 200, 300). The parameter range for l in AdaNPC, CCnnearrrCALL.inaIICCAA remains consistent with their respective selections in Domain Generalization task. The top-performing test-time adaptation approach on the Image Corruption is selected as the base method for LinearTCA*. The best performance results obtained for each method are selected as the final experimental outcomes. For the pre-trained model on ImageNet-C dataset, we utilize the model provided by Torch Vision.
During the Test-Time Adaptation phase, both the Domain Generalization and Image Corruption tasks utilize specific batch size for different backbones. ResNet- 18 and ResNet-50 use a batch size of 128, whereas the ViT-B/16 is configured with a batch size of 64.
For the implementation of the TCA method, we first obtain the embeddings of all test data during the testing phase. Based on the inter-class proportion of the test data, we perform high-certainty filtering select instances that match this proportion to construct the pscudo-source domain. Subsequently, we use the correlation distance between the pscudo-source domain and the test domain to compute the linear transformation matrix W. Finally, we apply this linear transformation to the previously retained embeddings of the test data and make final prediction.
E. Additional Experimental Results
E.1. Comparison Results Details
Tables 9 to 17 provide the detailed results of our experimental results on Domain Generalization task, and Tables 18 to 26 offers detailed overview of the outcomes from our Image Corruption task. These results demonstrate that our TCA method consistently outperforms other state-of-the-art TTA approaches across most domians and corruption types, effectively validating the TCA's capability to robustly enhance accuracy performance during the test phase.
E.2. Analysis Details
Figures 5 and 6 illustrate the adaptation process of LinearTCA to datasets with linear and nonlinear shifts, respectively. Figures (a) to (f) depict the gradual alignment process of linear and nonlinear shifts. Notably, LinearTCA demonstrates significantly better performance in adapting to linear shifts compared to nonlinear ones, which the LinearTCA's proficiency in handling simpler, linear distribution shifts while revealing its limitations when addressing more complex, nonlinear transformations.
We also provide the code for generating source and target domain features with both linear and nonlinear distribution shifts. The features are generated using Py and serve as synthetic examples. The source domain features (X,, x(?) consist of clusters sampled from normal distributions with fixed offsets. The target domain features (X,, x(2) are scaled and shifted versions of normal distributions to simulate linear and nonlinear domain shifts. The generated features can be visualized using 2D scatter plots for better understanding of the distributional changes.
21

---

Test-time Correlation Alignment
Linear Shift Code:
Linear Shift
Source domain features
s torch. cat ((torch. randn (30, 2),
torch. (30, 2) + 15,
torch.randn (30, 2) + torch. ([O, 10])), dim=0)
# Target domain features
X torch. cat ( (torch. randn (250, 2) 2 + 7,
torch. randn (250, 2) * 2.5 torch. tensor ([0, 20)),
torch. randn (250, 2)
* 3 21), dim=0)
Nonlinear Shift Code:
Nonlinear Shift
Source domain features
X S 2 torch. atforch.rande (30, 2),
corch.rande (30, 2) + 10,
corch.rand (30, 2) torch.tensos (IO, 101),
torch. (30, 2) + torch. ([-5, -10] dim=0)
# Target domain
features
2_t = torch.cal cat
(torch. randn (250, 2) 3 5,
corch.rande (250, 2) + 10,
torch.rande (250, 2) * torch. ([0, 20)),
corch.rande (250, 2)
2.5 torch... tensor ([-9, 11) dim=0)
PACS
Backbone
Method
Avg
Hyper-parameters
A
C
P
Source (He al., 2016)
78.37
77.39
95.03
76.58
81.84
nan
BN (Schneider al., 2020)
80.91
80.80
95.09
73.81
82.65
T3A (Iwasawa Matsuo, 2021)
80.27
79.56
95.57
77.40
83.20
fk=50
AdaNPC (Zhang et al., 2023)
80.81
79.14
96.17
77.81
83.48
fk=100 k=5
TENT (Wang et al., 2020)
82.86
82.12
96.11
79.82
85.23
1r=5e-3
(Lee, 2013)
81.69
81.36
95.87
73.71
83.16
1r=1e-3
ResNet-18
EATA (Niu al., 2022)
82.71
81.36
94.79
74.34
83.30
1r=le-2
SAR (Niu et al., 2023)
83.30
82.55
95.09
80.68
85.41
1r=
TIPI (Nguyen et al., 2023)
85.50
84.90
96.05
83.13
87.39
Ir=5e-3
TEA (Yuan al., 2024)
86.47
85.79
95.69
80.81
87.19
1r=5e-3
TSD (Wang al., 2023)
86.96
86.73
96.41
81.22
87.83
1r=le-4 fk=100
LincarTCA
80.91
81.02
95.69
76.74
83.59
fKTСA=30
LincarTCA
88.38
87.12
96.59
83.00
88.77
TSD 1KTCA=25
Table 9. Accuracy comparison of different TTA methods on PACS dataset based on ResNet- 18 backbone. The best results are highlighted boldface, the second ones are underlined.
22

---

Test-time Correlation Alignment
PACS
Backbone
Method
Avg
Hyper-parameterss
C
P
Source (He al., 2016)
83.89
81.02
96.17
78.04
84.78
BN (Schneider al., 2020)
85.50
85.62
96.77
72.05
84.99
nan
T3A (Iwasawa Matsuo, 2021)
84.86
82.47
97.01
78.52
85.71
fk=100
AdaNPC (Zhang et al., 2023)
85.11
82.85
97.13
79,41
86.12
fk=200 k=10
TENT (Wang et al., 2020)
88.09
87.33
97.19
79.69
88.07
Ir=le-3
PLC (Lee, 2013)
86.52
84.94
97.01
77.88
86.59
Ir=le-3
ResNet-50
EATA (Niu al., 2022)
84.72
85.20
96.35
72.46
84.68
Ir=5e-5
SAR (Niu et al., 2023)
85.55
85.62
96.77
75.24
85.79
I=-e-22
TIPI (Nguyen al., 2023)
88.18
87.93
97.13
78.80
88.01
1=le-3
TEA (Yuan al., 2024)
88.67
87.80
97.54
80.99
88.75
Ir=le33
TSD (Wang al., 2023)
90.43
89.89
97.84
81.80
89.99
Ir=le-4 fk=100
LinearTCA
86.28
83.92
96.95
79.99
86.78
fkТСA=30
LincarTCs TCA
90.92
90.10
97.84
83.86
90.68
TSD 1иTСA=30
Table 10. Accuracy comparison of different TTA methods on PACS dataset based on ResNet-50 hackbone. The best results are highlighted boldface, and the second ones are underlined.
PACS
Backbone
Method
Avg
Hyper- paramettrseterrs
A
S
Source (He et al., 2016)
86.96
84.30
98.02
78.77
87.02
BN (Schneider al., 2020)
0.00
0.00
0.00
0.00
0.00
nan
T3A (Iwasawa & Matsuo, 2021)
88.23
85.96
98.86
79.18
88.06
fk=50
AdaNPC (Zhang al., 2023)
89.01
87.37
98.98
81.06
89.11
fk=200 k=10
TENT (Wang et al., 2020)
89.60
73.08
97.90
79.33
84.98
Ir=5e-3
PLC (Lee, 2013)
87.70
85.28
98.62
80.30
87.97
Ir=5c-4
VIT-B/16
EATA (Niu et al., 2022)
87.45
84.17
97.84
76.92
86.60
Ir=5e-3
SAR (Niu et al., 2023)
86.96
84.30
98.02
79.18
87.12
Ir=5e-2
TIPI (Nguyen et al., 2023)
87.99
84.17
98.20
81.55
87.98
Ir=5e-4
TEA (Yuan al., 2024)
88.77
85.41
97.96
77.35
87.37
Ir=le-3
TSD (Wang et al., 2023)
90.72
85.41
97.96
59.63
83.43
Ir=le-5 fk=20
LCA
88.57
86.52
98.26
81.09
88.61
fKТСA=15
LincarTCA
88.96
86.90
98.26
83.05
89.30
TIPI fKTСA=30
Table 11. Accuracy comparison of different TTA methods on PACS dataset based ViT-B/16 backbone. The best results are highlighted boldface, and the second ones are underlined.
OfficeHome
Backbone
Method
Avg
Hyper-parameters
C
Source (He al., 2016)
56.45
48.02
71.34
72.23
62.01
(Schneider al., 2020)
55.62
49.32
70.60
72.66
62.05
nan
T3A (Iwasawa Matsuo, 2021)
56.61
50.06
73.39
72.99
63.26
fk=20
AdaNPC (Zhang et al.. 2023)
55.95
49.42
73.10
73.05
62.88
fk=20 k=5
TENT (Wang et al., 2020)
56.94
50.65
71.86
72.92
63.09
Ir=1c-3
PLC (Lec, 2013)
55.95
49.37
70.83
72.73
62.22
Ir=5c-5
ResNet-15 18
EATA (Niu al., 2022)
56.41
49.62
71.66
72.27
62.49
1c=1
SAR (Niu et al., 2023)
57.15
50.31
70.24
72.34
62.51
Ir=5e-2
TIPI (Nguyen al., 2023)
57.03
50.61
72.07
73.28
63.25
1r=le-3
TEA (Yuan al., 2024)
58.55
50.47
71.75
72.94
63.43
Ir=5e-4
TSD (Wang al., 2023)
58.06
49.81
71.37
70.67
62.47
Ir=le4 fk=10
Linear TCA
59.46
50.40
72.02
72.78
63.66
1KTCA=10
LinearTCA
59.83
51.80
72.29
73.17
64.27
TEA 1KTСA=10
Table 12. Accuracy comparison of different TTA methods OfficeHome dataset based on ResNet- 18 backbone. The best results highlighted boldface, and the second ones are underlined.
23

---

Test-time Correlation Alignment
OfficeHome
Backbone
Method
Avg
Hyper-parameters
C
P
Source (He al., 2016)
64.85
52.26
75.04
75.88
67.01
BN (Schneider al., 2020)
63.54
52.71
73.89
75.05
66.30
nan
T3A (Iwasawa Matsuo, 2021)
65.02
53.31
76.10
76,96
67.85
fk=100
AdaNPC (Zhang et al., 2023)
63.74
52.33
75.72
76.43
67.05
fk=200 k=5
TENT (Wang et al., 2020)
64.65
54.85
75.04
76.15
67.67
Ir=5e-4
PLC (Lee, 2013)
63.82
52.83
74.09
75.03
66.44
Ir=5e-5
ResNet-50
EATA (Niu al., 2022)
63.95
53.95
74.57
75.56
67.01
Ir=
SAR (Niu et al., 2023)
64.77
55.92
75.24
75.81
67.94
I=-e-22
TIPI (Nguyen al., 2023)
64.73
56.24
75.47
77.00
68.36
1=le-3
TEA (Yuan al., 2024)
65.97
57.57
74.72
75.97
68.56
Ir=le33
TSD (Wang al., 2023)
65.51
56.54
76.17
76.31
68.63
Ir=le-4 fk=1
LinearTCA
66.50
54.39
75.76
77.07
68.43
fkTСA=5
Lincar TCA
67.16
56.22
76.86
77.05
69.32
TSD 1KTСA=10
Table 13. Accuracy comparison of different TTA methods OfficeHome dataset based on ResNet-50 backhone. The best results highlighted boldface, and the second ones are underlined.
OfficeHome
Backbone
Method
Avg
Hyper- parameeers
A
R
Source (He et al., 2016)
73.51
63.18
82.68
85.06
76.11
BN (Schneider al., 2020)
0.00
0.00
0.00
0.00
0.00
nan
T3A (Iwasawa & Matsuo, 2021)
77.79
65.57
85.92
86.18
78.87
fk=5
AdaNPC (Zhang al., 2023)
75.57
63.76
84.30
85.43
77.26
fk=200 k=5
VIT-B/16
TENT (Wang et al., 2020)
74.58
64.15
83.74
85.36
76.95
Ir=1c-3
PLC (Lee, 2013)
74.41
63.51
82.81
85.31
76.51
Ir=Ic-4
EATA (Niu et al., 2022)
74.17
64.81
83.58
85.38
76.98
Ir=1c-3
SAR (Niu et al., 2023)
74.95
63.07
83.58
85.06
76.66
Ir=le-1
TIPI (Nguyen et al., 2023)
74.50
64.47
83.92
85.49
77.09
Ir=le-3
TEA (Yuan al., 2024)
73.71
63.23
82.74
84.92
76.15
Ir=le-4
TSD (Wang et al., 2023)
75.94
55.95
84.75
85.33
75.49
Ir=le-5 fk=20
LCA
76.02
67.35
84.12
85.56
78.26
fKTСA=5
LincarTCA
77.21
68.36
84.64
85.88
79.02
TIPI fKTCA=5
Table 14, Accuracy comparison of different TTA methods OfficeHome dataset based on ViT-B/16 backbone. The best results highlighted boldface, and the second ones are underlined.
DomainNet
Backbone
Method
Avg
Hapeeeparammters
C
S
Source (He al., 2016)
57.30
16.86
45.03
12.69
56.89
46.00
39.13
BN (Schneider al.. 2020)
57.26
11.55
43.32
11.77
56.58
47.09
37.93
nan
T3A (Iwasawa & Matsuo, 2021)
58.44
18.57
46.80
14.54
57.66
47.72
40.62
fk= 100
AdaNPC (Zhang al., 2023)
57.61
15.83
44.89
18.44
59.72
46.53
40.50
fk= k=10
TENT (Wang et al., 2020)
58.41
13.09
45.17
13.02
57.89
48.94
39.42
Ir=1c-4
PLC (Lee, 2013)
57.45
12.60
44.77
10.01
55.74
47.20
37.96
Ir=Ic-5
R8sNet-18
EATA (Niu et al., 2022)
59.18
16.22
46.65
18.04
59.59
50.21
41.65
Ir=lc-3
SAR (Niu et al., 2023)
59.13
13.10
45.75
4.88
58.25
49.83
38.49
Ir=Se-3
TIPI (Nguyen et al., 2023)
58.42
11.68
42.53
5.37
50.76
47.58
36.05
Ir=Se-4
TEA (Yuan al., 2024)
58.01
12.83
45.10
14.33
57.55
48.80
39.43
Ir=Se-5
TSD (Wang et al., 2023)
57.73
44.58
12.78
55.94
48.31
38.59
Ir=le-5 fk=100
LinearTCA
58.67
18.60
46.85
14.88
57.93
47.80
40.79
1KTСA=10
LinearTCA
59.95
16.89
47.68
18.35
59.67
50.66
42.20
EATA IKTСA=10
Table 15. Accuracy comparison of different TTA methods DomainNet dataset based ResNet-18 18 backbone. The best results highlighted boldface, and the second ones are underlined.
24

---

Test-time Correlation Alignment
DomainNet
Backhone
Method
Avg
Hyper-parameters
C
P
Q
R
S
Source (He et al., 2016)
63.68
20.93
50.35
12.95
62.16
51.42
43.58
nan
BN (Schneider et al., 2020)
63.30
14.84
48.54
10.83
62.02
52.12
41.94
nan
T3A (Iwasawa & Matsuo, 2021)
63.76
21.06
49.82
18.46
64.05
52.39
44.92
fk=100
AdaNPC (Zhang al., 2023)
64.38
20.12
51.07
17.34
65.59
52.51
45.17
fk=200 k=10
TENT (Wang ct al., 2020)
64,95
17.46
51.58
11.28
64.04
54.51
43,97
Ir=lc-4
PLC (Lee, 2013)
63,56
14.89
49.09
8.83
60.83
52.61
41.63
Ir=le-5
ResNet-50
EATA (Niu ct al., 2022)
65.89
19.88
52.67
20.36
66.58
55.99
46.89
Ir=5e-4
SAR (Niu al., 2023)
65,44
14.63
50.68
3.94
63.94
54.49
42.19
Ir=Sc-3
TIPI (Nguyen et al., 2023)
64.97
17.47
51.63
11.67
64.03
54.69
44.08
Ir=le-4
TEA (Yuan et al., 2024)
64.87
16.95
51.40
11.48
61.90
54.31
43.48
Ir=lc-4
TSD (Wang et al.. 2023)
64.31
16.53
50.75
8.52
58.97
53.63
42.12
Ir-Se-5 fk=5
LinearTCA
64.58
23.79
50.06
14.10
63.60
53.21
44.89
IKTCA=
LinearTCA
66.46
21.04
51.61
20.47
66.86
56.57
47.17
EATA IXTCA=5
Table 16. Accuracy comparison of different TTA methods on DomainNet dataset based on ResNet-50 backbone. The best results are highlighted boldface, and the second underlined.
DomainNet
Backbone
Method
Avg
Hyper-parameters
R
Source (He al., 2016)
71.62
25.59
57.34
18.07
71.90
57.24
50.29
BN (Schneider et al., 2020)
0.00
0.00
0.00
0.00
0.00
0.00
nan
T3A (Iwasawa & Matsuo, 2021)
73,56
26.95
59.77
27.34
75.77
60.24
53.94
fk=100
AdaNPC (Zhang et al., 2023)
73.76
25.52
59.86
24.42
75.99
60.14
53.28
fk=200 k=5
TENT (Wang al., 2020)
72.65
18.00
35.08
4.20
74.03
35.82
39.96
Ir=le-4
PLC (Lee, 2013)
72.29
19.27
56.69
5.00
72.45
58.04
47.29
Ir=5e-5
ViT-B/16
EATA (Niu et al., 2022)
73.91
28.76
61.71
24.79
75.39
61.84
54.40
1e=1e-3
SAR (Niu et al., 2023)
73.06
17.42
40.94
11.37
73.67
40,39
42.81
Ir=5e-2
TIPI (Nguyen et al.. 2023)
72.71
17.62
33.37
4.54
73.98
35,96
39.70
1r=le44
TEA (Yuan et al., 2024)
71.96
24.17
55.31
8.83
72.20
58.00
48.41
Ir=5e-5
TSD (Wang al., 2023)
72.40
23.47
59.20
4.12
73.54
59.60
48.72
Ir=le-6 fk=50
LincarTCA
73.37
28.42
60.88
20.66
73.39
60.03
52.79
fKTСA=5
LincarTCA
75.02
30.11
63.33
25.14
76.05
63.31
55.49
EATA fKTСA=5
Table 17. Accuracy comparison of different TTA methods DomainNet dataset based ViT-B/16 backbone. The best results highlighted boldface, and the second ones are underlined.
CIFAR- 10-C
Method
Avg
Imp.
Def.
Zoo.
Sno.
Fog
Ela.
Jpc.
Source (He et al., 2016)
27.43
33.56
21.57
43.64
40.48
51.26
51.29
68.18
54.52
66.65
87.50
27.59
67.06
48.86 86
72.37
50.80
(Schneider
T3A (Iwaswwn & Matsao, 2021)
44.16
50.32
29.64
56.98
49.02
60.85
62.29
70.20
60.883
70.75
87.23
37.00 67.68
71.60
58.59
73.22
58.89
AdaNPC (hhng et al., 2023)
40.96
47.54
27.01
54.84
46.47
59.93
61.91
70.12
60.59
71.18
87.16
35.13
71.06
58.15
73.73
57.72
TENT (Wang et al., 2020)
65.09
72.78
58.93
82.78
59.02
81.01
83.92
77.82
75.83
79.34
$8.10
82.77
72.10
76.47
72.26
75.21
PLC (Lec, 2013)
66.06
68.25
$6.92
82. 66
$7.60
79.78
82.29
74.84
74.33
78.91
87.07
81.82
70.49
73.63
71.00
73.72
al., 2022)
SAR (Niu ot al., 2023)
66.28
68.23
$8.30
82.34
59.20
79.78
82.32
74.99
74.53
78.85
87.22
82.51
70.32
73.61
71.00
73.97
TIPI (Nguyen al.,
67.69
73.21
83.80
62.36
84.15
78.
76.90
88.63
82.99
72.46
73.11
76.10
TEA (Yuan et al., 2024)
70.76
72.46
61.44
83.40
60.45
81.56
84.05
77.57
76.12
81.07
87.97
82.82
72.51
76.51
74.26
76.20
TSD (Wang al., 2023)
72.33
75.73
64.84
83.24
61.45
82.49
83.92
78. 29
75.79
81.96
87.55
79.43
73.07
78.48
75.36
76.93
LinearTCA
55.61
57.08
1.
67.17
41.70
70.28
LinearTCA
73.11
75.93
65.30
83.23
63.13
82.21
83.87
78.41
76.25
82.12
87.42
79.32
73.48
78.60
75.62
77.13
Table 18. Accuracy comparisons of different TTA methods CIFAR- 10-C datasct damage level of 5, based on ResNet-18 backbone. The best results are highlighted in boldface, and the second ones are underlined.
25

---

Test-time Correlation Alignment
CIFAR-I06 10-C
Method
Avg
Gan.
Sho.
Imp.
Def.
Gla.
Mot.
700.
Sno.
Fro.
Fog
Bri.
Con.
Ela.
Pix.
Source (Ho ot al.. 2016)
30.81
37.09
24.71
38.07
41.66
51.97
51.17
68.49
60.52
66.79
86.19
28. 25
65. 15.19
38.95
71.66
50.77
BN (Schneider al. 2020)
61.98
63.05
56.25
82.58
54.49
80.11
82.61
74.16
72.36
79.28
87.04
81.06
67.
71.27
70.22
72.24
T3A (wwasaw & Matsao, 2021)
45.34
49.51
36.76
39. 19.10
46.88
56.85
53.52
65.88
57.84
68.82
84.61
33.13
68.20
46.08
70.49
54.87
AdaNPC (Zhang et 2023)
41.93
47.29
32.50
41.60
45,70
56.09
56.38
67
59.62
69.90
85,51
32.24
68.27
46.08
70.88
54.75
TENT (Wang et 2020)
72.43
71.39
70.32
72.33
PLC (Lee, 2013)
62.35
62.71
56.09
82.57
54.07
80.24
82.91
74.54
72.26
79.37
87.20
81.09
67.62
71.39
70.71
72.34
(Niu et 2022)
67.76
SAR (Niu et al., 2023)
65.12
66.49
58.49
82.58
55.65
80.12
82.61
75.10
73.60
79.63
87.04
81.56
68.49
72.63
71.47
73.37
TIPI (Nguyen et 2023)
62,02
63.61
55.37
$2.80
54.43
80.29
83.11
74.81
72.77
78.96
87.52
81.35
67.49
71.72
70.70
72.46
TEA (Yúan et al., 2024)
63.92
65.15
55.73
82.32
52,34
80.54
83
74.99
73.17
80.08
87.58
80.90
67.57
70.47
70.26
72.54
TSD (Wang et al., 2023)
64.42
65,56
56. 16
83,06
53.95
80.88
83.32
75.18
73.58
80.17
87.84
81 49
38
72.91
71.61
73.23
LincarTCA
62.03
71.53
69.92
Lincar
65.27
66.63
59.15
82.87
56.37
80.78
82
75.05
72.69
79.61
86.85
80.97
69. 10
72.74
72.05
73.53
Table 19. Accuracy comparisons of different TTA methods CIFAR- 10-C datasct damage level of 5, based on ResNet-50 backbone. The best results are highlighted boldface, and the second ones are underlined.
CIFAR-IO-C
Avg.
Gam.
Sho.
Imp.
Def.
Gla.
Mot.
Zoo.
Sno.
Fro.
Fog
Bri.
Con.
Ela.
Pix.
Jpc.
Source (He et al., 2016)
37.25
44.31
39.99 09.94
70.31
83
85.
87.15
85.06
79.19
92.75
29.73
84.73
84.58
71,48
BN (Schneider et al., 2020)
0.00
0.00
0.00
0.00
0.00
03.00
0.00
0.00
0.30
0.00
0.00
0.00
0.00
0.00
0.00
0.00
T3A (wwasawa & Matsuo,
52.78
51.52
73.06
85 05.66
84.96
79.53
36.44
84.49
84.81
74.21
AdaNPC (Zhang et al., 2023)
48.70
54.14
50.95
83.31
74. 10
83.46
85.
87.20
85.31
80.09
92.70
37.0 92.92
84.70
85.84
84.70
74.60
TENT (Wang
2020)
37.25
44.31
39.94
83.16
70.31
83.54
85.80
87.15
85.05
79.19
92.75
29.73
84.73
84.68
54.58
71.48
PLC (Lee, 2013)
37.18
44.27
39.84
83.15
85.77
87.15
85.07
92.75
29.72
84.74
84.69
84.59
71,46
EATA (Niu et al., 2022)
46.55
48.34
31.91
86.30
69.31
$4.78
86.56
88.62
87.25
80.32
93.05
45.84
84.87
86.99
84.29
73.67
SAR et 2023)
39.94
87.15
29.73
84.73
TIPI (Nguyca et 2023)
37.24
44.32
39.93
83.17
70.30
83.55
85.77
87.16
85.07
79.18
92.74
29.73
84.75
84.69
84.58
71.48
TEA (Yuam al., 2024)
37.23
44.31
39.92
83.17
70.30
83.56
85.79
87.15
85.06
79.20
92.75
29.73
84.74
84.69
84,58
71.48
TSD (Wang et al., 2023)
37.17
44.22
39.80
83.18
70.35
83.58
85.80
87.16
85.08
79.20
92.75
29.70
84.74
84.70
84.59
71.47
LinearTCA
56.10
60.11
55.13
85.21
76.10
84 94.90
87 50
87 89
87.00
82,26
92.86
45.61
85. 64
20
85.37
77.26
LinearTCA
87,24
88.35
93.09
60.32
85.72
88.16
Table 20, Accuracy comparisons of diffferent TTA methods on CIFAR- 10-C dataset at damage level of 5, based on ViT-B/16 backbone. The best results are highlighted in boldface, and the second ones are underlined.
CITAR-100.C
Method
Avg
Sho.
Imp.
Def.
Zoo.
Sno.
Fro.
Fog
Bri.
Con.
Ela.
Pix.
Jpc.
Source (He et al., 2016)
10.46
12.49
3.36
34.44
23.63
38.10
42.67
39.25
33.01
32.84
55.78
11.55
46.48
34.88
46.15
31.01
BN (Schacider et al., 2020)
39.78
39.81
29.95
56, 18
40.92
54.71
58.68
48.52
49.59
46.79
61.89
48.63
50.26
54.61
45.37
48.38
T3A (Iwaswa & Matsuo, 2021)
10.51
11.59
3.93
36.77
26. 94
40.54
45.08
39.49
34.68
35.63
56.05
13.23
47.61
40.63
45.15
32.52
AdaNPC (Zhagg al., 2023)
3.64
33.57
32.28
12.29
43.32
(Wang ot al., 2020)
43.19
44.38
31.70
58.86
43.29
56.57
61.00
$1.19
50.66
50.75
64.02
47.77
52.08
57.74
49.11
50.82
PL. (e. 2013)
39.65
39.47
30.25
56.31
40.70
54.50
58
48.56
49.37
46.73
62.04
48
50.44
54.20
45.45
48.35
EATA (Niu et al., 2022)
41.95
41,87
31.96
$7.55
42.62
55.94
59.00
49.47
50.43
48.48
62.54
49.57
$1.12
55.64
47.50
49.71
SAR (Nin et al., 2023)
44.07
45.12
33.37
$9,80
43.69
57.21
61.15
$1.70
$1.97
$1.49
63.90
50.46
$2.64
$7.97
49.52
$1.60
TIPI (Nguyen
2023)
32.86
57.89
60.08
52.16
$1.69
$7.42
TEA (Yuan et al., 2024)
43.78
43.43
32.68
5H,20
42.62
56.30
60.67
50.84
$1.32
50.16
63.87
49.95
51.78
56.60
47.83
50.67
TSD (Wang al., 2023)
41.77
42.52
32.16
57.88
41.38
56.08
59. 89
49.
50.43
49,65
62.83
43.52
50.49
55.23
47.20
49.35
LCA
13.98
16.45
5.42
38.96
29.15
42.56
46.30
42.40
39.41
39.56
56.78
15.33
49.51
42.56
47.07
35,03
LincarTCA
44.70
45.77
33,76
59.77
44.45
57.41
61.49
52.25
52.52
51,92
64.25
51,18
53,28
58.68
49.81
52.08
Table 21. Accuracy comparisons of different TTA methods on CIFAR-100-C dataset at damage level of based ResNet-18 hackbone. The best results are highlighted in boldface, and the second ones are underlined.
26

---

Test-time Correlation Alignment
CIFAR-1004
Method
Avg
Gam.
Sho.
Imp.
Def.
Gla.
Mot.
700.
Sno.
Fro.
Fog
Bri.
Con.
Fla.
Pix.
Source (Ho ot al.. 2016)
17.23
19.42
9.77
35.34
31.87
39.15
41.98
41.99
38.68
32.00
$4.56
11.18
47.57
42.51
47.00
34.02
BN (Schneider al. 2020)
42.09
42.22
31.37
56.23
42.36
54.61
57.22
48.43
49.61
45.29
60.06
45.07
50.52
55.09
45.96
48.41
T3A (wwasawn & Matsao, 2021)
18.46
20.58
10.98
37.34
34.81
40.71
44.04
40.74
39.09
33.48
53.89
10.94
47.23
45.59
46.20
34.94
AdaNPC (Zhang et 2023)
17.41
18.85
10.53
35.26
31.43
37.49
40.99
37.25
35.78
30.65
49,75
10.68
43.74
41.78
42.53
32.27
TENT (Wang et 2020)
31.76
58.87
50.64
52.37
57.94
PLC (Lee, 2013)
41.80
42.50
31.57
55.98
42.62
54.60
57.27
48.35
48.83
45.28
60.03
44.77
50.47
55.23
46.34
48.38
(Niu et 2022)
34.96
51.65
49.89
SAR (Niu et al., 2023)
44.59
44,64
34.57
58.26
43.55
56.41
58.62
50.08
50.74
47.77
61.39
46.76
51.49
56.85
48.07
50.25
TIPI (Nguyen et 2023)
46.12
46.31
34.13
57.48
43.46
55.63
58.51
51.32
52.45
48.56
61.05
40.80
51.28
57.93
49,48
50.30
TEA (Yúan et al., 2024)
44.64
45.79
34.71
57.63
43.66 66
56.11
58.37
50.18
50.21
48.86
61.11
45.59
51.21
56.46
48.61
50.21
TSD (Wang et al., 2023)
45.37
46. 18
34.51
57.85
42.44
55. 98
58.50
50.33
50.54
49.66
60.61
36.
50.92
56.05
48.19
49.60
LincarTCA
49.49
LincarTCA
47.29
48.95
36.13
00
44.46
55. 68
58.
53.31
52.11
48.68
61.78
41.87
$1.49
58.48
50.99
51.17
Table 22, Accuracy comparisons of different TTA methods on CIFAR-100-C datasct at damage level of 5, based on ResNet-50 backbone. The best results are highlighted boldface, and the second ones are underlined.
CIFAR-100-C
Avg.
Gam.
Sho.
Imp.
Def.
Gla.
Mot.
Zoo.
Sno.
Fro.
Fog
Bri.
Con.
Ela.
Pix.
Jpc.
Source (He et al., 2016)
21.71
24.74
19.53
62.41
14
61.13
67.65
66.34
67.48
54.03
77.43
60.09
60.48
56.17
51.71
BN (Schneider et al., 2020)
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
T3A (wwasawa & Matsuo,
24.32
27.25
23.6
65.04
47.66
63
69.56
67
69.16
78.16
36.55
62.36
63.61
58.53
54.24
AdaNPC (Zhong et al., 2023)
22.75
25.77
22.65
63.54
46.46
62.02
68. 99
66.78
68.55
56.45
77.60
35. 34
61.63
62.87
56.82
53.21
TENT (Wang
2020)
10.95
13.94
4,40
66.79
45.92
67.13
71 ...28
67.83
69.92
59,26
78.42
49.29
62.18
66.26
$7.29
52.72
PLC (Lee, 2013)
21.66
24.71
19.49
62.42
43.16
61.15
67.61
66.34
67.53
54.04
33.29
60. 10
60.49
56.17
51.71
EATA (Niu et al., 2022)
50.06
$2.96
44.88
20.07
54.45
69.01
70.21
66.45
70.10
62.13
78.08
60.10
62.59
66.26
$8.61
62.40
SAR et 2023)
71.48
67.99
70.19
78.17
52.90
61.29
TIPI (Nguyca et 2023)
7.95
9.85
3,77
67.08
45.89
66.96
71.98
68.01
70.63
59.47
78.24
47.70
62.37
67.37
$8.17
52.36
TEA (Yuam al., 2024)
10.99
17.39
8.09
66.54
45.55
65.24
70.78
67.06
69.09
58.30
76.44
45.15
61.60
64.82
$7.56
52.31
TSD (Wang et al., 2023)
21.53
24.49
19.03
62.61
43.25
61.34
67.72
66.34
67.67
54.15
77.46
33.36
60.10
60.73
56.26
51.74
LinearTCA
27.46
30.02
25.33
65.29
47.98
64.26 26
69.91
68.32
70.01
58.49
78.
39.42
62.74
65.09
58.82
55.42
LinearTCA
71.00
56.07
67.5
71.01
78.61
62.35
63.42
67.73
Table 23. Accuracy comparisons of different TTA methods on CIFAR- 100-C dataset at damage level of 5, based on ViT-B/16 backbone. The best results are highlighted in boldface, and the second ones are underlined.
ImageNet
Method
Avg
Sho.
Imp.
Def.
Zoo.
Sno.
Fro.
Fog
Con.
Ela.
Pix.
Jpc.
Source (He et al., 2016)
1.54
2.27
1.48
11.44
8.68
11.12
17.62
10.64
16.21
14.02
51.52
3.44
16.49
23.35
30.67
14.70
BN (Schacider et al., 2020)
13.65
14.84
14.17
11.95
13.04
23.34
33.89
29.18
25.42
40.80
58.11
12.09
38.92
44.35
37.08
27.59
T3A (Iwaswa & Matsuo, 2021)
161
2.35
1.65
10.57
8.20
10.12
17.38
11.03
16.14
15.19
49.23
3.24
18.00
23.37
30.31
14.56
AdaNPC (Zhagg al. 2023)
1.42
8.23
8.50
12.08
2.77
15.41
(Wang ot al., 2020)
23.45
25.71
24.08
18.79
20.90
33.5 54
42.85
39.64
32.95
50.36
60.13
10.68
48. 11.81
51.96
46.98
35.39
PL. (e. 2013)
13.64
14.85
14.16
11.96
1.3022
23. 36
33.91
29.18
28.43
40.78
58.12
12.08
38.91
44.35
17.08
27.59
EATA (Niu et al., 2022)
28.24
30.16
28.88
30
25. 74
36.61
43.71
41.80
36.42
50.87
59.12
31.75
49. 10
$2.33
47.82
39.19
SAR (Nin et al., 2023)
28.04
29.59
27.8
23.66
23.90
36.16
43.40
40.94
36.71
51.01
60.18
27.38
48.95
52.47
47.98
38,55
TIPI (Nguyen et 2023)
24.75
20.37
39.31
33.47
48.41
TEA (Yuan et al., 2024)
18.82
20.50
19.00
16.27
17.68
28.51
39.17
35.19
32.26
46.92
59.16
15.42
44.39
48.81
43.64
32.38
TSD (Wang al., 2023)
15.60
16.99
16.13
15.59
15.41
28.69
38.07
32.92
30.01
45,90
58.69
7.62
41.06
47.47
41.52
30.11
LCA
2.22
3,05
2.15
11.44
9.11
11.56
19.46
13.19
18.71
17.07
52.18
3.70
19.56
25.43
32.30
16.07
LincarTCA
28.25
30,20
28.60
25.34
25.74
36.50
43,73
41,82
36.52
50.91
59,14
31.79
49.17
52.37
47.88
39,21
Table 24. Accuracy comparisons of different TTA methods on ImageNet-C dataset at damage level of 5, based on ResNet-18 backbone. The best results are highlighted in boldface, and the second ones are underlined.
27

---

Test-time Correlation Alignment
ImageNet-C
Avg
Gan.
Sho.
Imp.
Def.
Gla.
Mot.
Zoo.
Sno.
Fro.
Fog
Bri.
Con.
Ela.
Pix.
Jpe.
Source et al., 2016]
17.91
BN (Schneider et al., 2020)
16.32
17.09
16.97
15.23
15.54
26.64
39. 19.38
34.46
33.45
48.43
65.67
17.08
44.62
49.49
40.47
32.06
T3A (Iwasawa & Matsuo,
3.38
2.65
17.05
9.37
22.63
16.98
22.83
25.34
57.68
5.05
18.41
20.31
32.36
18.05
AdaNPC (Zhang et al., 2023)
2.61
3,00
2,36
14.19
7.92
11.14
18.88
14.22
18.65
22.08
52.65
4.21
16.57
17.73
28.04
15.62
TENT (Wang
2020)
26.70
28.90
28.01
25.01
24.72
38.66
48
46.04
40.49
57.04
67.96
24.42
53.73
57.72
51.54
41.32
PLC (Loe, 2013)
16.32
17.09
16.96
15.24
15.54
26.64
39.39
34.44
33.45
48.44
65.66
17.07
44.61
49.50
40.47
32,06
EATA (Nio et 2022)
35.38
37.75
36.34
33.29
32.74
47.51
53 03.07
$2.53
46,30
60.54
68
43.82
58.42
61.15
55.72
48.17
2023)
TIPI (Nguycn et al., 2023)
27.96
31.36
31.37
24.74
24.39
42.68
49.93
48.01
37.40
57.94
66.51
16.66
$6.19
58.94
53.90
41.87
TEA (Yuan 2024)
22.51
23.94
22.70
20.70
36.06
44.91
39.37
56.06
67.01
22.79
$2.65
56.86
38.90
TSD (Wing al.,, 2023)
18.90
18.80
19.63
18.31
17.49
31.12
44.27
39.74
36.02
53.38
67.00
13.05
47.97
54.54
45.95
35.08
LCA
3.35
4.39
3.08
17.67
10.01
15.12
22
19.21
25.61
27.30
59.50
5.81
20.39
21:72
34.11
19.34
46.32
Table 25, Accuracy comparisons of different TTA methods on ImageNet- dataset at damage level of 5, based on ResNet-50 backbone. The best results are highlighted in boldface, and the second underlined.
ImageNet-C
Avg
Gan.
Sho.
Imp.
Def.
Gla.
Mot.
Zoo.
Sno.
Fro.
Fog
Bri.
Con.
Ela.
Pix.
Jpc.
et 2016)
34.20
BN (Schneider et al., 2020)
0.00
0.00
0.00
0.00
0.00
(1.00
0.00
0.00
0.30
0.00
0.00
0.00
0.00
0.00
0.00
0.01
T3A (wwasawa Matsuo,
25.95
AdaNPC (Zhang et 2023)
30.01
26.86
30.98
28.19
23.40
36.38
29.65
21.18
26.59
52.78
61.24
44.53
34.44
50.50
54.98
36.78
TENT (Wang 2020)
51.19
50.00
52.48
47.35
42.95
54.40
45.19
7.43
16.27
64.87
70.90
64.35
25.97
63.35
63.48
48.01
PLC (Lee, 2013)
29.10
29.61
31.58
31.19
25.10
24.58
30.18
48
34.17
52.52
55.11
38.74
EATA (Niu et al., 2022)
56.63
56.20
$7.49
56.13
$7.13
62.26
62.89
64.02
62.77
73.71
77.m7
70. 3.4
67.39
71.42
69.96
64. 16
SAR al. 2023)
17.20
TIPI (Nguyen et al., 2023)
56.76
$6.70
58.45
55.90
56. 10
61.75
19:67
1.88
4.51
64.18
75.91
69.24
6.37
70.05
69.97
48.50
TEA (Yuam al., 2024)
39.46
38.72
41,90
24.47
28.03
42.01
33.46
13.43
33.46
53.89
66.09
58
34.95
55.65
$6.41
41.37
TSD (Wang al.,, 2023)
36.78
33.68
37.61
32.28
26.36
40.93
32.65
25.20
31.49
56.17
65.65
54.27
35.
54.61
57.17
41.33
LinearTCA
30.45
30.77
32.83
33.50
27.47
42. 14
34.91
26.98
32.85
57.65
64.88
56.60
38. 18
54.08
57.22
41.37
LineurTCA
57.63
77.11
Table 26. Accuracy comparisons different TTA methods on ImageNet-C datasct at damage level of 5, based on ViT-B/16 backbone. The best results are highlighted in boldface, and the second underlined.
28

---

Test-time Correlation Alignment
Linear Shift
30
25
25
20
20
20
15
15
10
10
5
Source Donain X
Domain
Donsin X,
X,
0-
X,
Trunsfemed T
Transferma Tes X
Trunsfommed
10
20
10
20
30
10
20
30
(a)
(b)
(c)
30
30
30
25
25
25
15
15
15
10
10
Scunce Domain X,
Sourcee Domain
Sounce Domain X,
Test Domain X,
Test Domain
0-
Test Domain X,
Transfomedd Teat X
Transformal Tei X,
Tranefommed Test X,
10 (d)
20
30
10 (
20
30
(c)
Figure 5. Adaptation process of LinearTCA to datasets with linear shifts.
29

---

Test-time Correlation Alignment
Nonlinear Shift
20
20
20
Source Domain X,
Source Domain X,
Soarce Domain X,
Domain X,
Domazi X,
Domoin X,
-10
Trensfomed
-10
Transformed
-10
Transformal
-10
10
-10
10
-10
10
10
10
10-
Soutce Domain X,
Source Domain X,
Soarce Domain X,
Test Domain X, Treeformed TestX,
Test Domais X,
"Tsst Domain X,
-10
-10
Transformed Test
-10
Teansformal Text
Figure 6, Adaptation process of LinearTCA to datasets with nonlinear shifts
30


Reviews:
[{"review_id": "YOvoSxLTZX", "reviewer": "Reviewer_SZVF", "summary": "The paper introduces TCA, which addresses some limitations of existing test-time adaptation (TTA) methods, such as neglecting feature correlation alignment and relying on backpropagation. TCA aligns feature correlations between test data and pseudo-source domains without accessing the source data. Two methods are proposed: LinearTCA, which uses linear transformations for correlation alignment, and LinearTCA+, a module to extend other TTA methods. The approach is supported by theoretical analysis and evaluated on benchmarks such as OfficeHome, PACS and CIFAR-10/100-C, showing non-trivial performance gains with reduced computational overhead.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank Reviewer SZVF for the constructive and valuable comments. The concerns are addressed as follows. (**Click [https://anonymous.4open.science/r/2025ICML_TCA_Rebuttal/README.md](https://anonymous.4open.science/r/2025ICML_TCA_Rebuttal/README.md) for Rebuttal Figures and Tables.**)\n\n---\n\n## Q1: Concerns about the learning rate (lr)  \n\nWe acknowledge the reviewer’s concerns and clarify the **misunderstandings** as follows:  \n\n- **Fixed learning rate:** Our method maintains a fixed learning rate $ lr = 10^{-3} $ across all datasets and domains. The only adjustable parameter in our method is the number of pseudo-source samples $k$ (as stated in Section 5.4, line 439 of the original paper). Any observed lr variability applies only to certain baseline methods, not to ours.  \n- **Hyperparameter tuning is dataset-specific, not domain-specific:** The hyperparameter tuning in our experimental setup is performed across datasets, not across domains. Specifically, for all TTA methods, we first tune hyperparameters on the default domain (e.g., domain A in PACS) and then apply them directly to other domains (e.g., P, C, and S in PACS). The details of this setup are provided in the Appendix D.3, line 1040 of the original paper.  \n- **Consistency with prior work:** Tuning hyperparameters at the dataset level is a widely adopted experimental protocol in TTA research [1-8].  Specifically, Tent [1], EATA [2], SAR [3], TIPI [4], and TEA [5] select hyperparameters from the source domain during training for each dataset, whereas T3A [6], TSD [7], and AdaNPC [8] choose the optimal hyperparameters from the validation set of each training dataset. \n- **Fixed hyperparameter experiment:** We fully agree with the reviewer that, in real-world scenarios, prior knowledge is often unavailable. To address this concern, we conducted additional experiments using fixed hyperparameters for all methods across two datasets. As shown in **Rebuttal Table 1**, our TCA method demonstrates strong robustness and generalization capabilities, even in the absence of prior knowledge. In contrast, some methods experienced significant performance degradation when applied across different datasets.  \n\nWe will improve the clarity of Appendix D.3 in the final version to better communicate these details. \n \n---\n\n## Q2: Concerns about the number of pseudo-source samples  \n\n- **Addressing the full-rank issue:** This concern precisely aligns with why we suggest using gradient descent to solve for the transformation matrix $W $ (as stated in Section 4.2, line 327). In our experiments, we set a fixed learning rate $ lr = 10^{-3} $ and use 20 optimization steps to obtain $ W $.  \n- **Effect of pseudo-source sample quantity:** A small number of high-confidence samples are sufficient to approximate the correlation structure effectively. Increasing the number of low-confidence samples often degrades performance.  \n\n  As illustrated in **Rebuttal Figure 1**, the correlation distance in the pseudo-source domain initially decreases but then increases as more samples are included. This occurs because, as additional samples are added, their confidence levels drop, leading to a growing discrepancy between the computed pseudo-source domain and the true source domain.  \n\n  These findings align with our theoretical results (line 200 in Theorem 3.5 of the original paper), demonstrating that high-confidence samples are closer to the true source distribution.  \n\nWe appreciate these valuable insights and will incorporate additional discussions and refinements in the final version of the paper.  \n\n## Reference  \n\n[1] Tent: Fully test-time adaptation by entropy minimization. ICLR 2021  \n[2] Efficient test-time model adaptation without forgetting. ICML 2022  \n[3] Towards stable test-time adaptation in dynamic wild world. ICML 2023  \n[4] Tipi: Test time adaptation with transformation invariance. CVPR 2023  \n[5] Tea: Test-time energy adaptation. CVPR 2024  \n[6] Test-time classifier adjustment module for model-agnostic domain generalization. NeurIPS 2021  \n[7] Feature alignment and uniformity for test time adaptation. CVPR 2023  \n[8] AdaNPC: Exploring Non-Parametric Classifier for Test-Time Adaptation. ICML 2023", "claims_and_evidence": "The below claims have been supported by the experiment results in the manuscript.\n1. TCA improves TTA by aligning feature correlations rather than just instance-wise features.\n2. High-certainty test instances can approximate the source domain's feature correlation.\n3. LinearTCA achieves competitive or superior performance compared to SOTA TTA methods.\n4. LinearTCA significantly reduces memory and computational overhead.", "methods_and_evaluation_criteria": "The paper evaluates TCA on:\n1. Datasets: PACS, OfficeHome, CIFAR-10C, CIFAR-100C.\n2. Backbones: ResNet-18, ResNet-50, ViT-B/16.\n3. Metrics: Classification accuracy, correlation distance, GPU memory consumption, computation time, and resistance to forgetting.", "theoretical_claims": "1. The key theoretical contributions establish that aligning pseudo-source correlation with test correlation reduces classification error bounds (Theorem 3.6, Corollary 3.7).\n2. The proofs appear rigorous, with clear assumptions (Lipschitz continuity, strong density conditions) and mathematical derivations supporting TCA's effectiveness.", "experimental_designs_or_analyses": "The experimental designs in this manuscript are enough and resonable.", "relation_to_broader_scientific_literature": "This work builds on and extends prior research in Test-Time Adaptation, Memory-Efficient Test-Time Adaptation.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "**Weaknesses:**\n\n1. Although the proposed method sounds reasonable, the learning rate used in the proposed method is variable across different domains, which is sensitive for a test-time methodology since no prior knowledge could be known in advance.\n\n2.  The authors mentioned that they used a small value to obtain the pseudo-source correlation, such as 10. However, to obtain a full-rank or more stable covariance matrix, a larger number of samples relative to the feature dimension is usually required. By using 10 samples, I can imagine that only 10 eigenvalues will survive the decomposition. This worries me as the fit could be biased towards a few confident samples. Is this because only a few samples are enough to approximate the correlation? or does using more samples generally degrade performance due to their corrupted features?", "other_comments_or_suggestions": "It's better for authors to address the issues raised in the weaknesses to further improve the quality of the manuscript.", "questions_for_authors": "See the weaknesses above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZMWTJBHXcL", "reviewer": "Reviewer_cY7r", "summary": "The paper investigates the feasibility of performing correlation alignment during test time. Specifically, the authors: \n1. Identifies the limitation of current TTA methods, and proposes Correlation Alignment (CORAL) as a potential soluton.\n2. Introduces a strategy to construct a pseudo-source domain by sampling high-confidence target data to address the challenge of absent source domain during testing. \n3. Provides theoretical justification of this pseudo-source approach under Lipschitz assumption. \n4. Conducts experiments to: (1) validate the theoretical findings, and (2) demonstrate superior performance compared to existing TTA methods, yielding promising results.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate Reviewer cY7r’s insightful comments and constructive feedback. Below, we address each concern in detail. (**Click [https://anonymous.4open.science/r/2025ICML_TCA_Rebuttal/README.md](https://anonymous.4open.science/r/2025ICML_TCA_Rebuttal/README.md) for Rebuttal Figures and Tables.**)\n\n---\n\n## Q1. Evaluation on closed-source foundation models (e.g., CLIP)\n\nAs suggested, we conduct additional experiments incorporating CLIP on PACS, OfficeHome, and VLCS. The experimental setup and implementation details follow [1]. As shown in **Rebuttal Table 2**, our method consistently outperforms all baselines. Specifically:\n\n- Compared to other methods, both LinearTCA+ and LinearTCA achieve superior performance, with improvements of 1.28%, 2.08%, and 2.85% on the three datasets, respectively. This is because CLIP computes image-text similarity directly, whereas our TCA methods explicitly align the embedding distributions with the source domain, making them particularly effective in multi-modal models like CLIP.\n- The performance of LinearTCA+ and LinearTCA is similar, though LinearTCA+ achieves slightly better results. This suggests that even a simple correlation alignment method can significantly improve performance on widely used models like CLIP. Moreover, it demonstrates that LinearTCA+ functions as a plug-and-play module that enhances various adaptation methods when used as a complement.\n\nWe appreciate this valuable suggestion and will incorporate these results and discussions in the final version for clarity.\n\n---\n\n## Q2. Upper performance bound for TCA\n\nTo evaluate the upper bound of our method, we conducted two additional experiments (see **Rebuttal Table 3**):\n\n(a) Fine-tuning directly on the target distribution.  \n(b) Applying LinearTCA and LinearTCA+ with real source distributions.\n\n- Compared to the original LinearTCA+, approach (b) further enhances performance, achieving improvements of 0.38% on PACS and 1.03% on OfficeHome.  \n- Similarly, both approaches (a) and (b) outperform the original LinearTCA across multiple domains. Notably, on the OfficeHome dataset, even the relatively simple LinearTCA (b) surpasses TCA(a), underscoring the crucial role of source domain information in domain adaptation. This further reinforces the necessity of approximating the source distribution in TCA.\n\n---\n\n## Q3. Why integrating CORAL into TTA\n\nWe clarify that incorporating CORAL into test-time adaptation (TTA) is supported by **prior research, empirical observations, theoretical analysis, and experimental results**. Specifically:\n\n- **Extensive studies** (see Appendix A.1. of the original paper) have demonstrated that correlation alignment (CORAL) is both effective and efficient for domain adaptation (DA). Since TTA is a subfield of DA, it is a natural extension to apply CORAL within TTA frameworks.\n- **Empirical observations** (see Section 1, line 066, and Figure 1 of the original paper) indicate that correlation distance increases during domain shifts, further motivating the need for correlation alignment in TTA.\n- **Our theoretical analysis** (see Theorem 3.6 of the original paper) shows that the generalization error in TTA includes second-order terms, and CORAL can directly mitigates them by aligning second-order statistics.\n- **Experimental results** (see Section 5.2 of the original paper) validate our theoretical findings—pseudo-source domains constructed from high-confidence samples closely approximate the true source distribution, and applying CORAL in this context leads to both effective and efficient TTA.\n- **Furthermore**, since source domain data is unavailable in TTA scenarios, an important research problem is exploring how to effectively apply CORAL in this setting.\n### Summary:\nOur motivation stems from **prior research in domain adaptation (DA)** and **our own empirical observations**, and it is rigorously validated through both **theoretical analysis** and **experimental results**. These discussions will be included in Section 2.3 of the final version.\n\n---\n\n## Q4: Typographical Error\n\nWe apologize for any confusion in our notation. To clarify, $\\hat{\\Sigma}_s$ represents the estimated pseudo-source domain covariance, while $\\Sigma_s$ denotes the true source domain covariance. We will carefully review the entire manuscript to ensure clarity and consistency in notation.\n\n---\nWe appreciate these valuable insights and will incorporate additional discussions and refinements in the final version of the paper.\n## Reference\n[1] WATT: Weight Average Test-Time Adaptation of CLIP. NeurIPS 2024", "claims_and_evidence": "While the throretical and emperical claims in the paper appear sound, I have concerns regarding the introduction of CORAL in the context of TTA. Specifically, Section 2.3 seems to lack detailed discussion and verification. I'd like to hear more evidence demonstrating the necessity of incorporating CORAL into TTA tasks.", "methods_and_evaluation_criteria": "The evaluation dataset and benchmark for domain adaptation appears robust.", "theoretical_claims": "I checked the derivation of Theorem 3.5 and Theorem 3.6, their derivation to Corollary 3.7, and the consistency with experiments in section 5.2. Appears adequate.", "experimental_designs_or_analyses": "The experiment design appears adequate. However, the models and benchmark considered, such as ResNet18 and ViT-B/16,might be somewhat outdated, which limits its applicability in real scenario. Given that the scenario of \"lack of access to source domain data\" aligns closely with the constraints of closed source foundation models that people often use today (e.g. CLIP), would the authors be interested in evaluate the proposed method on such a benchmark?", "relation_to_broader_scientific_literature": "The paper is well-presented with a clear narrative. Its approach to test-time distribution alignment could offer methodologies to complement existing test-time adaptation toolkits.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "N/A", "other_comments_or_suggestions": "1. Typo, the 7th line of section 2.2: n_s should be n_t. \n2. The authors should standardize the usage of $\\hat{\\Sigma_s}$ and $\\Sigma_s$. Sometimes they represents pseudo-source interchangeably, which is confusing.", "questions_for_authors": "1. For experiment part, I'm curious about the upper performance bound for TCA under two scenarios: (a) directly fine-tuning on target distribution, and (b) applying LinearTCA and LinearTCA+ sampled from real source distribution. \n2. As indicated in \"Claims And Evidence\" section, I'd like to hear discussion/evidence of the necessity of bringing CORAL.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "N/A"}}, {"review_id": "Wf9h7owqAN", "reviewer": "Reviewer_Knrt", "summary": "This paper proposes Test-time Correlation Alignment (TCA), a novel method to address the challenges of test-time adaptation (TTA) in deep neural networks. TCA aims to enhance model performance on out-of-distribution test data by aligning feature correlations between high-certainty test instances and the test domain, without requiring access to source data. The authors introduce two algorithms, LinearTCA and LinearTCA+, which achieve correlation alignment through linear transformations without model updates. LinearTCA is efficient and memory-friendly, while LinearTCA+ serves as a plug-and-play module to boost existing TTA methods. Experiments across various benchmarks and backbones demonstrate that TCA significantly improves accuracy, efficiency, and resistance to domain forgetting compared to state-of-the-art TTA methods. Theoretical analysis and ablation studies further validate the effectiveness of TCA in reducing test classification error and adapting to distribution shifts.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer Knrt for the valuable comments. (**Click [https://anonymous.4open.science/r/2025ICML_TCA_Rebuttal/README.md](https://anonymous.4open.science/r/2025ICML_TCA_Rebuttal/README.md) for Rebuttal Figures and Tables.**)\n\n---\n\n## Q1. Try to use more layers for transformation.\n\nFollowing the suggestion, we extend LinearTCA/LinearTCA+ by introducing MLP-based transformations with two (MLP2) and three (MLP3) layers. As shown in **Rebuttal Table 4**, incorporating nonlinear activations in deeper architectures leads to further performance improvements. As noted in the Conclusion of the original paper, future work may explore nonlinear transformations to enhance correlation alignment further. We welcome any suggestions from the reviewers regarding additional methods for TCA.\n\n---\n\n## Q2 & Q4: Comparisons with More Methods and Large-Scale Datasets\n\nAs suggested, we add comparisons with PLC [1] and T3A [2] on DomainNet [4] and ImageNet-C [5]. The results in **Rebuttal Tables 5-8** demonstrate that:\n\n- LinearTCA+ consistently improves accuracy across different backbones and on both large-scale datasets.  \n- LinearTCA performs well on DomainNet, outperforming most methods. However, on ImageNet-C, while LinearTCA improves upon the source model, it lags behind other methods. This is primarily because LinearTCA employs only linear transformations, whereas ImageNet-C contains 15 different types of corruptions, making purely linear adjustments insufficient.  \n- PLC and T3A exhibit good efficiency and memory usage. LinearTCA remains the most efficient among all methods.  \n\nIt is noted that AdaNPC [3] modifies the loss function during training (using kNN loss) and uses source domain data at test time, which does not strictly follow the TTA setting. Given the difficulty of modifying the original code within a short timeframe, we plan to incorporate AdaNPC in the final version after the necessary code modifications.\n\n---\n\n## Q3: TSD [6] with ViT-B/16 on PACS Shows Large Degradation\n\nWe observed a significant accuracy drop for TSD on PACS (*S domain: 78.77 → 58.63*, see Appendix Table 7) and conducted an in-depth analysis to investigate this issue. Our findings are:\n\n- We confirmed again that the hyperparameters provided in Appendix Table 7 correctly reproduce the reported results.  \n- The reason is that **TSD is highly sensitive to hyperparameter tuning on the PACS dataset**, and our tuning strategy differs from TSD’s:  \n   (1) Our approach: Tune hyperparameters on a single domain and apply them directly to other domains.  \n   (2) TSD's original approach: Tune hyperparameters on the source domain before adaptation.  \n\n- To address the reviewer’s concern, we conducted extensive hyperparameter tuning for TSD on the S domain (see **Rebuttal Table 9**). The results indicate that **our method consistently outperforms TSD regardless of TSD’s hyperparameters** (TCA 83.05 v.s. TSD 81.73).\n\n---\n\n## Q5. Performance and Estimation Error between Pseudo-Source and Source under Different Batch Sizes\n\nAs presented in **Rebuttal Table 10**:\n\n1. Even with a batch size of 1, LinearTCA still outperforms the source model by 2.16%.  Additionally, LinearTCA+ robustly improves the baseline method TEA across different batch sizes. \n   - This is primarily because the covariance matrix of the test domain is calculated incrementally. The estimated covariance matrix progressively converges to the true covarianc of the entire domain. \n   - When the batch size is small, the main effect is on the predictions made by the initial arriving samples. As more samples are processed, the impact of the batch size diminishes.  Therefore, in practice, if the total number of samples is large enough, the effect of the batch size becomes relatively insignificant.  \n\n2. The estimation error of the pseudo-source domain is independent of the batch size.  \n   - This is because the pseudo-source domain is also calculated incrementally, and unlike the test domain, only a few dozen high-confidence samples (as shown in Figure 4 (c) of the original paper) are needed to achieve good performance.  \n\nWe appreciate this valuable suggestion and will make this point clearer in the final version.\n\n---\n\n## Q6. Minor Typo\n\nWe apologize for any confusion caused by our notation. We will carefully review the entire manuscript to ensure clarity and consistency in the notation.\n\nThank you for your attention to detail.\n\n---\n\n## Reference\n[1] Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. 2013  \n[2] Test-time classifier adjustment module for model-agnostic domain generalization. NeurIPS 2021  \n[3] AdaNPC: Exploring Non-Parametric Classifier for Test-Time Adaptation. ICML 2023  \n[4] Moment Matching for Multi-Source Domain Adaptation. ICCV 2019  \n[5] Benchmarking Neural Network Robustness to Common Corruptions and Perturbations. ICLR 2019  \n[6] Feature Alignment and Uniformity for Test Time Adaptation. CVPR 2023", "claims_and_evidence": "YES", "methods_and_evaluation_criteria": "* Do authors try to use more layers for transformation $W$, e.g. two linear layers with non-linear activation ? Does it will better performance?\n\nSee other sections for more comments.", "theoretical_claims": "Yes. I have carefully checked the correctness of proofs.", "experimental_designs_or_analyses": "* I recommend adding baselines such as PLC (using pseudo-labels to update the linear classifier) [1], T3A [2], and AdaNPC [3], which are also computation-light methods.\n\n* The result of TSD [4] with ViT-B/16 on PACS is very low, which is very different from the original paper.\n\n* There is a lack of large-scale datasets, such as DomainNet [5] and ImageNet-C [6].\n\n* Batch size has an effect inTTA . I am curious about the performance under a small batch size because small batch sizes. It is better to show performance and estimation error between pseudo-source and source under difference batch size\n\n\n\n**Reference**  \n[1]  Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. 2013  \n[2] Test-time classifier adjustment module for model-agnostic domain generalization. NeurIPS 2021  \n[3] AdaNPC: Exploring Non-Parametric Classifier for Test-Time Adaptation. ICML 2023  \n[4] Feature Alignment and Uniformity for Test Time Adaptation. CVPR 2023  \n[5] Moment Matching for Multi-Source Domain Adaptation. ICCV 2019  \n[6] Benchmarking Neural Network Robustness to Common Corruptions and Perturbations. ICLR 2019", "relation_to_broader_scientific_literature": "Previous works mainly study instance-wise methods in TTA. This paper presents another perspective: correlation alignment. It aligns the correlation of the target domain and the source domain. Due to the lack of source domain data, the paper proposes using a pseudo-source domain to estimate the correlation of the source domain.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "See other sections", "other_comments_or_suggestions": "Minor typo:\n* inconsisent expression, e.g. Eq.12 in Line 189. It should be Eq.(12) which is consistent with others.", "questions_for_authors": "See other sections", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper proposes Test-time Correlation Alignment (TCA), a theoretically grounded and efficient approach to Test-Time Adaptation (TTA) that aligns feature correlations between high-confidence test samples and the test domain without accessing source data. Reviewers found the method well-motivated and appreciated its empirical gains, low computational cost, and theoretical contributions, though initial concerns included hyperparameter sensitivity, limited evaluation on foundation models, and small pseudo-source sample size. The rebuttal convincingly addressed these points with new experiments on CLIP and large-scale datasets, detailed clarifications on experimental protocols, and ablations on sample count and batch size. All reviewers acknowledged the improvements and raised their scores to weak accept or accept. Overall, the paper is a solid contribution to TTA with promising practical value. For further improvement, the authors are encouraged to expand large-scale evaluations and clarify the robustness of TCA under varying test conditions.
Decision: Accept (poster)