Lifelong Test-Time Adaptation via Online Learning in Tracked Low-Dimensional Subspace
Authors: Dexin Duan
Rui Xu
Peilin Liu
Fei Wen
Venue: NeurIPS 2025 poster ()
Keywords: Test-time Adaptation
Continual adaptation
Low-dimensional subspace
Primary area: applications

Abstract: Test-time adaptation (TTA) aims to adapt a source model to a target domain using only test data. Existing methods predominantly rely on unsupervised entropy minimization or its variants, which suffer from degeneration, leading to trivial solutions with low-entropy but inaccurate predictions. In this work, we identify *entropy-deceptive* (ED) samples, instances where the model makes highly confident yet incorrect predictions, as the underlying cause of degeneration. Further, we reveal that the gradients of entropy minimization in TTA have an intrinsic low-dimensional structure, driven primarily by *entropy-truthful* (ET) samples whose gradients are highly correlated. In contrast, ED samples have scattered, less correlated gradients. Leveraging this observation, we show that the detrimental impact of ED samples can be suppressed by constraining model updates within the principal subspace of backward gradients. Building on this insight, we propose LCoTTA, a lifelong continual TTA method that tracks the principal subspace of gradients online and utilizes their projections onto this subspace for adaptation. Further, we provide theoretical analysis to show that the proposed subspace-based method can enhance the robustness against detrimental ED samples. Extensive experiments demonstrate that LCoTTA effectively overcomes degeneration and significantly outperforms existing methods in long-term continual adaptation scenarios. Code is available online.

Lifelong Test-Time Adaptation via Online Learning in Tracked Low-Dimensional Subspace
Dexin Duan, Rui Xu, Peilin Liu, and Fei Wen'
School of Integrated Circuits, School of Information Science and Electrical Engineering Shanghai Jiao Tong University. Shanghai, China, 200240 {jumpywizard, Rui Xu, liupeilin wenfei eedeee.eeeed........ee.mm.. cn
Abstract
Test-time adaptation (TTA) aims to adapt source model to a target domain using only test data. Existing methods predominantly rely unsupervised entropy minimization or its variants, which suffer from degeneration, leading to trivial solutions with low-entropy but inaceurate predictions. In this work, we identify entropy-deceptive (ED) samples, instances where the model makes highly confident yet incorrect predictions, the underlying cause of degeneration. Further, we reveal that the gradients of entropy minimization in TTA have an intrinsic low- dimensional structure, driven primarily by entropy-truthig gradients are highly correlated. In contrast, ED samples have scattered, less corre- lated gradients. Leveraging this observation, we show that the detrimental impact of ED samples can be suppressed by constraining model updates within the principal subspace of backward gradients. Building on this insight, we propose LCoTTA, a lifelong continual TTA method that tracks the principal subspace of gradients online and utilizes their projections onto this subspace for adaptation. Further, we provide theoretical analysis to show that the proposed subspace-based method can enhance the robustness against detrimental ED samples. Extensive experiments demonstrate that LCoTTA effectively overcomes degeneration and significantly outperforms existing methods in long-term continual adaptation scenarios. Code is available https://github.. http://ggthubcoooo///o www.e/oooemeecceee DhunderDaviidLCooTAA
(ET) samples whose
Introduction
Test-time adaptation (TTA) aims to address domain shifts by adapting the model to the target domain during inference, without requiring access to source data 2 4 6
8) This need for on-the-fly
adaptation arises broadly across other tasks under distribution shifts 9 10 II [2 [3], motivating TTA as general deployment-time solution. Unlike traditional domain adaptation methods, TTA operates in fully unsupervised manner. It leverages only the unlabeled test data received during inference, enabling the model to adaptively improve performance when faced with domain shift on-the-fly. As a fully unsupervised online learning paradigm, TTA methods predominantly rely unsupervised learning objectives, such as entropy minimization or its variants [2 However, such methods are prone to instability. For instance, entropy minimization suffers from trivial solution that assigns all predicted probability to the most probable (but may incorrect) class. Consequently, these methods susceptible to performance degeneration during continual adaptation.
To address this, various methods have been proposed. For example, 2 leverages entropy minimiza- tion at the batch level while restricting updates to only the normalization parameters. Moreover, some approaches employs teacher-student networks, where a teacher model provides stable guid- ance through moving average of parameters to prevent collapse g 6 4 Furthermore, model
*Corresponding author: Fei Wen, cmaillwenteii-sjjuudd...o
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

resetting strategies, which periodically reset partial or full model parameters to their initial states, have demonstrated effectivvness in mitigating error accumulation during adaptation g T3 [6]. Generally, these strategies have shown effective under the current mainstream evaluation protocol, which typically involves single-epoch adaptation on a test set. However, in more realistic lifelong scenarios, where the model is required to adapt continually over the long term, these methods still face challenges in maintaining consistently robust performance.
Although numerous heuristic approaches have been proposed to mitigate the degeneration, an in-depth analysis of its underlying cause remains lacking. this work, we delve into the underlying cause of the degeneration problem and provide novel insights to address it at its core. Specifically, we show that a main underlying cause of the degeneration entropy-miniminn minimization based methods the presence of cntropy-deceptive (ED) samples, where the model produces highly confident yet incorrect predictions. These samples frequently arise when facing out-of-distribution data in the TTA setting, which mislead model updates and exacerbate enror accumulation over time, especially in long-term continual adaptation scenarios.
Furthermore, we show that the backward gradients of the entropy loss in TTA exhibit an intrinsic low-dimensional structure. is primarily driven by entropy-truthful (ET) samples, whose gradients are highly correlated as they tend to share similar update directions. In contrast, gradients from ED samples are scattered and less correlated. Based on this observation, we present nontrivial finding that, the detrimental impact of ED samples can be suppressed by constraining weight updates in low-dimensional principal subspace of the gradients. Building on this insight, we propose LCoTTA. a lifelong continual TTA method that tracks the principal subspace of gradients online and utilizes their projections for adaptation. This approach can effectively suppress the impact of ED samples and ensures robust performance in challenging long-term continual adaptation scenarios. In summary, the main contributions are as follows:
An observation that the degeneration of the entropy minimization based TTA method in continual adaptation is primarily caused by ED samples, instances where the model makes highly confident but incorrect predictions.
A novel finding that, without using any supervision information, the detrimental impact of ED samples be suppressed by exploiting the intrinsic low-dimension structure of the gradients, which is primarily formed by the correlated gradients of ET samples.
A lifelong continual TTA method LCoTTA, which tracks the principal subspace of gradients online and utilizes their projections into this subspace for adaptation. LCoTTA does not suffer from degeneration and enables robust long-term continual adaptation.
A theoretical analysis that shows the proposed subspace projection based method can enhance the robustness against detrimental ED samples.
Extensive experiments demonstrate that LCOTTA maintains robust performance in long-term continual adaptation scenarios and significantly outperforms cxisting continual TTA methods.
While this work focuses on TTA, the method of exploiting ET and ED gradient structure to effectively distinguish them in an unsupervised manner holds promise for broader applications in unsupervised learning tasks. The proposed method is complementary existing approaches for robust TTA, such as reacher-student networks, model resetting, and weight regularization. Integrating it with these methods can be expected to further improve performance.
2 Related Work
Test-time Adaptation. I first proposed updating the activation statistics of BN to enhance model robustness. Further, MemBN (17] introduced memory- based BN approach, which aggregates and adaptively weights stored statistics to achieve robust TTA. Tent [2]. one of the pioneering works, updates only the affine transformation parameters of BatchNorm layers by entropy- citroop-mmimmzzation. combines entropy minimization with diversity regularization to achieve robust TTA. leverages visual prompts to efficiently adapt to target domains and bootstrap source
SHOT 3
DePT 18 representations. Furthermore, single-sample methods have been explored in [19 20 while non- parametric approaches are proposed 21 22.. Moreover, neuro-inspired method and energy-based
model have been considered in 22 and 24

---

Continual Test-time Adaptation. Continual TTA aims to adapt a model to dynamic and evolving target domains. CoTTA [8] introduces teacher-student framework with consistency loss, while
ECOTTA 6 employs meta-networks and self-distilled regularization to improve memory efficiency. leverages supervised pretraining on labeled source datasets improve initialization,
TTACOPE [25]
while BЕCoTТA [26] utilizes a mixture-of -inture-of-dommin domainn low-rank experts for domain-adaptio routing.
Moreover, there exists a number of recent works designed for robust continual TTA [27 [27 28 22 30 31
32 33
34
35
36
16 37
38 39 40). Despite the effectiveness of model-resetting, teacher-student
learning, and weight regularization approaches, they still face challenges in maintaining consistently
robust performance in long-ter term-termmm continual adaptation scenarios. More importantly, there lacks an in-depth analysis on the underlying cause of degeneration in unsupervised TTA, which is crucial for understanding and mitigating it.
lon-dimensional learning. The low-dimensional structure of neural-netwon learning has been extensively studied [41
46 47]. These studies reveal that loss landscapes of neural- networks reside within an intrinsic dimension, which enables model weights to be optimized in a low-dimensional subspace. Unlike these works focusing on the low-dimensional structure of weights sampled along the optimization trajectory, we reveal the low-dimensional structure of batch-based stochastic gradients during the adaptation process, which is driven by ET samples in TTA and can help suppress the gradients of ED samples.
42
43 44/45
3 Analysis on the Degeneration of Entropy-Minimizttio Based TTA
3.1 Prelimineries
Let f(z; 8) be model with weights 0 pre-trained on source domain D. = { b:) which follows a distribution (z). Given the source model (x; the goal of TTA to adapt to test data D, {xx))ss1 where the data distribution P; deviates from the training distribution, i.e.
(x + (x). Without access to source data and without using any supervision information on test
data, TTA methods typically utilize unsupervised losses, such as the entropy loss (2
-
ΣE (og.)<))))(()))
where p(x;)) the model predicted probability on sample F,, encourages the model to make confident predictions by minimizing the entropy of the model predictions.
Traditional TTA methods typically focus on adaptation on test data over limited number of steps, but world scenarios often involve continuously evolving domain shifts, requiring models to adapt continually over the long term. Lifelong TTA addresses this by adapting model sequentially arriving test data from evolving target domains {P. t 1, 2.2..... At each step f. the model predicts output fa;; 6,) for input I; and updates its parameters for future steps. i.e., O₂ - 9.+1. This setting is challenging models relying on unsupervised losses are prone to degradation over time.
3.2 Entropy Reliability of Test Samples
As discussed above, existing TTA methods, operating without access to source data and supervision on test data, predominantly rely on unsupervised entropy minimization or its variants [2] g 4 While simple and effective, entropy minimization as an unsupervised loss can be unreliable and mislead model updates. Specifically, entropy minimization has trivial solution that assigns all predicted probability to the most probable class. This is particularly problematic for samples with high-confidence but incorrect predictions, which frequently occur when the target domain significantly deviates from the source domain. For instance, given ground-truth label [0,0,0,0,1], a model prediction of 1........0.4.0.111 1,0.2) optimized via entropy minimization is likely converge to 10,0,1,0,0], resulting in incorrect predictions.
To address this degeneration problem, Tent 2 proposes to jointly optimize batched predictions while restricting updates to normalization parameters. However, this approach struggles to maintain satisfactory performance in continual adaptation. Although methods like model resetting and teacher- student frameworks 2 48 49] partially mitigate degeneration, as shown our experiments (Figure 4). they still suffer from performance degradation in long-term adaptation scenarios. Press
3

---

et al. [50] further show that EM first improves accuracy by embedding test data close to the class means of training data, but over many iterations, pushes test embeddings far from the training data, resulting in degraded accuracy. Here, we offer new perspective on EM by focusing on the entropy reliability of predicted test samples.
To better characterize the reliability of predictions, we introduce the Entropy Reliability Score (ERS). Let fi. P2, .............................. ................................ ...]] denote the model's predicted probability distribution over C' classes, where Σ P. = 1 and P. E [0, 1]. For a sample with ground- truth class c E {1,2..... C)2, its ERS is defined as
S(p) :- Pe max Ps. ifc
where Pe the predicted probability for the true class c, and maxiyle Pr is the highest predicted probability among all incorrect classes. The ERS measures the confidence gap between the true class and the most competitive incorrect class, providing quantitative indicator of prediction reliability. Based on ERS, we further define ET and ED samples.
Given model prediction P and with the definition of the reliability score in (2, sample is classified as entropy-truthful if S(p) > O; otherwise, it is classified as entropy-deceptive.
Definition 1:
3.3 The Degeneration of Entropy Minimization
Here we further dive into the underlying cause of the degeneration problem entropy-minimizatio based methods. We present observation that this issue is primarily driven by ED samples. From Definition 1, ET samples indicate reliable predictions where the model assigns the highest confidence to the true class. In contrast, ED samples reflect unreliable predictions, where the model assigns higher confidence to incorrect classes, potentially misleading optimization processes such as entropy- minimization. This issue is particularly pronounced for highly confident ED samples, which exhibit low entropy despite being incorrect.
To analyze the impact of dif- ferent types of samples on the adaptation process, we con-
00%
30%
25%
duct experiments under a typi- cal continual TTA scenario the CIFAR 100C and ImageNetC datasets with Gaussian corrup- tion at severity-5. The sam- ples sorted by ERS, and pre- trained Rt-2e and ResNet- 50 from [51] adapted using munoopy-iniiiization with dif- ferent subsets of these samples. Among the samples ranked by ERS, the top 58.17% on Ci- FAR100C and 11.59% % on Ima- geNetC are identified ET samples with S(p) > Figure shows the performance of continual adaptation over 50 epochs.
20%
1pp0%
7pp0%
7op70%
20%
Topt0's
11775%
7p500%
11p00%
1o00%
10%
Top00
Tap03%
10%
0%
20
Esechs
(a) CIFARI00C (ResNcXt-29)
(b) ImageNetC (ResNet-50)
Accuracy of matropy-minimization based continual TTA using different subsets of the samples. Among the sam- ples ranked by ERS under Gaussian corruption, the top 58.17% % CIFAR 100C and 11.59% on ImageNetC are identified ET samples with S(p) > 0.
Figure 1:
The results demonstrate the detrimental impact of ED samples on continual adaptation. For example. on CIFAR 100C, when adaptation is performed exclusively on ET samples (e.g., the top 40% of samples), the model does not exhibit degeneration during continual adaptation. In contrast, expanding the sample set to include ED samples (e.g., the top 75% or 80% samples) leads to significant degeneration. A similar trend is observed on ImageNetC.
4 Method
The above analysis suggests that the degeneration of entropy-minimization can be avoided by using only ET samples for adaptation. However, identifying ET samples without supervision is challenging. In this section, we reveal that the impact of ED samples can be suppressed by leveraging the low- dimensional structure of ET gradients. Then, we propose robust lifelong continual TTA method.

---

4.1 Correlated ET Gradients Forming A Low-Dimensional Structure
As discussed above, we identified ED samples as the underlying cause of model degeneration. Here, we show that gradients from ET samples are highly correlated, which form now-dimensional subspace. contrast, gradients from ED samples are scattered and less-correlated.
Figure (a) and (c) shows
the correlation between
Batch Index (Sorted by ERS)
100%
the gradients of different
batches CIFARIOM
Batch Index (Sorted by ERS)
+P
Contributioon (%6
30%
Cumulative EV (9%)
0.4
80%
and ImageNerC (Gaussian noise with severity-5) for ResNeXt-29 and ResNet- The
20
Index
Batch
0.3
20%
The first 10 principal components account for
30
with
50, respectively.
50
ERS=0
0.2
10%
samples are sorted by their ERS values from the most antropy-truthful to the most entropy-deceptive. be seen that gradients of samples are highly correlated, while gradients of ED samples are less correlated.
60
0.1
40%
0%
20 Principal Component Index (b) PCA (CIFARIOOC)
10
15
25
0.0
(a) GradSim (CIFARIOUC)
can
Batch Index (Sorted by ERS)
30%
100%
e +
+
Batch Index (Sorted by ERS)
0.30
Cumulative CV (%)
Batch Index
80%
20
ERS=0
0.25
20%
The first principal components account 90%
30
0.20
60%
This phenomenon arises as gradients of ET samples tend to share similar up-
40
0.15
10%
0.10
40%
date directions in parame- ter space, thus are corre- lated. In contrast, the gradi- ents of ED samples are less- correlated, as their update directions are more scat- tered and do not have con- sistent directions. Based on this analysis, that gradients of entropy- minimization in TTA have diw-eeiennional structure, which is primarily formed by the principal update directions shared by ET sample gradients.
70
0.05
0%
10
15
20
25
0.00
Principal Component Index
(c) GradSim (ImageNetC)
(d) PCA (ImageNetC)
Figure 2: (a) and (c): Pairwise cosine similarity of batch gradients (GradSim) of test samples sorted by ERS. ET samples exhibit high gradient similarity, and the boundary (a sharp drop) in similarity closely aligns with the batch index where ERS=0, (b) and (d): Explained variance analysis of the gradient matrix G during TTA.
expect
Let 9: (BL, 0) denote the gradient of the entropy loss L. defined in ① with respect to the model parameters 9 on a batch of samples B,, To investigate the low-dimensional structure of stochastic gradients, collect T backward gradients G 191.92.-.- , 9rl E RaxT over T sequential batches sampled during the adaptation process, where n is the number of model parameters. Principal component analysis (PCA) of G is then performed using SVD.
Figure 2 shows typical PCA results in the TTA experiments on CIFAR100C (ResNeXt-29) and ImageNetC (ResNet-50). Gradients are sampled every 50 iterations with T 50. The bar plot shows the explained variance of individual principal components, while the red curve represents the cumulative explained variance. Notably, the first few principal components dominate, with the top 10 components capturing more than 90% of the total variance. This result demonstrates that parameter gradients during TTA reside in a low-dimensional subspace, formed by ET sample gradients. In contrast, ED gradients do not have such a structure, as shown in Figure 9 in Appendix
4.2 Suppressing Entropy-Deceptive Samples via Low-Dimensional Gradient Structure
We reveal that ET gradients are predominantly concentrated within a low-dimensional principal subspace, whereas ED gradients exhibit weaker alignment with this subspace. To show this, we first construct r-dimensional principal subspace of the gradients. Specifically, given collected
5

---

gradient matrix G E RaXT, we extract a r-dimension subspace of it via the following formulation
(U,, s.t. U,U,1I
max U,ER
where U, E [Rex contains the orthonormal bases corresponding to the largest eigenvalues of GG which spans the r-dimension principal subspace of G. with r < n. Formulation (3) is standard PCA problem that can be solved by eigen- decooposionnnnionnnn of GG
Given extracted sub- space spanned by Ur. we analyze the impact of gra- dient projection into this subspace. Specifically, for a gradient It. its low- dimensional representation is obtained via projection as U, It, and then back- projected to the original pa- rameter space as
0.8
Sorted ERS
1.00
Ron R2nb=125
0.7
0.75
0.5
Sank-500
Samk-77
0.75
0.6
Extrrpy Res labaattyy Score
0.50
0.5
Salitlllll
Delatbliy
0.3
Ranke1
6.25
Rank-3
0.2
Ranked
-0.50
0.1
0.0
Pank-500
Rank-20
-1.00
-0.25
0.3
Sorted ERS
-100
No2
Most doocotet
samples
(a) CIFARI00C (ResNeXt-29)
(b) ImageNetC (ResNet-50)
9. U,U,
(4)
Figure 3: The l--norm ratio [el| for different sample types. The
We then examine the effect of this principal subspace representation on ET and ED samples. Using the same experimental setting in Figure D with the CIFARI00C and Im- ageNetC, we sort the test samples by their ERS values. With the sorted samples, we compute the f2-norm ratio | /lll from the most truthful the most deceptive samples. Figure presents the results for different subspace dimensions, where the ratio |lr.1//|9e:11 is computed on batched samples.
samples sorted by their ERS values from most entropy- (left) to most entropy-deceptive (right).
Interestingly, as shown in Figure the ratio (@ell/1l9e]]] generally decreases from the most truthful to the most deceptive samples. This implies that the principal subspace representation (4) can effectively differentiate between samples to extent. Notably, the gradients of ED samples exhibit more pronounced decay in this representation, due to the weak correlation with correlated ET gradients.
4.3
Adaptation in Tracked Low-Dimensions Subspace
The above findings offers a way to mitigate the impact of ED samples by leveraging the low- dimensional structure of ET gradients. We utilize the oiw-dimeniional representation 9i of the gradient Yr for parameter update. Furthermore, considering that the test data distribution may vary continuously in continual adaptation scenarios, we dynamically track low-dimensional principal subspace of the gradients in an online manner. As evidenced by experiments (Table in Appendix), using a fixed subspace would degrade performance under varying test data distributions.
Specifically, during adaptation, we compute the gradient of the entropy loss with respect 0 for the 1-th batch as = VoL. 0). To capture the evolving gradient structure, we maintain queue of the most recent gradients Ge = [gt 1 9t 2 ge] E R=* where n is the dimensionality of the parameter space. Subsequently, PCA (3) is applied to Ge to extract r-dimensional principal subspace represented by the projection matrix Ur E [Raxr, where r < k < n.
The projection matrix Ur. detines the subspace used for parameter update during adaptation. based on which the component of 9: residing within this subspace is obtained by projecting St into the subspace and then model parameters are updated based on g as
it into the original parameter space: 9t- Ur.U UT,9.. Then, the
0 = 0: 79.,
where 77 is the learning rate. This approach ensures that the parameter updates are informed by the salient components of ET gradients, captured by the dynamically tracked low-dimensional subspace.
However, applying this method update all model parameters memory-intensive, as it requires maintaining the gradient queue G2 of size x k and the projection matrix U,, of size X r. This

---

significantly increases memory consumption, particularly for modern neural networks with large parameter dimensionality n, which makes it impractical for some on-device adaptation applications.
To address this issue, we restrict updates to the affine parameters of the normalization layers rather than the entire model. Previous studies 2 7 have demonstrated that updating normalization layers alone is sufficient to achieve strong performance in TTA, as they play crucial role in controlling the feature distribution and are sensitive distributional shifts. Notably, the BN and LN parameters typically constitute less than 1 % of the total model parameters. Hence, tracking the subspace of them introduces minimal computational and memory overhead.
Moreover, we also employ the entropy-based sample filtering (ESF) strategy [13] 7 before subspace tracking. Note that, ESF alone cannot distinguish between ET and ED samples, as shown in Figure Appendix G As shown in Section 4.1, ET sample gradients form a more prominent low-dimensional structure, while ED sample gradients scattered and less-correlated. Thus, removing high-entropy (ambiguous) samples (Figure 7 enables the subspace to better capture reliable update directions, which makes ED gradients more orthogonal to it and easier to suppress.
5 Analysis on the Enhanced Stability of Subspace Projected TTA
We present stability analysis to show that the proposed subspace projection method enhances TTA robustness against detrimental ED samples. Note that TTA is essentially a local fine-tuning of a source pretrained model. The analysis is restricted to neighbourhood of any local equilibrium, and relies on assumptions that ET gradients concentrate in a low-dimensional principal subspace, while ED gradients have weak inter-sample correlation and approximately isotropic dispersion.
Let Og E R. be the model parameters updated by mini-batch SGD during TTA. For each step 4, the mini-batch B, = {B., B.} contains ET samples B, and ED samples B,. Denote the gradients on B,
as
9t == VoL B,) = 9. + gr.
where áz VoT,(0;; Br) and dt Vgl,(0, B,) denote the gradient components of ET and ED samples, respectively. From the results in Section 4 we make assumptions: a) Mcan and variance models of ET and ED gradients: Eø(ğ] = GET. Cov CET. Es ÿED. Covla] MED = 4 + A, where NET and EED are batch-normalized covariance of ET and ED gradients, respectively, with || < FÂn- Weak correlation between ET and ED gradients: Cov(ğ 9ET, I (ED) b) Low-rank structure of ET gradients: ΣET has effective rank < n, let Σ = UAUT
be any degeppoccooooiiiioon with U, = [uy ..... u,_] being the eigenvectors corresponding to the largest cigenvalues, and denote P, == U,U c) Small-step limit: As 1) - the discrete dynamics 19tt converge to the stochastic differential equation (SDE) (52
of SGD f.+1= = O₂
d0(t) (()dd (0))())))
(5)
where g GET + ÍED and Σ = EgT EED-
Theorem (Local stability of full-space and subspace-projected TTA). Suppose that the above assumptions hold. Let 0 be an equilibrium satisfying g(0)) 0 (i.e. with zero gradient expectation). For the full-space SGD, if, for all 0 in neighbourhood of 0*
ger (0 0*) > 1 (0 0¹)l + 2 tr(E),
(6)
then the SDE (5) is mean-square stable at 6". Further, for rank-r subspace projection based lcarning. ie.. update in the principal subspace of ET gradients via replacing ge by Prgt. the corresponding contimuous-time SDE
do(t) (ry(O)dt /2),5"/(()dWW0))
mean-square stable at 0* if
(P.ğer) " 8*) > (P. (A - 0*) + 7 (P.EP!.)
(8)
The proof is given in Appendix A Theorem provides balance-of ooieee-ffforoee- conditions for the stability of both full-space SGD and subspace-projeete SGD. Stable local convergence guaranteed when the corrective drift from ET gradients exceeds the combined ED bias and diffusion noise. Conversely,
7

---

60%
Tent
CoTTA
Rotta Ada.
SAR DEYO
PETAL
50%
BN
ETA
Ours
Accuracy (%)
40%
0%
5
10
15
20
25
30
35
40
45
50
Adaptation Cycles
(a) ResNet-50 ImageNetC.
80%
Tent
CoTTA
Rotta SAR
DEYO
Ours
ETA
60%
Accuracy
40%
20%
0%
15
20
30
35
50
Adaptation Cycles
(b) ViT-B/16 on ImageNerC
Figure 4: Accuracy continual adaptation with ResNet-50 and ViT-B/16 over 50 cycles on Ima- geNetC. Each cycle contains 15 corruptions with 50000 samples for each corruption, resulting in total of 3.75 10" samples used in the 50 cycles.
the ED contribution outweighs corrective ET drift, the dynamics may diverge or converge to an inferior minimum, which leads to model degeneration. Comparing the conditions (6) and (8), the subspace projection method relaxes the stability requirement and thus is more robust.
Specifically, when the ET gradient mean ÿET lies mostly in the subspace Ur, we have P.õEr ØEr- Mcanwhile, projection onto this subspace removes most out-of-subspace component of the ED- gradient bias such that P,GED 2 0 for & n. Then, the stability condition (8) simplifies to
GET (0 8*)
" 2
Σ
As + rOTDD
(9)
where Ex >; TT( pf with X being the i-th eigenvalue of Σ Compared with the full-space case with tr(2)) Σ X no subspace projection largely reduces the diffusion < 1. This significantly enlarges the admissible range of ED bias and noise that still satisfies the stability condition. Consequently, the subspace-projecte method is substantially more robust to detrimental ED samples.
energy factor شー A tro
Autnogo
6 Experiments
conduct experiments on ImageNetC dataset 53 which consists 15 corruptions each with 5 severity levels. We experiment with two representative model architectures: ResNet-50 with batch normalization (54) and the ViT-B/16 with layer normalization. Results on CIFAR100C and semantic
segmentation provided in Appendix K and L

---

We evaluate our method under a challenging long-ten continual TTA setting, where the model adapts continuously over 50 cycles 15 corruption types (severity=5), total of 37.5 million test samples. The model performs unsupervised continual adaptation without any external intervention from the very first beginning, such as comain-sppecife information, model resetting. or warm-up. We compare our method (r 25) with several SOTA continual TTA methods, including AdaContrast [5]. BN []. TENT [2], CoTTA [8) SAR 77]. RoTTA 44, ETA [3], PETAL [55] and DeYO [56). Notably, CoTTA, and RoTTA employ teacher-student networks, while SAR adopts model resetting strategy.
6.1 Results on HhageNet-to-lmagelle
Figure presents the results on the SongeNet-to-ImageNee task over 50 continual adaptation cycles. Clearly, our method demonstrates robust and superior long-term adaptation performance on ImageNetC. Most of the compared methods suffer significant degradation during long-term adaptation. For instance, the accuracy of Tent drops below 10% within the first 10 adaptation cycles, whilst that of CoTTA gradually declines over continual adaptation. In contrast, our method consistently achieves high performance throughout the entire adaptation process, attributed to its ability to effectively suppress the detrimental impact of ED samples. These results demonstrate the robustness and adaptability of our approach in challenging long-term adaptation scenarios.
Results under a standard TTA setting, where model adapts to one corruption at a time for a single epoch, are provided in Tables aande in Appendix C which demonstrate that our method can also achieve competitive performance in short-term adaptation scenarios.
6.2 Analysis and Ablations
Ablation of subspace projection. We evaluate the effectiveness of the subspace projection method. As shown in Table the method without using any strategy quickly collapse under entropy-basee based continual adaptation. Entropy filtering improves initial performance but still suffers from ED samples due to the nature of entropy, leading degeneration in long-term. Our method achieves consistent robustness over long-term adaptation.
Table 1: Accuracy (%) different adaptation cycles ImageNetC with ResNet-50.
Strategy
Cycle I Cycle 25
Cycle 50
Cycle 75 Cycle 100
39.13
0.70
0.69
0.70
0.70
Entropy filtering
37.70
36.09
34.79
31.56
28.68
+Sbsspace
36.71
36.45
36.87
36.34
36.68
+Entropy filtering and Subspace (Ours)
40.70
42.70
43.10
42.79
42.66
Effect of subspace dimension. Figure shows the performance of our method with different dimensions of the subspace, r E {10, 25, 50, 100). in continual adaptation on ImagenetC over 50 cycles. Using a reasonably low dimension (e.g., r = 25) can achieve satisfactory performance during long-term continual adaptation. However, as the rank increases, the restricting effect of the subspace diminishes, leading to performance degradation in later cycles. On the other hand, too low a rank would limit the adaptation performance.
Effect of hyperparameters. We conduct exper- iments the sampling interval and the length of the gradient queue k. As shown in Figure
43%
Accuracy (%)
42%
when the queue length for storing gradients exceeds a certain threshold (e.g., 25), it cap- tures sufficient gradient information to compute a stable subspace, thus maintaining stable per- formance. Similarly, the model exhibits robust long-term adaptation performance across wide range of sampling intervals. Table 5 in Ap- pendix F further shows that proposed subspace method exhibits strong robustness to a wide range of learning rate choices.
41%
40%
10
rank-50
39%
rank
25
rank=100
0
10
15
20
25
30
35
40
45
50
Adaptation Cycles
Figure 5: Ablation on subspace dimentsion on ImageNetC with ResNet-50.

---

46%
Accuracy (%)
k=25
k=75
46%
step=1
step=50
44%
k-50
k-100
Accuracy
44%
step=25
step=100
42%
42%
40%
20
Adaptation Cycles
Adaptation Cycles
Figure 6: Results on gradient queue length &s (left) and sampling interval(right).
Ablation on subspace tracking. Ablation study on the proposed subspace tracking approach is provided in Ta Appendix D which demonstrates the effectiveness and indispensability of it in achieving robust continual adaptation under continually varying data distributions.
Computational Complexity and Efficiency. Our method only introduces additional memory to store a gradient queue to compute subspace (Section 4.3), which typically less than 0.01kn x 0.5n for h = 50. Thus, it does not incur significant memory and computational costs. Memory and runtime comparison on ImageNetC with ResNet-50 is given in Table in Appendix
Conclusion
To address the critical challenge of performance degeneration in unsupervised continual TTA, this work identified ED samples underlying cause of the degeneration in entropy-minimizzaioo methods. Furthermore, we revealed that the backward gradients an intrinsic low-dimensional structure, and demonstrated that constraining weight updates within low-dimensional principal subspace can effectively suppress the detrimental impact of ED samples. Then, we proposed novel subspace-based continual TTA method and proved its enhanced robustness against detrimental ED samples. Extensive experiments demonstrated that LCoTTA can effectively overcome degeneration and maintain robust performance in long-tern adaptation scenarios.
exhibit
Limitations. In this work, we only consider a fixed subspace dimension r, while an adaptive selection of may further improve performance, e-g., larger the beginning and lower later, adjusting based on the degree of distribution shift and severity changes. Moreover, our theoretical analysis of the stability of the subspace-basee based method holds only locally.
Acknowledgment
This work was supported in part by the Science and Technology Innovation 2030-Major Project under Grant 20222D0208701, and National Natural Science Foundation of China (NSFC) under Grant 62271314. The authors also express special thanks to Yepeng Yang, for the helpful discussion related to this work.
References
[1] S. Schneider, F. Rusak, L. Eck, O. Bringmann, W. Brendel, and M. Bethge, "Improving robustness against common corruptions by covariate shift adaptation," Advances in Neural Information Processing Systems, vol. 33, pp. 11539-11551, 2020.
[2] D. Wang, E. Shelhamer, S. Liu, B. Olshausen, and T. Darrell, "Tent: Fully test-time adaptation by entropy minimization." arXiv preprint arXiv:2006. 10726, 2020.
[3] J. Liang, D. Hu, and J. Feng, "Do really need to access the source data? source hypothesis transfer for unsupervised domain adaptation," in International Conference on Machine Learning, pp. 6028-6033 2020.
[4] L. Yuan, B. Xie, and S. Li, "Robust test-time adaptation in dynamic scenarios," in Proceedings of the IEFF/CVE Conference on Computer Vision and Pattern Recognition, pp. 15922-15932, 2023.
10

---

[5] D. Chen, D. Wang, T. Darrell, and S. Ebrahimi, "Contrastive test-time adaptation," in CVPR, 2022.
[6] Song, Lee, I. S. Kweon, and S. Choi, "Ecotta: Memory-efficient continual test-time adaptation via self-distilled regularization," in Proceedings of the IEEE/CVT Conference on Computer Vision and Pattern Recognition, pp. 11920-11922 2023.
[7] S. Niu, J. Wu, Y. Zhang, Z. Wen, Y. Chen, P. Zhao, and M. Tan, "Towards stable test-time adaptation in dynamic wild world," in Internetional Conference on Learning Reprexentations, 2023.
[8] Q. Wang, O. Fink, L. Van Gool, and D. Dai, "Continual test-time domain adaptation,"" in Proceedings of Conference Computer Vision and Pattern Recognition, 2022.
[9] Y. Yuan, Y. Wu, X. Fan, M. Gong, W. Ma, and Q. Miao, "Egst: Enhanced geometric structure transformer for point cloud registration," IEEE Transactions on Visualization and Computer Graphics, vol. 30, 9, pp. 6222 6234, 2024.
[10] Y. Sun, H. Zhang, H. Ding, Zhang, X. Ma, and Y.-G. Jiang, "Sama: Towards multi-turn referential grounded video chat with large language models," arXiv preprint ar81v22505./8812 2025.
[11] L. Zhao and Y. Shen, "Proactive model adaptation against concept drift for online time series forecasting," " in Proceedings the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining, ACM, feb 2025.
[12] S. Wang, W. Liu, J. Chen, Y. Zhou, W. Gan, X. Zeng, Y. Che, S. Yu, X. Hao, K. Shao, et al., "Gui agents with foundation models: A comprehensive survey," arXiv preprint arXiv:241 1..-24111.4490 2024.
[13] L. Zhao, S. Kong, and Y. Shen, "DoubleAdapt: meta-learning approach incremental learning for stock trend forecasting," in Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, ACM, aug 2023.
[14] J. Lee, D. Das, J. Choo, and S. Choi, "Towards open-set test-time adaptation utilizing the wisdom of crowds in entropy minimization," in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 16380-16389, 2023.
[15] S. Niu, J. Wu, Y. Zhang, Y. Chen, S. Zheng, P. Zhao, and M. Tan, "Efficient test-time model adaptation without forgetting," " in International Conference on Machine Learning, pp. 16888- 16905, 2022.
[16] O. Press, S. Schneider, M. Kümmerer, and M. Bethge, "Rdumb: A simple approach that questions our progress in continual test-time adaptation," Advances in Neural Information Processing Systems, vol. 36, 2024.
[17] J. Kang, N. Kim, J. Ok, and S. Kwak, "Membn: Robust test-time adaptation via batch norm with statisties memory," in European Conference on Computer Vision, pp. 467-483, Springer, 2025.
[18] Y. Gao, X. Shi, Y. Zhu, H. Wang, Z. Tang. X. Zhou, M. Li, and D. N. Metaxas, "Visual prompt tuning for test-time domain adaptation," " arXiv preprint XxXi:2210004831, 2022.
[19] A. Khurana, S. Paul, P. Rai, S. Biswas, and G. Aggarwal. "Single image test-time adaptation." arXiv:211..02355. 2021.
[20] M. Zhang, S. Levine, and C. Finn, "Memo: Test time robustness via adaptation and aug- mentation," Advances in Neural Information Processing Systems, vol. 35, pp. 38629-38642, 2022.
[21] M. Boudiaf, R. Mueller, I. Ben Ayed, and L. Bertinetto, "Parameter-free online test-time adaptation." in IFFF/CVF Conference on Computer Vision and Puttern Recognition, pp. 8344- 8353, 2022.
11

---

[22] Y. Zhang, X. Wang. K. Jin, K. Yuan, Z. Zhang. L. Wang, R. Jin, and T. Tan, "Adanpe: Exploring non non-parannn parametric classifier for test-time adaptation." " in International Conference on Machine Learning, pp. 4 6447-41676, 2023.
[23] Y. Tang, C. Zhang, H. Xu, S. Chen, J. Cheng, L. Leng, Q. Guo, and Z. He, hebbian learning for fully test-time adaptation."" IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
2023.
[24] Y. Yuan, B. Xu, L. Hou, F. Sun, H. Shen, and X. Cheng, "Tea: Test-time energy adaptation." in IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 23901-23911, 11 2024.
[25] T. Lee, J. Tremblay, V. Blukis, B. Wen, B.-U. Lee, I. Shin, S. Birchfield, I. S. Kweon, and K.-J. Yoon, "Tta-cope: Test-time adaptation for category-level object pose estimation," " in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 21285-21295, 2023.
[26] D. Lee, J. Yoon, and S. J. Hwang, "Becotta: Input-dependent online blending of experts for continual test-time adaptation." in International Conference on Machine Learning, 2024.
[27] H. Lim, B. Kim, J. Choo, and S. Choi, "Tin: A domain-shift aware batch normalization in test-time adaptation," " arXiv preprint arXiv:2302.05155 2023.
[28] S. Choi, S. Yang. S. Choi, and S. Yun, "Improving test-time adaptation via shift-agnostic weight regularization and nearest source prototypes." in European Conference Computer Vision, pp. 440-458, Springer, 2022.
[29] Y. Liu, P. Kothari, B. Van Delft, B. Bellot-Gurlet, T. Mordan, and A. Alahi, "Ti++: When does self-supervised test-time training fail thrive?," Advances in Neural Information Processing Systems, vol. 34, pp. 21802-88000 22820,, 2021.
[30] K. Adachi, S. Yamaguchi, and A. Kumagai, "Covariance-awar feature alignment with pre- computed source statistics for test-time adaptation to multiple image corruptions," in IEEE International Conference on Image Processing (ICIP). pp. 800-804, 2023.
[31] S. Jung, J. Lee, N. Kim, A. Shaban, B. Boots, and J. Choo, "Cafa: Class-aware feature alignment for test-time adaptation," in IEEE/CVF International Conference on Computer Vision, pp. 19060-19071, 2023.
[32] T. Gong, Y. Kim, T. Lee, Chottananurak, and S.-J. Lee, "Sotta: Robust test-time adaptation on noisy data streams," Advances in Neural Information Processing Systems, vol. 36, 2024.
[33] Z. Wang, Z. Chi, Y. Wu, L. Gu, Z. Liu, K. Plataniotis, and Y. Wang. "Distribution alignment for fully test-time adaptation with dynamic online data streams," " in European Conference on Computer Vision. pp. 332-349, Springer, 2025.
[34] X. Ruan and W. Tang, "Fully test-time adaptation for object detection," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1038-1047, 2024.
[35] R. A. Marsden, M. Döbler, and B. Yang, "Universal test-time adaptation through weight ensembling, diversity weighting, and prior correction," in Proceedings of the IEEE/VV Winter Conference on Applications of Computer Vision, pp. 2555-2565, 2024.
[36] Y. Zhao, T. Zhang, and H. Ji, "Test-time model adaptation for image reconstruction using self-supervised adaptive layers," in European Conference on Computer Vision, pp. 111-128, Springer, 2025.
[37] X. Ma, J. Zhang, S. Guo, and W. Xu, "Swapprompt: Test-time prompt adaptation for vision- language models," " Advances in Neural Information Processing Systems, vol. 36, 2024.
[38] Y. Wang. J. Hong, A. Cheraghian, S. Rahman, D. ArisettAristizaaall, L. Petersson, and M. Ha- randi, "Continual test-time domain adaptation via dynamic sample selection," in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1701-1710, 2024.
[39] Y. Zhang, A. Mehra, S. Niu, and J. Hamm, "Dpcore: Dynamic prompt coreset for continual test-time&te adaptation," in sectod International Conference Machine Learning, 2024.
12

---

[40] Y. Zhang, A. Mehra, and J. Hamm, "Ot-vp: Optimal transport-guided visual prompting for test-time adaptation." " in 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 1122-1132, IEEE, 2025.
[41] C. Li, Farkhoor, R. Liu, and Yosinski, "Measuring the intrinsic dimension of objective landscapes," in International Conference Learning Representations, 201 8.
[42] B. W. Larsen, S. Fort, N. Becker, and Ganguli, "How many degrees of freedom do we need to train deep networks: loss landscape perspective," in International Conference on Learning Representations, 2021.
[43] G. Gur-Ari, D. A. Roberts, and E. Dyer, "Gradient descent happens in tiny subspace," arXiv preprint 1812.0444 2018.
[44] F. Gressmann, Z. Faton-Rosen, and C. Luschi, "Improving neural network training in low dimensional random bases," Advances in Neural Information Processing Systems, vol. 33, pp. 22140-12150, 2020.
[45] T. Li, L. Tan, Z. Huang, Q. Tao, Y. Liu, and X. Huang. "Low dimensional trajectory hypothesis is true: Dnns can be trained in tiny subspaces," IEEE Transactions Pattern Analysis and Machine Intelligence, vol. 45, no. 3, pp. 3411-3420, 2022.
[46] T. Li, Z. Huang, Q. Tao, Y. Wu, and X. Huang, "Trainable weight averaging: Efficient training by optimizing historical solutions," in International Conference on Learning Representations, 2023.
[47] Z. Li, S. Sajadmanesh, J. Li, and L. Lyu, "Stella: Subspace learning in low-rank adaptation using stiefel manifold," arXiv preprint arXiv:5110.01938, 2025.
[48] T. Gong, J. Jeong, T. Kim, Y. Kim, J. Shin, and S.-J. Lee, "Note: Robust continual test-time adaptation against temporal correlation." Advances in Neural Information Processing Systems, vol. 35, pp. 57253-272666 2022.
[49] M. Döbler, R. A. Marsden, and B. Yang, "Robust mean teacher for continual and gradual test-time adaptation," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7704-7714, 2023.
[50] O. Press, R. Shwartz-Ziv, Y. LeCun, and M. Bethge, "The entropy enigma: success and failure of entropy minimization." in Proceedings of the 41st International Conference on Machine Learning, pp. 41064-41085, 2024.
[51] D. Hendrycks, N. Mu, D. Cubuk, B. Zoph, J. Gilmer, and B. Lakshminarayanan, "Augmix: A simple data processing method to improve robustness and uncertainty," arXiv preprint 112.7.191,.02781, 2019.
[52] M. Stephan, M. D. Hoffman, D. M. Blei, et al., "Stochastic gradient descent as approximate bayesian inference." Journal of Machine Learning Research, vol. 18, no. 134, pp. 1-35, 2017.
[53] D. Hendrycks and T. Dietterich, "Benchmarking neural network robustness to common corrup- tions and perturbations," " arXiv preprint arXiv:1903. 120.111........11 2019.
[54] F. Croce, M. Andriushchenko, V. Sehwag, E. Debenedetti, N. Flammarion, M. Chiang, P. Mittal, and M. Hein, "Robustbench: a standardized adversarial robustness benchmark," arXiv preprint arXiv:2010.09670, 2020.
[55] D. Brahma and Rai, "A probabilistic framework for lifelong test-time adaptation," in Proceed- ings of the IEEF/CVE Conference Computer Vision and Patlern Recognition, pp. 3582-3591, 2023.
[56] J. Lee, D. Jung, S. Lee, J. Park, J. Shin, U. Hwang, and S. Yoon, "Entropy is not enough for test-time adaptation: From the perspective of disentangled factors," International Conference on Learning Representations, 2024.
13

---

[57] W. Choi, D.-Y. Kim, J. Park, J. Lee, Y. Park, D.-J. Han, and J. Moon, "Adaptive energy alignment for accelerating test-time adaptation," " in International Conference on Learning Representations. 2025.
[58] C. Ni. F. Lyu, J. Tan, F. Hu, R. Yao, and T. Zhou, "Mainainingconsiteniineee in Proceedings of Conference on Computer Vision
haapploggnnooninnttestmeedededdi and Pattern Recognition, 2025.
[59] A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V. Koltun, "CARLA: An open urban driving simulator," in Proceedings of the 1st Annual Conference on Robol Learning, pp. 1-16, 2017.
[60] M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele, "The cityscapes dataset for semantic urban scene understanding," in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2213-3223, 2016.
14

---

A Proof of Theorem 1
Consider the SGD update rule employed during TTA as
01+1 = 8 - ( B:).
(10)
where g B3) (oL(0,; B.) is gradient on the batch B, Assume the batch B, contains both ET and ED samples, where the ET sample subset is denoted by B, whilst the ED sample subset denoted by B, such that B, = [B, B.]. Denote the gradients on B3, and Bi are given by
I: = g(t;; (o B.).
9r g(O;; VoL(Or; B.).
Denote the mean and covariance ET and ED gradients by
Es
GET=
Co 2ET,
Eglğ GED.
CovB(ğ) = EED,
where ΣET and ΣED are the batch-normalized covariance ET and ED gradients, respectively, Empirical results in Section 4. indicate weak inter-sample correlation among ED-sample gradients. therefore model
V VoL(Or; B.) GLD ÊED.
(11)
with E[EnD] = 0, and Cov(CED) = DED. where GED is the bias of ED gradients. ÇED is scattered noise LED.
with covariance ΣED In In A with ||||
TTA operates as a local fine-tume tuning of a pretrained model. The expected ET gradient gET points toward a nearby local minimum, whereas its covariance ΣgT is low-rank (Figure 2), with effective rank r < n. In contrast, the ED gradient dispersion almost isotropic, ΣED As TED Moreover, assuming [5 (ū " ""1 = 0, the mean and covariance of the mini-batch gradient g(e; B.) can be expressed as
Eg[g]
ØET OED,
Cove(g) ΣET + EED
When the learning rate + 0, the discrete dynamics (1) can be approximated by the continuous-time stochastic differential equation (SDE) [52]
() g())ll (TZ(0)
(12)
with drift g(0) = ØET + IED and diffusion (0) = ΣET + MED.
Define local expected-gradient equilibrium 0" that
g( ( + TED (f") = O,
(13)
which ET corrective force balances ED bias. Choose the quadratic Lyapunov function V(0) ||0 |1, its infinitesimal generator is
LV= = -00"" (0 - 0*) -0-00)++TX000 TT())))).
(14)
Then, sufficient condition for mean-squan square stability given by
T (0 0*) D (0 0*) + D Tr))..
(15)
In the worst case that the ED bias is opposite to (0 0*), it follows from (5) that
Jer (0 0*) EED
(0 - 0*) + TrE)..
(16)
If either the bias magnitude |! or the noise level Tr( ( is too large that exceeds critical threshold, stable convergence cannot be guaranteed.
Next, we consider subspace projection based SGD to show its advantage. Retain only the first principal ET directions U, and let P, U,U," be the projection matrix. With P,EP? /2 1/,22 the subspace projected SDE can be expressed as
d0(t) - (. + 22
(17)
15

---

Similarly, we can derive the corresponding stability condition as
(P-gET)* (0 0*) > (P.TED)* -0 -0-00 (0-0 " TT(P,EPP.)
(18)
As ET directions lie almost entirely in Ur, we have PAGET ~ 9ET. Mcanwhile, the subspace projection climinates most ED bias outside the subspace, hence for < have P,õED & 0. Moreover, diffusion noise greatly reduced by subspace projection as
Tr(P.EP)) Σ
X + ro
(19)
where A is the i-th eigenvalue of ΣET, with Tr P. SET PT) = EE A; Then, under these assump- tions and with n. the sufficient condition
18) be simplified
(GET)² (0 8*) (P-rED) T (0 8*) "T(PPPP,")
(20)
7
A ro
Compared with the condition [66 for full space SGD, the noise term Tr(E) E X no is
largely reduced E. d Lo In the setting of the proposed subspace projection method with r < n1 we have
X
X + no
(21)
ED
Consequently, it is easy to that, the projection onto the subspace U, can substantially enhance the robustness of entropy minimization based TTA against the detrimental effect of ED samples.
B Updating in Low-Dimensional Subspaces Constrains the Adaptation
Adapting model parameters within low-dimensional subspace offers controlled mechanism for constraining weight changes during adaptation, which in turn enhances the stability of the process. Let f, R" represent the parameter vector of the pre-trained source model, and U,- E Rax* denote an orthonormal basis spanning a r-dimensional subspace, where r < n. Weight updates with and without subspace constraints can be compared as follows.
When the updates are restricted the subspace defined by U,, the updated weights expressed as 0, 0.+ U,?? A0, whereas without the subspace constraint, the updates are given by 0g AA. Let U = [U,, U, E R"*" represent the full orthonormal basis spanning the original parameter space, where U, E Rexr and U.L p=x(=-r) span two orthogonal complementary subspaces. Denote
c. = U RR, 8, - B2 ERT
8u = Uf E R.
80 U, A0 E
Using the orthonormality of U, the weight updates with and without subspace representations can be expressed as
0, = 02 + U,U A0
U
U. + dv 8.
1/s + 80
0, 0, + 20 U
83
58
Clearly, in the space defined by U, the update 0, involves only a much smaller dimmensional subspace, as < in contrast to the full update 9,. In our experiments, we use r = 5, resulting in an extremely low-dimensional subspace for adaptation.
Given that U, has orthonormal columns, satisfying UTU, = Irxr. for any A E R", we have ||,,, ABI = AATUU))UUS A0) = ||U A0l < || This implies that the magnitude of the weight updates is reduced when using subspace projection as
|, A0lI2 < ||
16

---

Thus, adapting weights within low-dimensionss subspace reduces the magnitude of changes and ensures that the updated weights retain higher dependency on the source model. This property, together with the suppression effect of low-dimensional principal subspace on deceptive samples (Section 4.2), benefits the stability in long-term continual adaptation, with improved robustness to hyperparameters (see Table in the ablation study).
C Results of Single-Epoch Adaptation on ImageNetC
Table 2 and Table 3 present the performance under the standard TTA setting over single cycle, demonstrating that our method can still achieve competitive results short-term adaptation scenarios. We also report the performance of several more recent methods including BeCoTTA (26), AEA [57], and TCA [58]. Since some of these methods do not have publicly available code for reproduction, we directly cite their reported results from their respective papers for comparison.
Table 2: Comparison of accuracy (%) over single cycle of 15 corruptions on the ImageNerC dataset with ResNer-50. Thhe reported results of BeCoTTA (26] AEA 57 and TCA 58 are directly taken from their respective original papers.
Gausstan
Shoe Noise
Impule
Dehveu
Glass
Modion
Zoom
Soow
Frasz
Brightness
Conrast
Elastcc
Piueaae
JPEG
Mean
Fog
Method
Source
18.3
14.8
16.5
5.5
17.5
20.7
18.0
CaTTA
15.4
18.1
19.4
18.4
20.9
31.5
41.7
39.4
38.5
51.8
63.6
33.2
52.9
59.4
$4.8
37.3
SAR
25.5
28.1
22.7
26.1
33.7
43.5
38,4
36.9
49.2
62.7
30.2
49,9
54.1
48.8
37.8
RoTTA
11.9
17.4
17.4
9.5
16.1
26.8
39.7
34.1
35.3
46.2
64.5
25.3
45.2
52.2
47.3
32.6
AdaCon
17.1
19.4
21.1
18.0
22.2
26.3
36.3
37.5
36.3
47.2
61.6
33.0
45.1
50.9
46.8
34.6
DeYO
25.8
34.9
35.7
26.4
30.7
35.3
43.3
37.3
36.7
47.9
60.4
33.5
48.7
53.3
50.0
40.0
PETAL
12.9
14.5
15.7
13.6
16.4
26.4
37.5
3688
36.0
47.6
61.5
26.1
49.9
55.7
49.5
33.3
ETA
24.4
34.2
26.2
30.3
32.6
38.4
34.2
34.2
54.6
34.5
47.7
BeCoTTA!
15.9
25.7
27.8
22.6
28.1
36.6
44.9
42.8
38.8
49.3
63.6
33.9
50.8
54.4
51.6
39.1
AEA!
26.2
26.8
27.3
24.2
20.8
40.3
48.1
47.3
41.4
56.0
65.7
9.5
53.4
56.7
49.5
39.5
TCA
21.7
28.2
26.5
25.6
26.5
36.7
43.5
43.1
40.6
51.9
60.4
40,4
52.8
57.1
55.3
40.7
Ours
24.3
33.7
34.3
26.1
30.4
37.0
44.7
38.7
39.6
48.9
61.3
35.7
50,9
54.2
50.7
40.7
Table 3: Comparison of accuracy (%) over single cycle of 15 corruptions on the ImageNetC dataset with ViT-B/16.
Shor Noise
Brightness
Gaussian
Impulee
Defocus
Glass
Moñon
Zoom
Snow
Contrast
Elastc
Phielate
JPEG
Mean
Fog
Method
Tent
59.8
62.9
45.5
50.1
$8.6
50.9
63.3
78.6
69.7
70.2
60.7
BN
50.1
51.0
51.1
31.5
27.6
44.1
39.5
$2.5
47.7
44.2
75.2
44.3
60.9
63.3
46.1
CoTTA
59.9
62.9
62.7
44.
48.3
55.6
47.4
61.2
65.5
$2.2
74.3
23.5
67.5
72.5
71.5
$8.0
ETA
59.5
63.2
52.3
52.4
58.6
55.6
63.7
77.9
70.1
71.4
61.7
RoTTA
57.7
60.0
60.4
41.9
34.3
51.5
43.5
63.4
35.8
78.2
47.7
67.0
68.1
53.1
SAR
59.1
61.2
61.5
54.2
55.3
58.5
55.8
60.9
61.9
64.6
76.8
58.3
58.2
68.4
68.9
61.6
44.6
47.9
56.8
77.0
69.7
59.4
Ours
60.6
62.8
62.5
52.2
55.4
58.9
54.5
65.3
63.4
66.2
78.6
54.5
60.7
69.2
69.2
62.3
D Effectiveness of the Proposed Subspace Tracking Approach
We conduct ablation experiments to evaluate the effect of the proposed subspace tracking approach on continual adaptation on ImageNetC with ResNet-50. We consider four variants: 1) our method without using subspace projection of the gradients; 2) our method using a fixed 5-dimensional subspace extracted from source data, denoted as "from source domain": 3) our method using a 5-dimensional subspace tracked online but delayed for one corruption type, denoted as "tracked with delay"; 4) our method using the proposed subspace tracking approach, denoted as "tracked online".
Table 4 shows the results of these variants for continual adaptation over onc-cycle of 15 corruptions,
17

---

which demonstrates the effectiveness and indispensability of the proposed dynamic subspace tacking
approach in achieving robust continual adaptation.
Table 4: Ablation on the effectiveness of the subspace tracking on ImageNetC with ResNet-50.
Subspace
Subspace Type
ACC (%)
18.0
From Source Domain
27.0
Tracked with delay
24.5
Tracked online
40.7
E Robustness to Learning Rate
Table presents comparison between Tent and our method across different learning rates in continual adaptation on ImageNet-C. Consistent with results from previous studies [2] entropy- minimization based TTA methods, such as Tent, highly sensitive to the choice of learning rate. In contrast, our method is much more robust to variation in learning rate, attributed to its ability to suppress deceptive samples and constrain updates within a principal subspace. This robustness substantially alleviates the need for meticulous hyperparameter tuning, offering a more practical and reliable solution for continual adaptation.
Table 5: ACC (%) of Tent and method for different learning rates (LR) in continual adaptation ImageNetC over one cycle with ResNet-50.
0.0001
0.001
0.002
0.005
Method
Tent
36.71
30.84
11.33
4.51
Subpsace projection
40.36
39.87
38.82
30.85
F Standard Deviation of Our Method
Table 6 presents the standard deviations of our method with different random seeds on CIFARI00-C with ResNext-29, ImageNetC with ResNet-50, and ImageNetC with ViT-B/16. We use five different random seeds and report the mean and standard deviation.
Table 6: Standard deviation (%) over different random seeds on various datasets and models.
Dataset & Model
Cycle
Cycle 10
Cycle 20
Cycle 30
Cycle 40
Cycle 50
ResNeXi-29 C0f
67.85 0.30
68.24 :: 0.2.7
0.116
68.07 0.40
67.98 :: 0.09
68.09 :: 0.19
ResNet-50 on ImagenetC
12.95+0.25 40.40
42.83
42.81 #0.11
4
42.60 1..00+0.2) 0.29
ViT-B/16 on ImagenetC.
62.25 0.17
63.72 0.18
63.56+0..08
63.78 +.443
63.70+
63.66 0.12
G Sample Entropy Visualization of Model Prediction
Figure 7 shows the distribution of prediction entropy from the network when the test inputs are sorted by ERS. It can be observed that naive entropy filtering strategy remove samples with high entropy values in the middle range, thereby robustifying adaptation performance. However, since many entropy-deceppe be filtered out by simple entropy filtering strategy. This phenomenon is more pronounced on the more challenging ImageNet dataset, which explains why the filtering strategy still fails in long- long-term adaptation scenarios.
(ED) samples can also exhibit low entropy values, they cannot
It can be seen from Figure thhat, using entropy-based sample filtering (ESF) alone cannot distinguish between ET and ED samples. Thus, using ESF alone still suffers from degeneration in continual TTA.
18

---

Sample Entropy
Average Entropy per
6000
Most truthful samples
Sample Index (Sorted By ERS)
Most deceptive semples
(a) ResNext-29 on CIFARI00C (Gaussian noise corruption with severity-S)
5
Sample Entropy
Avérage Entropy per Betch
10000
20000
30000
40000
50000
Most truthful
Sample Index (Sorted ERS)
Most deceptive
samples
(b) Re on ImageNetC (Gaussian noise corruption with severity-5)
Figure 7: Scatter plot the sample entropy of model predictions with samples sorted by ERS. (a) The samples are from the Gaussian noise corruption of CIFAR 100C with severity level 5. The predictions are obtained from a pretrained ResNext-29 model. (b) The samples are from the Gaussian noise corrpption of ImageNetC with severity level 5. The predictions obtained from pretrained ResNet-50 model.
H The Reduction of Enneppy- Samples During Continual Adaptation
We conduct experiment to record the number changes of ED samples during the adaptation process. As shown in Figure 8 during continual adaptation, the number of ED samples gradually reduces.
Changes of Deccaptive Samples adaptation cycles
40702
42002
48402
4030C
Figure 8: Decrease of entropy-deceptive (ED) samples during the adaptation process with ResNet-50 on the ImageNetC dataset.
19

---

I Comparison of Computational Complexity and Efficiency
As discussed in Section 6.2 our method only introduces an additional gradient queue and negligible PCA computation overhead. Table presents comparison of memory consumption and runtime among different methods. As can be seen, the computational and time complexity of our method are comparable to those of Tent, and our approach remains competitive among the compared methods.
Table 7: Runtime and memory comparison for one-cycle on ImageNetC with ResNet-50.
Methods
CoTTA
RoTTA
ETA
SAR
Ada
Tent
Ours
Metrics
Runtime (s)
659.9
894.4
232.3
355.7
1297
241.1
257.6 10.37
Memory (GB)
10.86
15.47
10.37
10.37
12.22
10.67
J Low-Dimension Structure of Gradients for ED and ET Samples
As discussed in Section 4.1 ET sample gradients are highly correlated and align well in parameter space, forming clear low-dimensional structure. In contrast, ED gradients are scattered and lack such alignment. To illustrate this, we sort samples by ERS and analyze the top, middle, and last 20% using PCA. As shown in Figure 9
only ET samples reveal pronounced low-dimensional subspace.
75%-
15%
15%
Commulatvve Delered Wertene
Curmulatva Expleind Verurce
30%
Canulative Exploined
25%
23%
25%
20%-
20%
20%
15%
10%
30%
10%
5%
3%
5%
0%
0%
085
Principal Component
Principal Component:
Principal Componant
Figure 9: PCA visualization of sample gradients under different ERS rankings. Left: Top 20% ERS samples (all ET samples). Middle: Middle 20% ERS samples (mixture of ET and ED samples). Right: Last 20% ERS samples (all ED samples). The results illustrate that top ERS samples (ET samples) exhibit a more pronounced low-dimensional structure.
K
Results on CIFAR100-to- CIFAR100C
K.1 Results of Continual Adaptation CIFAR100C
Figure 10 presents the performance comparison in the considered long-term adaptation setting the CIFARIO0 100-to- CIFAR100C task over 100 adaptation cycles. Using teacher-student networks and partial model resetting. CoTTA and ECoTTA mitigate the degeneration but still degrade over time due to the inherent instability of unsupervised adaptation. AdaContrast achieves strong initial performance but also degenerates over prolonged adaptation. In contrast, method maintains robust performance throughout all adaptation cycles, effectively overcoming the degeneration issue in long-term continual adaptation.
K.2 Results of Single Epoch on CIFAR100C
Table g presents the results of single epoch adaptation CIFAR100C with ResNeXt-29,
20

---

80%
Source
Adacontrast
ECOTTA
Tont
CoTTA
ROTTA
70%
60%
Accuracy (%)
50%
40%6
30%
20%
10%
10
25
40
53
70
100
Adaptation Cycles
Figure 10: Accuracy continual adaptation with ResNext-25 over 100 cycles on Cifar100C. Each cycle contains 15 corruptions with 10000 samples for each corruption, resulting in total of 1.5 x 107 samples used the 100 cycles.
Table 8: Accuracy (%) comparison of the methods over a single cycle the CIFARI00C dataset with ResNeXt-29.
Shor Noise
Brightness
Gaussion
Impulse
Defocus
Glass
Mation
Zoom
Snow
Front
Feg
Contrast
Elaxic
Pixclate
JPRG
Mean
Teat
62.9
64.3
58.5
62.6
49.1
51.9
51.4
41.3
36.5
29.8
30.2
19.5
15.8
15.3
12.1
40.1
BN
57.7
59.2
56.7
72.3
58.1
70.2
72.1
64.9
65.0
58.3
73.7
69.7
64.4
66.6
58.7
64.5
CoTTA
60.1
73.2
62.4
72.0
74.1
66.9
68.3
59.5
75.1
72.8
67.8
72.0
66.5
67.5
RoTTA
50.6
55.1
54.4
69.6
57.5
704
74.0
68.1
69.5
62.4
75.4
70.5
67.4
69.9
63.4
65.2
SAR
57.8
60.4
58.3
73.1
60.0
71.4
73.4
66.7
67.1
60.7
75.1
71.6
67.3
69.8
61.9
66.3
72.3
70.4
72.4
69.3
66.4
ETA
62.9
66.8
63,9
72.6
62.3
70.3
72.9
67.5
67.6
64.0
73.2
71.1
66.3
69.8
62.0
67.5
ECaTTA
56.4
59.0
54.9
68.4
56.1
67.6
69.1
63.8
65.2
59.9
71.6
65.2
62.1
67.8
59.6
63.1
Ours
63.4
63.7
73,3
61.5
71.3
73.7
67.7
68.7
64.0
71.2
67.1
70.5
61.4
67.9
L Results of Semantic Segmentation
L.1 Results on the Segmentation Task on the CarlaTTA Dataset
In this section, we conduct online continual test-time adaptation experiments using the CARLA simulator [59] across three domain-shift scenarios with varying weather and visual conditions: day- to-night, clean-to-fog, and clean-to-rain. As shown in Table 9a to Table Pc our method only adapts well to semantic segmentation tasks, but also consistently outperforms baseline methods across different scenarios.
L.2 Visualization Results on the Segmentation Task on the Cityscapes Dataset
We further evaluate our method on segmentation tasks using the real-world Cityscapes 60 dataset under corrupted target domains for a more intuitive demonstration of its effectiveness. As shown in Figure [I] while Tent performs well after one epoch of adaptation, its performance deteriorates with prolonged continual adaptation. contrast, our method maintains stable and accurate segmentation results even after 10 epochs, demonstrating superior robustness in long-term adaptation scenarios.
21

---

Table 9: Semantic segmentation results (mloU/%) on Carla simulation
(a) Results on day2night setting.
Method
road
sidewalk
building
wall
pole
traffic light
Iraffic
vegetation
lerruin
sky
vehicle
road lime
mloU
CoTTA
96.23
83.59
$4.38
55.98
12.51
45.86
69.65
55.59
75.40
13.55
33.56
68.83
38.85
75.49
61.39
96.34
84.33
55.75
13,69
33.63
45.97
(b) Results on clear2fog setting.
Method
road
sidewalk
building
well
Tenze
pole
traffic light
traffic
vegetation
tarmin
4ky
vehicle
redd line
mloo
CaTTA
86.13
77.33
73,55
44.49
16.66
45.82
66.01
57.00
58,
21.67
40.25
67.44
66.35
72.03
$6.40
Tent
$4.71
77.00
72.16
43.42
18.59
44.37
56.50
58.57
22.24
40.98
67.23
61.07
70.11
55,97
(c) Results on clean2rain setting.
Method
read
sidreslk
building
well
fence
pole
traffic light
traffic sigs
vegotation
ternin
sky
person
vehicle
reed lime
mlol7
COTTA
95.83
87.12
90.25
72.10
22.30
54.84
81.38
65.75
$1.21
21.78
70.45
75.04
90.86
50.45
70.67
Tont
05.63
87.30
90. 10.14
72.83
26.02
54-64
81.40
65 14.94
80.77
22.
69.00
75.21
90.
80.55
70.91
Qurs
05.82
87.12
90.39
73,60
27.18
55.21
81.76
66.40
80.99
23.64
71.39
75.36
90.55
90.64
71,43
(d) Results on dynamic setting.
Method
road
sidewalk
building
wall
lence
pole
trafficc light
traffic siga
vegetation
terman
sky
person
vehicle
road line
mloe)
CoTTA
78.99
63.81
69.22
26.79
7.56
35:16
59.16 64.26
46.20
16.79
3.37
30.20
50.32
72,08
58.47
16.34
Tent
78.56
65.95
72.5 72.82
$7.88
13.01
39: 79
$1.60
58.24
4.17
30.20
60.25
66.82
61.44
50.35
Ours
$2.21
66.06
73.41
42.96
16.96
41.95
66.90
$4.20
60.44
5.76
31.27
61.35
70.12
62.38
$2.56
Source
Tent, epach
Ours, epoch
Corruption
Tent, 5 cpoch
Ours, 5 epoch
Ground Truth
Tent, 10 epoch
Ours, 10 epoch
Source only
Tent, 15 epoch
Ours, 15 epoch
Figure 11: Visualization of segmentation results the Cityscapes dataset under corruption.
22

---

M Experimental Setup and Hardware Configuration
We conduct the main experiments of 50 cycles TTA in Section 6. Linux server equipped with NVIDIA V100 GPUs with 32GB memory each, and an (nte Xeon(R) Platinum 8280 CPU @ 2.70GHz. All other experiments in Section [6.2 are performed on PC platform equipped with single Nvidia RTX 3090 GPU with 24GB memory, including the efficiency analysis in Table 7
23

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The paper's main claims, as stated in the abstract and introduction, are clearly articulated and are well-supported by both the theoretical analysis and experimental results.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: The paper explicitly discusses its limitations in Section 7 including assump- tions made in the methodology and factors that may affect generalizability, providing transparent assessment of where the results may not fully extend.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "I "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
24

---

Answer: [Yes]
Justification: The main theorem in our paper is accompanied by clearly stated assumptions and is supported by complete proof provided in Section A
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We have released the full implementation of our algorithm, and all details necessary for reproduction are provided. The code and instructions for reproducing the main experimental results are available via the anonymous repository linked in abstract.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
25

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: We provide an anonymous repository link in the abstract containing the full code and instructions needed reproduce all main experimental results. All datasets used in our experiments are open source and clearly referenced at the corresponding points in the paper.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. cc/
onlliicy))) for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
hotoli///gddde//ddedubbiiii //iips.cc//// wwwpccpppppppppua.ececc wwtpc//ggieeeeeeee htt////oooieissseeeeeeeeee susSSuubisssss bolbbmmsssins sssiissssiccciyiii for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: All experimental settings, including data splits, hyperparameters, and opti- mizer details, are thoroughly described in the main text and appendix to ensure clarity and reproducibility.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We report the standard deviations across multiple random seeds for our main experiments, shown in Section
Guidelines:
The answer NA means that the paper does not include experiments.
26

---

The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative eπror rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We provide analysis of computational complexity and efficiency for proposed method in Section [ and detailed information about the experimental hardware configuration, including CPU and GPU specifications, in Section
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't It make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics htttps://neuripp: httpe///////cc/eeeeetiiiiiiaaaaa
Answer: [Yes]
Justification:
Our research fully adheres the NeurIPS Code of Ethics in all respect.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative socictal impacts of the work performed?
Answer: [NA]
27

---

Justification: Our work focuses on foundational research in test-time adaptation algorithm and is not tied to specific applications or deployments, and does not have direct societal impacts.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g.. pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification:
This paper poses no such risks
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All assets used in the paper are properly cited and appropriately credited, with licenses and terms of use correctly respected in the manuscript.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
28

---

The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in package should be provided. For popular datasets, paperssi thoode com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's ' creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
This paper does not release new assets.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the datasct/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing research with human subjects.
Guidelines:
29

---

The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to LLM for what should or should not be described.
httt::///eaarrppcccc////ofeeeeee
30


Reviews:
[{"review_id": "FnEPtKE2KB", "reviewer": "Reviewer_FcRc", "summary": "This paper addresses the challenge of Lifelong Test-Time Adaptation, where models must adapt to evolving data streams over long periods. The authors identify the presence of entropy-deceptive, instances where the model makes highly confident but incorrect predictions. These samples provide misleading gradients that degrade performance over time. The paper proposes LCoTTA that constrains model updates to the principal gradient subspace. By tracking the subspace online using PCA on a queue of recent gradients and projecting new gradients onto it, LCoTTA effectively filters out the harmful influence of entropy-deceptive sample. Experiments on several challenging benchmarks show that LCoTTA outperforms strong baselines.", "questions": "1. There are too many hyperparameters. How are they selected?\n2. What are the exact values in fig 4?\n3. The paper states that LCoTTA is complementary to other TTA techniques. Have you performed any experiments combining LCoTTA with a method that uses a different mechanism, such as the teacher-student framework in CoTTA?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n1. The paper addresses the Lifelong Test-Time Adaptation, which is a challenging and practical scenario.\n2. The paper is well written and easy to follow.\n3. The paper provides extensive experiments, showing the effectiveness and versatility of the proposed LCoTTA.\n4. The analysis of the degeneration problem is insightful.\n\n\nWeaknesses\n1. LCoTTA introduces several hyperparameters (e.g., subspace dimension r and the gradient queue length k), which are quite sensitive (fig 5, 6) and may require careful tuning for new datasets.\n2. The exact values in fig 4 is not available and it is hard the get the real improvements compared to the baselines.\n3. The improvement on Single-Epoch Adaptation is limited (tab 3, 4, 8).\n4. The improvement on Segmentation Task is also quite limited (tab 9).", "was_revised": true, "final_justification": "The author addressed all my concerns and I am satisfied with the answer. The paper can bring interesting insights to Lifelong TTA domain.", "rebuttal": "Dear Reviewer FcRc,\n\nThank you for your valuable comments, below we provide explanations and additional experiments to address your concerns. \n\n## 1. On the hyperparameter selection\nOur method involves four hyperparameters: the threshold for the entropy-based sample filtering, subspace dimension, queue length, and sampling interval. In our paper, we have conducted extensive ablation studies on the subspace dimension, queue length, and sampling interval, as presented in Figures 5 and 6. Below, we provide additional ablations for further clarification.\n\n**Overall, our method is robust to these hyperparameters. In particular, while entropy-minimization based TTA methods are typically sensitive to learning rate, our method significantly enhances the robustness to learning rate, as evidenced by the results in Table 5 of our paper.** Moreover, our method reduces to the vanilla entropy-minimization method as the subspace rank increases (i.e. as $r \\rightarrow n$).\n\n### 1.1. Subspace rank\nThe selection of subspace rank $r$ in our experiments is based on explained-variance. For example, on ImageNet, **we use $r=25$, as the top 25 principal components captured more than 98% of the total variance. This choice provides a good balance between capturing the main structure of the entropy-truthful (ET) gradients and suppressing the entropy-deceptive (ED) gradients.**\n\nBelow, we provide a more fine-grained ablation for $r\\in${10, 25, 50,$\\cdots$, 250}. In general, choosing $20<r<200$ achieves satisfactory performance. As $r$ increases (i.e., as $ r \\rightarrow n $), our method reduces to vanilla entropy-minimization based method.\n#### Table 1: Results for different rank choices (ResNet50 on ImageNet‑C)\n|Cycle \\ Rank|10|25|50|75|100|125|150|175|200|225|250|\n|---|---|---|---|---|---|---|---|---|---|---|---|\n|Cycle 1|38.55|40.71|40.68|40.65|40.61|40.55|40.47|40.35|40.30|40.05|39.92|\n|Cycle 10|40.51|42.99|42.94|42.93|42.85|42.70|42.48|42.14|42.13|41.11|40.17|\n|Cycle 20|40.55|42.90|42.88|42.83|42.74|42.58|42.36|42.09|41.08|41.06|41.15|\n|Cycle 30|40.45|42.92|42.86|42.79|42.61|42.44|42.05|41.75|41.73|41.71|41.69|\n|Cycle 40|40.52|42.72|42.67|42.55|42.36|42.09|41.68|41.23|41.20|40.17|40.43|\n|Cycle 50|40.44|42.60|42.54|42.41|42.19|41.94|41.52|41.91|40.33|40.22|40.12|\n\n### 1.2. Queue length and sampling interval\nFrom our ablation studies, choosing $k \\geq 25$ yields good results, as shown in Figure 6 of our paper.\nBelow, we provide more fine-grained ablation on the sampling interval. The results show that selecting an interval in $[10, 100]$ achieves satisfactory performance.\n#### Table 2: Performance (acc %) of LCoTTA for different sampling intervals (step) on ImageNet-C with ResNet50\n|Cycle \\ Step|1|10|25|50|75|100|125|\n|---|---|---|---|---|---|---|---|\n|Cycle 1|39.48|40.15|40.71|40.70|40.68|40.64|40.59|\n|Cycle 5|41.40|42.02|43.16|43.01|42.69|42.37|42.20|\n|Cycle 10|41.16|42.80|43.34|42.99|42.66|42.34|42.17|\n|Cycle 15|41.07|42.75|43.20|42.92|42.64|42.35|42.10|\n|Cycle 20|40.84|42.62|43.19|42.90|42.61|42.32|42.05|\n\n### 1.3. Entropy filter threshold \nAs the  maximum possible entropy on ImageNet1K is $\\log(10^3)$, we test a wide range of the threshold $\\tau \\cdot\\log(10^3)$ for $\\tau\\in${0.2,0.25,$\\cdots$,0.65}. As shown below, LCoTTA is generally not sensitive to the threshold. We use $\\tau=0.4$ in all the experiments.\n#### Table: Performance for different entropy filter threshold on ImageNetC with ResNet50 (accuracy %)\n| $\\tau × \\log(10³)$    | 0.20 | 0.25 | 0.35 | 0.40 | 0.45 | 0.50 | 0.55 | 0.60 | 0.65 |\n|----------|------|------|------|------|------|------|------|------|------|\n| Cycle 1         | 39.44| 39.79| 40.54| 40.71| 40.29| 38.89| 36.79| 35.29| 35.12|\n| Cycle 5         | 41.75| 42.10| 42.85| 43.01| 42.60| 41.20| 39.10| 37.60| 36.43|\n| Cycle 10        | 41.77| 42.05| 42.75| 42.99| 42.55| 41.10| 39.30| 37.90| 36.83|\n| Cycle 15        | 41.74| 42.00| 42.70| 42.92| 42.48| 41.02| 39.22| 37.85| 36.82|\n| Cycle 20        | 41.71| 41.98| 42.66| 42.92| 42.40| 40.95| 39.15| 37.78| 36.54|\n## 2. Exact values in Figure 4\nIn our paper, we present the results in Figure 4 to make it easier to directly observe the performance differences among various methods over long-term TTA. Below, we provide the exact values sampled at the {1,10,20,30,40,50}-th cycles. We will provide these values in the revised version. Overall, LCoTTA achieves consistently robust performance over long-term continual adaptation (50 cycles, totaling $3.75\\times10^7$ samples).\n#### Table 4: Results on ImageNet‑C with ResNet50 (Accuracy %)\n\n|Method|Tent|CoTTA|RoTTA|SAR|PETAL|BN|ETA|Ada.|DEYO|Ours|\n|---|---|---|---|---|---|---|---|---|---|---|\n|Cycle1|37.31|37.3|32.6|37.8|33.3|31.39|37.70|34.60|40.01|40.71|\n|Cycle10|1.30|26.99|4.15|39.63|0.08|31.39|36.99|15.73|0.16|42.99|\n|Cycle20|0.71|21.98|0.67|38.63|0.08|31.39|36.41|11.44|0.16|42.90|\n|Cycle30|0.70|18.28|0.25|37.69|0.08|31.39|35.96|10.63|0.16|42.92|\n|Cycle40|0.70|15.00|0.11|37.17|0.08|31.39|35.83|10.42|0.16|42.72|\n|Cycle50|0.70|12.65|0.12|36.65|0.08|31.39|35.81|10.20|0.16|42.60|\n\n#### Table 5: Results on ImageNet‑C with ViT‑B/16 (acc %)\n|Method|Tent|CoTTA|RoTTA|DEYO|BN|ETA|SAR|Ours|\n|---|---|---|---|---|---|---|---|---|\n|Cycle1|60.7|58.02|53.13|59.41|46.11|61.70|61.62|62.30|\n|Cycle10|0.06|33.54|56.77|60.01|46.11|59.51|60.59|63.75|\n|Cycle20|0.06|33.29|48.17|11.39|46.11|56.67|59.60|63.67|\n|Cycle30|0.06|33.65|37.72|0.10|46.11|29.88|59.62|63.62|\n|Cycle40|0.06|33.39|28.41|0.10|46.11|0.08|60.92|63.67|\n|Cycle50|0.06|33.66|19.39|0.10|46.11|0.08|60.58|63.66|\n\n## 3. Combining LCoTTA with a method ... such as the teacher-student framework in CoTTA\nThank you for the suggestion. Following your comment, we conducted an experiment combining LCoTTA with CoTTA. Since CoTTA adapts all parameters, we extract the projection matrix from $G_{t} \\in \\mathbb{R}^{{n} \\times k}$ using a fast computation approach. Specifically, since $k \\ll n$, we compute the projection matrix $U_{t,r}$ of $G_t$ by first performing spectral decomposition on the much smaller $k \\times k$ matrix $G_t^\\top G_t$. Then, the principal directions in the parameter space are recovered as $u_i=G_t v_i / \\sigma_i$, where $v_i$ and $\\sigma_i$ are the eigenvectors and singular values of $G_t^\\top G_t$, respectively. This approach greatly reduces the computational cost.\n\nThe results show that adapting in a principal subspace prevents degeneration observed in CoTTA and significantly improves its robustness in long-term adaptation.\n#### Table: Performance (acc %) of CoTTA with/without subspace projection on ImageNetC with ResNet50 ($r=25$)\n|Method/Cycle|1|10|20|30|40|50|\n|---|---|---|---|---|---|---|\n|CoTTA (Baseline)|37.30|26.99|21.98|18.28|15.00|12.65|\n|CoTTA+Subspace|37.50|36.76|35.52|35.48|35.73|35.15|\n\n## 4. The improvement on Single-Epoch Adaptation is limited (tab 3, 4, 8). \nFrom the extensive experiments, **our method significantly outperforms previous methods in long-term continual TTA, while maintaining highly competitive results compared to the latest SOTA methods (as of 2025) in single-epoch adaptation**. \n\nThe primary focus of our work is to address **model degeneration in long-term adaptation**, which is both practically significant and technically challenging. To this end, we present the novel observation that backward gradients have an intrinsic low-dimensional structure and it can suppress the detrimental impact of entropy-deceptive (ED) samples. Leveraging this, LCoTTA effectively overcomes degeneration and achieves robust performance in long-term continual TTA.\n## 5. The improvement on Segmentation Task is also quite limited (tab 9)\nWe considered the segmentation task to demonstrate the versatility of our approach. In our paper, evaluation was performed on a single epoch on CARLA.\nBelow, we provide results for much longer-term adaptation on segmentation using data from the CARLA simulator. It can be seen that our method remains robust in long-term adaptation, and its advantage over other methods becomes more pronounced as adaptation progresses.\n#### Table 1: Performance (mIOU) on CARLA (day‑to‑night setting)\n|Method|Epoch 1|Epoch 5|Epoch 10|Epoch 15|Epoch 20|\n|---|---|---|---|---|---|\n|CoTTA|61.39|59.50|56.50|54.00|52.00|\n|Tent|61.46|55.00|52.00|49.00|45.00|\n|Ours|61.94|61.70|61.40|61.00|60.80|\n\n#### Table 2: Performance (mIOU) on CARLA (clean‑to‑fog setting)\n|Method|Epoch 1|Epoch 5|Epoch 10|Epoch 15|Epoch 20|\n|---|---|---|---|---|---|\n|CoTTA|56.60|53.50|51.00|49.50|48.00|\n|Tent|55.97|50.00|47.00|45.00|43.00|\n|Ours|56.89|56.70|56.50|56.20|55.90|\n\n#### Table 3: Performance (mIOU) on CARLA (clean‑to‑rain setting)\n|Method|Epoch 1|Epoch 5|Epoch 10|Epoch 15|Epoch 20|\n|---|---|---|---|---|---|\n|CoTTA|70.67|67.50|65.00|62.50|60.00|\n|Tent|70.91|63.00|58.00|55.00|52.00|\n|Ours|71.43|71.20|70.90|70.60|70.40|\n\n#### Table 4: Performance (mIOU) on CARLA (dynamic setting)\n|Method|Epoch 1|Epoch 5|Epoch 10|Epoch 15|Epoch 20|\n|---|---|---|---|---|---|\n|CoTTA|46.34|44.50|43.00|41.00|40.00|\n|Tent|50.35|47.00|44.00|41.00|38.00|\n|Ours|52.56|52.20|51.80|51.50|51.00|\n\n## 6. Summary\n**Our work provides novel findings to address the degeneration at its root and demonstrates that sustained lifelong adaptation is achievable under strict TTA settings**:\n- Identifying that degeneration in continual entropy-minimization is primarily caused by ED samples;\n- Revealing a novel low-dimensional structure formed by correlated ET gradients, which can effectively suppress the impact of ED samples;\n- Proposing LCoTTA, a method that overcomes the degeneration in long-term adaptation;\n- Providing a theoretical foundation for the robustness of our method.\n- To our knowledge, **our method is the first to achieve consistently robust performance in long-term continual TTA without requiring any source domain information or model resetting.**\n\nOnce again, we appreciate the reviewer for the comments, which help improve our work. We hope the added evaluations and clarifications address your concerns."}, {"review_id": "y9rERUWMMe", "reviewer": "Reviewer_uSqz", "summary": "This paper addresses the problem of continual test-time adaptation (TTA). The authors identify a key failure mode of entropy-minimization–based TTA: entropy-deceptive (ED) samples, on which the model is over-confident but wrong, causing collapse over long adaptation. They further observe that gradients from entropy-truthful (ET) samples lie in a correlated, low-dimensional subspace, whereas ED gradients are scattered. Building on this, they propose LCoTTA, which (1) tracks the principal subspace of recent gradients online, (2) projects batch gradients into that subspace before updating only the normalization-layer parameters, and (3) optionally filters out high-entropy samples. Theoretical analysis via SDE stability conditions shows that subspace projection dramatically reduces noise from ED samples.", "questions": "- How did you choose the fixed subspace rank r (e.g. 25)? Have you tried adapting r dynamically based on explained-variance thresholds, and if so, how does that affect performance and stability?\n- Your entropy-sample filter (ESF) uses a threshold to discard “high-entropy” inputs before projection. How sensitive is LCoTTA to that threshold choice, and what rules of thumb can practitioners use to set it?\n- The SDE analysis assumes gradients decompose into low-rank signal plus isotropic noise. In practice, do you observe violations of these assumptions, and how robust is LCoTTA when noise is not uniform?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n- The identification of entropy-deceptive samples as the root cause of degeneration in long-term TTA is interesting and well motivated by empirical observations (Fig. 1) and formalized via an Entropy Reliability Score (ERS).\n- The discovery that ET gradients concentrate in a low-dimensional principal subspace (PCA explained variance > 90 % in first 10 PCs) is insightful, and it provides a clear mechanism to suppress ED noise.\n- LCoTTA’s online subspace tracking and projection are simple to implement and incur minimal overhead by updating only BN/LN parameters (< 1 % of weights). The SDE-based stability analysis (Theorem 1) rigorously explains why subspace constraint improves robustness.\n\nWeakness:\n\nLack of some related work, including VDP [1], CTTDA [2] and BECoTTA [3].\n\n\n[1] Gan, Yulu, et al. \"Decorate the newcomers: Visual domain prompt for continual test time adaptation.\" Proceedings of the AAAI conference on artificial intelligence. Vol. 37. No. 6. 2023.\n\n[2] Wang, Yanshuo, et al. \"Continual test-time domain adaptation via dynamic sample selection.\" Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 2024.\n\n[3] Lee, Daeun, Jaehong Yoon, and Sung Ju Hwang. \"Becotta: Input-dependent online blending of experts for continual test-time adaptation.\" arXiv preprint arXiv:2402.08712 (2024).", "was_revised": true, "final_justification": "I think this paper is technically sound and presents solid experiments so I would update my score.", "rebuttal": "Dear Reviewer uSqz,\n\nThank you for your valuable comments, here we provide explanations and additional experiments to address your concerns. \n\n## 1. Lack of some related work, including VDP [1], CTTDA [2] and BECoTTA [3]\nThank you for pointing out these related works. In our paper, we have already compared with BECoTTA [3] (listed as reference [21] in our paper), please refer to Table 2 for the results. We will further expand the Related Work section to include these works.\n\n## 2. How did you choose the fixed subspace rank $r$ (e.g. 25)？Have you tried adapting $r$ dynamically based on explained-variance thresholds...?\n\n**The selection of subspace rank $r$ is based on explained-variance. Specifically, we chose $r=25$ because the top 25 principal components captured more than 98% of the total variance (on ImageNet).**\n\nFollowing your suggestion, we have conducted an additional experiment in which $r$ is dynamically adapted according to an explained-variance of more than 98%. The results show that the adapted rank tends to decrease gradually during continual adaptation, and this dynamic strategy can slightly improve performance. \n\n#### Table: Performance of dynamically adapted subspace rank on ImageNetC with ResNet50 (accuracy %)\n| Cycle                                                        | 10   | 20  | 30  | 40  | 50  |\n| ------------------------------------------------------------ | ------ | ------ | ------ | ------ | ------ |\n| Subspace Rank $r$                                            | 26     | 17     | 13     | 10     | 4      |\n| Accuracy Improvement per Cycle (vs. baseline with fixed $r=25$) | +0.09%      | +0.12% | +0.22% | +0.37% | +0.39% |\n| Cumulative Explained Variance (First rank $r$ components)      | 98.14% | 98.18% | 98.09% | 98.07% | 98.15% |\n## 3. How sensitive is LCoTTA to entropy-sample filter (ESF) threshold choice, and what rules of thumb can practitioners use to set it?\nAs the  maximum possible entropy of a prediction on ImageNet1K is $\\log(10^3)$, we have tested a wide range of the threshold $\\tau \\cdot\\log(10^3)$ for $\\tau\\in${0.2,0.25,$\\cdots$,0.65}. As shown below, LCoTTA is generally not sensitive to the threshold. We used $\\tau=0.4$ in all the experiments.\n\n#### Table: Performance for different entropy filter threshold on ImageNetC with ResNet50 (accuracy %)\n| $\\tau × \\log(10³)$ | 0.20  | 0.25  | 0.35  | 0.40  | 0.45  | 0.50  | 0.55  | 0.60  | 0.65  |\n| ------------------ | ----- | ----- | ----- | ----- | ----- | ----- | ----- | ----- | ----- |\n| Cycle 1            | 39.44 | 39.79 | 40.54 | 40.71 | 40.29 | 38.89 | 36.79 | 35.29 | 34.12 |\n| Cycle 5            | 41.75 | 42.10 | 42.85 | 43.01 | 42.60 | 41.20 | 39.10 | 37.60 | 36.43 |\n| Cycle 10           | 41.77 | 42.05 | 42.75 | 42.99 | 42.55 | 41.10 | 39.30 | 37.90 | 36.83 |\n| Cycle 15           | 41.74 | 42.00 | 42.70 | 42.92 | 42.48 | 41.02 | 39.22 | 37.85 | 36.82 |\n| Cycle 20           | 41.71 | 41.98 | 42.66 | 42.92 | 42.40 | 40.95 | 39.15 | 37.78 | 36.54 |\n\n## 4. The isotropic noise assumption in the SDE analysis\n\nThank you for the thoughtful question. In our analysis, we assume isotropic noise for mathematical simplicity. However, **the isotropic assumption is not strictly necessary for the effectiveness of LCoTTA. The key requirement is that the noise is not strongly correlated with the gradients of entropy-truthful (ET) samples. As long as the noise does not align with the principal subspace spanned by the correlated ET gradients, the subspace projection can effectively suppress it.**\n\nSpecifically, from the stability condition:\n$$     (P_r\\bar  g_{\\mathrm{ET}})^{T}(\\theta-\\theta^\\bullet)>\n     \\bigl|(P_r\\bar g_{\\mathrm{ED}})^{T}(\\theta-\\theta^\\bullet)\\bigr|\n     +\\frac{\\eta}{2} \\mathrm{tr}\\bigl(P_r\\Sigma P_r^{T}\\bigr), $$ it follows that when the noise is not aligned with the ET subspace, i.e., $\\mathrm{tr}\\bigl(P_r\\Sigma_{\\mathrm{ED}} P_r^{T}\\bigr)\\ll\\mathrm{tr}\\bigl(\\Sigma_{\\mathrm{ED}} \\bigr)$ when $r\\ll n$, the noise can be largely suppressed. In practice, **our empirical analysis (see Section 4.1 and Figure 2) shows that ED gradients (the noise component) are scattered and have weak correlation both among themselves and with ET gradients, thus supporting the validity of this assumption.**\n\nTherefore, LCoTTA remains robust even when the noise is not uniform, as long as the noise does not exhibit significant correlation with the ET gradients. We have not observed violations of this in our experiments.\n\n## 5. Summary\n\nOur work aims to address model degeneration in entropy-minimization based long-term adaptation, which is both practically significant and technically challenging. We hope to further clarify some points on the contributions of our work:\n1.  Identifying that degeneration in long-term continual entropy-minimization is primarily caused by ED samples; \n2. Revealing a novel low-dimensional structure formed by correlated ET gradients, which can suppress the impact of ED samples;\n3. Proposing LCoTTA, a method that does not suffer from degeneration in long-term continual adaptation; \n4. Providing a theoretical foundation for the robustness of our method. \n\n**Our work provides novel findings and new method to address the degeneration at its root, and demonstrates that sustained lifelong adaptation is achievable under strict TTA settings. To the best of our knowledge, LCoTTA is the first to achieve consistently robust long-term adaptation without requiring any source domain information or model resetting.**\n\nOnce again, we appreciate the reviewer for the valuable feedback. We hope that our additional clarifications and evaluation results have addressed your concerns."}, {"review_id": "qNOrTaBXPm", "reviewer": "Reviewer_5ZDA", "summary": "The paper studies the impact of correctly classified and incorrectly classified samples on the stability of entropy minimization. It argues that (i) incorrectly classified samples negatively impact the stability of entropy minimization during long-term adaptation and that (ii) the gradients of wrongly predicted samples are not correlated, compared to correctly predicted ones. Based on this observation, the paper proposes to “denoise” the gradient by using a projection onto the $K$ principal components instead. The paper shows that entropy minimization can be stabilized with this approach, despite adapting on millions of samples.", "questions": "- I understand there is no resetting mechanism that resets the model to the source model. Is that correct? If not, when is it triggered?\n- How does the method scale with the number of adapted parameters? Would it be feasible to use the method for adapting more than just BN parameters, i.e., computing PCs on a larger set of model parameters?", "limitations": "Yes.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths:**\n\n- The experimental results of the paper are promising: the adaptation accuracy remains high despite long-term adaptation on 37 million samples.\n- The analysis on the correlation of gradients is interesting and provides new insights into why and how entropy minimization can fail.\n- The paper provides theoretical justification for the stability of gradient descent with subspace gradients.\n\n**Weaknesses:**\n\n- The analysis in Section 3.3 is interesting and the results are intuitive. However, the observation that a large share of incorrectly predicted samples leads to degeneration in entropy minimization has also been made by [1]. This connection could be discussed more explicitly.\n\n**Minor comments:**\n\n- It would be helpful to use consistent color coding for different methods in Figure 4.\n- Line 308: Should it be c) instead of d)?\n\n\n[1] Press, Ori, et al. \"The entropy enigma: Success and failure of entropy minimization.\" ICML (2024).", "was_revised": true, "final_justification": "I raised my score from 4 to 5. The authors have addressed (i) the link to related work (Press et al. 2024) and (ii) a question on applicability of the method to more parameters by providing additional experiments. Overall, I think the paper provides a working approach with promising experimental results for the difficult problem of long-term adaptation", "rebuttal": "Dear Reviewer 5ZDA,\n\nThank you for your feedback, here we provide explanations and additional experiments to address your concerns.\n\n## 1. ... the observation that a large share of incorrectly predicted samples leads to degeneration in entropy minimization has also been made by [1]. This connection could be discussed more explicitly.\nThank you for bringing up the connection to [1]. We have carefully reviewed this work, which analyzes why entropy minimization (EM) initially improves adaptation but eventually leads to degeneration after many steps. Press et al. [1] show that EM first improves accuracy by embedding test data close to the class means of training data, but over many iterations, it pushes test embeddings far from the training data, resulting in degraded accuracy. Another striking observation is that, even after removing initially correctly classified samples, EM can still initially improve performance before eventually collapsing.\n\nThe observation of EM failure in continual adaptation aligns with our findings. **Our work complements [1] by identifying entropy-deceptive samples as a key cause of degeneration in long-term adaptation. Further, we  present a novel observation revealing the Low-Dimensional Structure of Gradients (LDSG) in TTA, and show that leveraging LDSG can effectively suppress the impact of entropy-deceptive (ED) samples and enables robust long-term adaptation, thereby addressing degeneration at its root.**\nWe will explicitly discuss the connection to [1] in the revised version.\n\n*[1] Press, Ori, et al. “The entropy enigma: Success and failure of entropy minimization.” ICML (2024).*\n\n## 2. I understand there is no resetting mechanism that resets the model to the source model. Is that correct?\nYes, our method does not employ any resetting mechanism. Its robustness stems from the subspace-based approach, which effectively suppresses the impact of entropy-deceptive (ED) samples. While model resetting can mitigate degeneration, it often leads to a *degeneration-resetting cycle* without addressing the root cause.\n\nTo fundamentally address degeneration at its core, **our work identifies ED samples as the main cause and presents a novel observation of the Low-Dimensional Structure of Gradients (LDSG) which can effectively suppress the impact of ED samples. Building on this, we propose a method that tackles degeneration at its root and enables robust long-term adaptation**. \n\n## 3. Would it be feasible ... adapting more than just BN parameters, i.e., computing PCs on a larger set of model parameters?\nThanks for the comment. Our method can scale to a larger set of model parameters at some memory cost for storing the projection matrix. As normalization layers are typically sufficient to capture representation shifts, it is a common practice in many TTA  works to update only the normalization layers.\n\nWe have conducted two additional evaluations to update the full model parameters, as shown below. \nWhen adapt the full parameters, we extract the projection matrix from $G_{t} \\in \\mathbb{R}^{{n} \\times k}$ using a fast computation approach. Specifically, since $k \\ll n$, we compute the projection matrix $U_{t,r}$ of $G_t$ by first performing spectral decomposition on the much smaller $k \\times k$ matrix $G_t^\\top G_t$. Then, the principal directions in the parameter space are recovered as $u_i = G_t v_i/\\sigma_i$, where $v_i$ and $\\sigma_i$ are the eigenvectors and singular values of $G_t^\\top G_t$, respectively. \n\n**(1)** We relax our method to update *all* parameters and extract the subspace for the full gradient space.  \n#### Table A. Results (acc %) of our method updating BN and full parameters (ImageNetC, ResNet50).\n| Method / Cycle               | 1     | 10    | 20    | 30    | 40    | 50    |\n| ---------------------------- | ----- | ----- | ----- | ----- | ----- | ----- |\n| Ours (BN Update)              | 40.71  | 42.99 | 42.90 | 42.92 | 42.72 | 42.60 |\n| Ours (Full-Parameter Update) | 40.75 | 42.95 | 42.82 | 42.89 | 42.77 | 42.54 |\n\n**(2)** **CoTTA+Subspace:** We integrated our subspace projection into CoTTA’s teacher-student framework while keeping its full-parameter update strategy.\n\n#### Table B. Results (acc %) of CoTTA with and without subspace on ImageNetC with ResNet50 (full-parameter update)\n| Method / Cycle   | 1     | 10    | 20    | 30    | 40    | 50    |\n| ---------------- | ----- | ----- | ----- | ----- | ----- | ----- |\n| CoTTA (Baseline) | 37.30 | 26.99 | 21.98 | 18.28 | 15.00 | 12.65 |\n| CoTTA + Subspace | 37.50 | 36.76 | 35.52 | 35.48 | 35.73 | 35.15 |\n\nThe results demonstrate that even when scaling to the full parameter space, our subspace projection-based approach remains effective for long‑term adaptation to significantly enhance the robustness.\n\n## 4. Other minor issues\nWe appreciate the reviewer’s careful reading and pointing out the issues and typos. We will correct them in the revised version.\n\n## 5. Summary\n\nWe again sincerely thank the reviewer for the valuable feedback. \nUnderstanding the mechanism behind model degeneration in long-term TTA is of practical importance. Our work offers a fresh perspective on this issue by identifying deceptive samples as a key cause. We further present a novel observation of the Low-dimensional structure of gradients in TTA, and demonstrate its effectiveness in suppressing the influence of deceptive samples and thereby enabling robust long-term adaptation. Building on this insight, we propose a method that addresses degeneration at its root, and provide a theoretical foundation for the robustness of our method.  \n\n**Our work demonstrates that sustained lifelong adaptation is achievable under strict TTA settings. To the best of our knowledge, LCoTTA is the first to achieve consistently robust long-term adaptation without requiring any source domain information or model resetting.**"}, {"review_id": "Xpl8DWHx1b", "reviewer": "Reviewer_fBg4", "summary": "Existing TTA methods primarily rely on unsupervised entropy minimization, which can lead to trivial low-entropy yet inaccurate predictions—a failure mode attributed to entropy-deceptive (ED) samples. To address this issue, the authors observe that ED samples have gradient directions misaligned with a low-dimensional subspace formed by the highly correlated gradients of entropy-truthful (ET) samples. Leveraging this insight, they propose LCoTTA, a method that constrains model updates within the ET-driven subspace to suppress the influence of ED samples. Both theoretical analysis and comprehensive experiments validate the robustness and effectiveness of the proposed approach.", "questions": "ED samples also appear in standard TTA scenarios, so why don’t existing TTA methods suffer from significant performance degradation in those settings?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n- The proposed method offers a novel and meaningful perspective on addressing performance degradation in test-time adaptation by analyzing the role of misleading samples and utilizing the low-dimensional structure of gradients.\n- The experiments are comprehensive and convincing, demonstrating the method's effectiveness across various benchmarks, model architectures, and adaptation scenarios.\n- The inclusion of theoretical analysis strengthens the work, providing additional justification for the method's design and helping to enhance its overall credibility.\n\n**Weaknesses**\n\n- The baselines used in the experiments are somewhat outdated. It would be better to include one or two more recent baselines for comparison.\n- The authors claim that \"The proposed method is complementary to existing approaches for robust TTA.\" It would be better if this claim were supported by some experiments.\n- Lines 22 and 92 contain excessive, unstructured citations that hinder clarity. The Introduction overreferences a well-known definition, while the Related Work section lists too many papers without context. Using fewer, representative works would improve focus and readability.", "was_revised": true, "final_justification": "The authors address my concern by providing additional experiments and I have no other questions now. So I keep my original score.", "rebuttal": "Dear Reviewer fBg4,\n\nThank you very much for your valuable feedback, here we provide explanations and additional experiments to address your concerns.\n\n## 1. More Recent Baselines\n\nThank you for the suggestion. In the original manuscript, we have already included several recent strong baselines for the single-epoch setting (Table 2), including BeCoTTA (ICML 2024), AEA (ICLR 2025), and TCA (CVPR 2025). \n\nFollowing your suggestion, we have evaluated AEA (ICLR 2025) and BeCoTTA (ICML 2024) for the 50-cycle setting. It is worth noting that AEA uses random parameters resets, whereas our method does not use any reset strategy.\nThe results of AEA and BeCoTTA sampled at {1,10,...,50} cycles are provided below, which show that they also suffer from degeneration under challenging long-term adaptation setting.\n\n#### Table 1: Results of AEA and BeCoTTA on ImageNet-C with ResNet50 (Accuracy %)\n| Method / Cycle    | 1     | 10    | 20    | 30    | 40    | 50    |\n| ----------------- | ----- | ----- | ----- | ----- | ----- | ----- |\n| AEA (ICLR’25)     | 39.52 | 29.12  | 15.03 | 0.34  | 0.32  | 0.78  |\n| BeCoTTA (ICML’24) |  39.12 | 37.50 | 34.27 | 28.43 | 23.28 | 20.56 |\n| Ours |  40.71  | 42.99 | 42.90 | 42.92 | 42.72 | 42.60 |\n\nWe will add these results into the revised version.\n\n## 2. Demonstrating Complementarity with Other TTA Methods\n\nWe appreciate the reviewer’s request for empirical evidence supporting our claim of complementarity. Although our central objective was to propose a new viewpoint for continual TTA—rather than a plug‑and‑play block—we nonetheless integrated our subspace projection into a representative, yet mechanistically different, method: the teacher–student pipeline of **CoTTA**.\n\nBy injecting the subspace constraint into CoTTA’s update rule, we observed that (i) the reverse gradients still concentrate in a low-dimensional subspace, and (ii) projecting onto this subspace effectively arrests the long-term collapse seen in the original method. \n#### Table: Performance (accuracy %) of CoTTA with/without subspace projection on ImageNetC ($r=25$)\n\n| Method / Cycle   | 1     | 10    | 20    | 30    | 40    | 50    |\n| ---------------- | ----- | ----- | ----- | ----- | ----- | ----- |\n| CoTTA (Baseline) | 37.30 | 26.99 | 21.98 | 18.28 | 15.00 | 12.65 |\n| CoTTA + Subspace | 37.50 | 36.76 | 35.52 | 35.48 | 35.73 | 35.15 |\n\nThese results substantiate that our approach can be layered on top of existing robust TTA strategies, yielding sustained accuracy across cycles while preserving their core mechanisms.\n\n## 3. ED samples also appear in standard TTA scenarios, so why don’t existing TTA methods suffer from significant performance degradation in those settings?\n\nThank you for your question. In fact, our experiments (e.g., Figure 4) demonstrate that existing TTA methods do experience significant performance degradation under long-term continual adaptation. Most existing works evaluate TTA methods in relatively short-term scenarios, typically a single cycle, with or without model resetting. In contrast, our study specifically investigates a long-term adaptation setting without any resetting.\n\nFor example, on ImageNet-C, a single evaluation cycle includes 15 types of corruption, each with 50,000 samples, resulting in 750,000 samples per cycle. In our experiments, we evaluate the methods over 50 cycles, corresponding to a total of $3.75 \\times 10^7$ samples, which is orders of magnitude longer than standard TTA protocols. This long-term setting reveals the performance degradation that might not be observed in shorter evaluations.\n\n## 4. Other Minor Issues  \nThank you for pointing these issues. We will streamline the citations in both the Introduction and Related Work, keeping only the most relevant references and organizing them thematically. \n\n## 5. Summary\n\nWe thank the reviewer again for the valuable feedback. Understanding model degeneration in long-term TTA is practically important. Our work highlights deceptive samples as a key cause and reveals a novel low-dimensional structure in TTA gradients that can suppress the detrimental impact of deceptive samples. Leveraging this insight, we propose a method with theoretical guarantees that effectively overcomes degeneration and enables robust long-term adaptation.\n\n**Our results show that reliable lifelong adaptation is possible even under strict TTA settings. To our knowledge, LCoTTA is the first to deliver consistently robust long-term adaptation without requiring any source domain information or model resetting.**"}]

Author rebuttal:

Meta-review: This paper makes a nice, empirical contribution to lifelong TTA. The identification of entropy-deceptive samples, discovery of low-dimensional ET gradient structure, and the subspace-projection method together provide both theoretical and practical advances. While improvements are less pronounced in single-epoch adaptation (though still better than recent SOTA), the focus on long-term continual settings is both novel and important.

All four reviewers voted favorably for the paper, found the analysis novel and insightful and regarded the experiments as comprehensive. There were some concerns raised (hyperparameter sensitivity, missing baselines, limited scope in some tasks) and they were convincingly addressed in rebuttal.

Overall, this work is technically solid and likely to be impactful in the area of continual and online adaptation.
Decision: Accept (poster)