Title: Lifelong Test-Time Adaptation via Online Learning in Tracked Low-Dimensional Subspace
Venue: NeurIPS 2025
Accepted: yes
Keywords: Test-time Adaptation

Continual adaptation

Low-dimensional subspace
Abstract: Test-time adaptation (TTA) aims to adapt a source model to a target domain using only test data. Existing methods predominantly rely on unsupervised entropy minimization or its variants, which suffer from degeneration, leading to trivial solutions with low-entropy but inaccurate predictions. In this work, we identify *entropy-deceptive* (ED) samples, instances where the model makes highly confident yet incorrect predictions, as the underlying cause of degeneration. Further, we reveal that the gradients of entropy minimization in TTA have an intrinsic low-dimensional structure, driven primarily by *entropy-truthful* (ET) samples whose gradients are highly correlated. In contrast, ED samples have scattered, less correlated gradients. Leveraging this observation, we show that the detrimental impact of ED samples can be suppressed by constraining model updates within the principal subspace of backward gradients. Building on this insight, we propose LCoTTA, a lifelong continual TTA method that tracks the principal subspace of gradients online and utilizes their projections onto this subspace for adaptation. Further, we provide theoretical analysis to show that the proposed subspace-based method can enhance the robustness against detrimental ED samples. Extensive experiments demonstrate that LCoTTA effectively overcomes degeneration and significantly outperforms existing methods in long-term continual adaptation scenarios. Code is available online.

Lifelong Test-Time Adaptation via Online Learning in Tracked Low-Dimensional Subspace

Abstract

Test-time adaptation (TTA) aims to adapt a source model to a target domain using only test data. Existing methods predominantly rely on unsupervised entropy minimization or its variants, which suffer from degeneration, leading to trivial solutions with low-entropy but inaccurate predictions. In this work, we identify entropy-deceptive (ED) samples, instances where the model makes highly confident yet incorrect predictions, as the underlying cause of degeneration. Further, we reveal that the gradients of entropy minimization in TTA have an intrinsic lowdimensional structure, driven primarily by entropy-truthful (ET) samples whose gradients are highly correlated. In contrast, ED samples have scattered, less correlated gradients. Leveraging this observation, we show that the detrimental impact of ED samples can be suppressed by constraining model updates within the principal subspace of backward gradients. Building on this insight, we propose LCoTTA, a lifelong continual TTA method that tracks the principal subspace of gradients online and utilizes their projections onto this subspace for adaptation. Further, we provide theoretical analysis to show that the proposed subspace-based method can enhance the robustness against detrimental ED samples. Extensive experiments demonstrate that LCoTTA effectively overcomes degeneration and significantly outperforms existing methods in long-term continual adaptation scenarios. Code is available at https://github.com/ThunderDavid/LCoTTA.

1 Introduction

Test-time adaptation (TTA) aims to address domain shifts by adapting the model to the target domain during inference, without requiring access to source data [1, 2, 3, 4, 5, 6, 7, 8]. This need for on-the-fly adaptation arises broadly across other tasks under distribution shifts[9, 10, 11, 12, 13], motivating TTA as a general deployment-time solution. Unlike traditional domain adaptation methods, TTA operates in a fully unsupervised manner. It leverages only the unlabeled test data received during inference, enabling the model to adaptively improve its performance when faced with domain shift on-the-fly. As a fully unsupervised online learning paradigm, TTA methods predominantly rely on unsupervised learning objectives, such as entropy minimization or its variants [2, 14]. However, such methods are prone to instability. For instance, entropy minimization suffers from a trivial solution that assigns all predicted probability to the most probable (but may be incorrect) class. Consequently, these methods are susceptible to performance degeneration during continual adaptation.

To address this, various methods have been proposed. For example, [2] leverages entropy minimization at the batch level while restricting updates to only the normalization parameters. Moreover, some approaches employs teacher-student networks, where a teacher model provides stable guidance through a moving average of parameters to prevent collapse [8, 6, 4]. Furthermore, model

resetting strategies, which periodically reset partial or full model parameters to their initial states, have demonstrated effectiveness in mitigating error accumulation during adaptation [8, 7, 15, 16]. Generally, these strategies have shown effective under the current mainstream evaluation protocol, which typically involves single-epoch adaptation on a test set. However, in more realistic lifelong scenarios, where the model is required to adapt continually over the long term, these methods still face challenges in maintaining consistently robust performance.

Although numerous heuristic approaches have been proposed to mitigate the degeneration, an in-depth analysis of its underlying cause remains lacking. In this work, we delve into the underlying cause of the degeneration problem and provide novel insights to address it at its core. Specifically, we show that a main underlying cause of the degeneration in entropy-minimization based methods is the presence of entropy-deceptive (ED) samples, where the model produces highly confident yet incorrect predictions. These samples frequently arise when facing out-of-distribution data in the TTA setting, which mislead model updates and exacerbate error accumulation over time, especially in long-term continual adaptation scenarios.

Furthermore, we show that the backward gradients of the entropy loss in TTA exhibit an intrinsic low-dimensional structure. It is primarily driven by entropy-truthful (ET) samples, whose gradients are highly correlated as they tend to share similar update directions. In contrast, gradients from ED samples are scattered and less correlated. Based on this observation, we present a nontrivial finding that, the detrimental impact of ED samples can be suppressed by constraining weight updates in a low-dimensional principal subspace of the gradients. Building on this insight, we propose LCoTTA, a lifelong continual TTA method that tracks the principal subspace of gradients online and utilizes their projections for adaptation. This approach can effectively suppress the impact of ED samples and ensures robust performance in challenging long-term continual adaptation scenarios. In summary, the main contributions are as follows:

An observation that the degeneration of the entropy minimization based TTA method in continual adaptation is primarily caused by ED samples, instances where the model makes highly confident but incorrect predictions.

A novel finding that, without using any supervision information, the detrimental impact of ED samples can be suppressed by exploiting the intrinsic low-dimension structure of the gradients, which is primarily formed by the correlated gradients of ET samples.

Alifelong continual TTA method LCoTTA, which tracks the principal subspace of gradients online and utilizes their projections into this subspace for adaptation. LCoTTA does not suffer from degeneration and enables robust long-term continual adaptation.

A theoretical analysis that shows the proposed subspace projection based method can enhance the robustness against detrimental ED samples.

Extensive experiments demonstrate that LCoTTA maintains robust performance in long-term continual adaptation scenarios and significantly outperforms existing continual TTA methods.

While this work focuses on TTA, the method of exploiting ET and ED gradient structure to effectively distinguish them in an unsupervised manner holds promise for broader applications in unsupervised learning tasks. The proposed method is complementary to existing approaches for robust TTA, such as teacher-student networks, model resetting, and weight regularization. Integrating it with these methods can be expected to further improve performance.

2 Related Work

Test-time Adaptation. [1] first proposed updating the activation statistics of BN to enhance model robustness. Further, MemBN [17] introduced a memory-based BN approach, which aggregates and adaptively weights stored statistics to achieve robust TTA. Tent [2], one of the pioneering works, updates only the affine transformation parameters of BatchNorm layers by entropy-minimization. SHOT [3] combines entropy minimization with diversity regularization to achieve robust TTA. DePT [18] leverages visual prompts to efficiently adapt to target domains and bootstrap source representations. Furthermore, single-sample methods have been explored in [19, 20], while nonparametric approaches are proposed in [21, 22]. Moreover, neuro-inspired method and energy-based model have been considered in [23] and [24].

Continual Test-time Adaptation. Continual TTA aims to adapt a model to dynamic and evolving target domains. CoTTA [8] introduces a teacher-student framework with consistency loss, while EcoTTA [6] employs meta-networks and self-distilled regularization to improve memory efficiency. TTACOPE [25] leverages supervised pretraining on labeled source datasets to improve initialization, while BECoTTA [26] utilizes a mixture-of-domain low-rank experts for domain-adaptive routing. Moreover, there exists a number of recent works designed for robust continual TTA [27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 16, 37, 38, 39, 40]. Despite the effectiveness of model-resetting, teacher-student learning, and weight regularization approaches, they still face challenges in maintaining consistently robust performance in long-term continual adaptation scenarios. More importantly, there lacks an in-depth analysis on the underlying cause of degeneration in unsupervised TTA, which is crucial for understanding and mitigating it.

Low-dimensional learning. The low-dimensional structure of neural-network learning has been extensively studied [41, 42, 43, 44, 45, 46, 47]. These studies reveal that loss landscapes of neuralnetworks reside within an intrinsic dimension, which enables model weights to be optimized in a low-dimensional subspace. Unlike these works focusing on the low-dimensional structure of weights sampled along the optimization trajectory, we reveal the low-dimensional structure of batch-based stochastic gradients during the adaptation process, which is driven by ET samples in TTA and can help suppress the gradients of ED samples.

3 Analysis on the Degeneration of Entropy-Minimization Based TTA

3.1 Prelimineries

̸

Let f ( x ; θ ) be a model with weights θ pre-trained on a source domain D s = { ( x i , y i ) } N i =1 , which follows a distribution P s ( x ) . Given the source model f ( x ; θ ) , the goal of TTA is to adapt it to test data D t = { x i } i M =1 , where the data distribution P t ( x ) deviates from the training distribution, i.e. P t ( x ) = P s ( x ) . Without access to source data and without using any supervision information on test data, TTA methods typically utilize unsupervised losses, such as the entropy loss [2]

where ˆ p ( x i ) is the model predicted probability on sample x i . It encourages the model to make confident predictions by minimizing the entropy of the model predictions.

Traditional TTA methods typically focus on adaptation on test data over a limited number of steps, but real-world scenarios often involve continuously evolving domain shifts, requiring models to adapt continually over the long term. Lifelong TTA addresses this by adapting a model to sequentially arriving test data from evolving target domains { P t : t = 1 , 2 , · · · } . At each step t , the model predicts output f ( x i ; θ t ) for input x i and updates its parameters for future steps, i.e., θ t → θ t +1 . This setting is challenging as models relying on unsupervised losses are prone to degradation over time.

3.2 Entropy Reliability of Test Samples

As discussed above, existing TTA methods, operating without access to source data and supervision on test data, predominantly rely on unsupervised entropy minimization or its variants [2, 8, 4]. While simple and effective, entropy minimization as an unsupervised loss can be unreliable and mislead model updates. Specifically, entropy minimization has a trivial solution that assigns all predicted probability to the most probable class. This is particularly problematic for samples with high-confidence but incorrect predictions, which frequently occur when the target domain significantly deviates from the source domain. For instance, given a ground-truth label [0,0,0,0,1], a model prediction of [0.1,0.2,0.4,0.1,0.2] optimized via entropy minimization is likely to converge to [0,0,1,0,0], resulting in incorrect predictions.

To address this degeneration problem, Tent [2] proposes to jointly optimize batched predictions while restricting updates to normalization parameters. However, this approach struggles to maintain satisfactory performance in continual adaptation. Although methods like model resetting and teacherstudent frameworks [2, 8, 48, 49] partially mitigate degeneration, as shown in our experiments (Figure 4), they still suffer from performance degradation in long-term adaptation scenarios. Press

et al. [50] further show that EM first improves accuracy by embedding test data close to the class means of training data, but over many iterations, it pushes test embeddings far from the training data, resulting in degraded accuracy. Here, we offer a new perspective on EM by focusing on the entropy reliability of predicted test samples.

To better characterize the reliability of predictions, we introduce the Entropy Reliability Score (ERS). Let ˆ p = [ˆ p 1 , ˆ p 2 , . . . , ˆ p C ] denote the model's predicted probability distribution over C classes, where ∑ C i =1 ˆ p i = 1 and ˆ p i ∈ [0 , 1] . For a sample with ground-truth class c ∈ { 1 , 2 , . . . , C } , its ERS is defined as

̸

̸

where ˆ p c is the predicted probability for the true class c , and max i = c ˆ p i is the highest predicted probability among all incorrect classes. The ERS measures the confidence gap between the true class and the most competitive incorrect class, providing a quantitative indicator of prediction reliability. Based on ERS, we further define ET and ED samples.

Definition 1: Given a model prediction ˆ p and with the definition of the reliability score in (2), a sample is classified as entropy-truthful if S (ˆ p ) > 0 ; otherwise, it is classified as entropy-deceptive .

3.3 The Degeneration of Entropy Minimization

Here we further dive into the underlying cause of the degeneration problem in entropy-minimization based methods. We present an observation that this issue is primarily driven by ED samples. From Definition 1, ET samples indicate reliable predictions where the model assigns the highest confidence to the true class. In contrast, ED samples reflect unreliable predictions, where the model assigns higher confidence to incorrect classes, potentially misleading optimization processes such as entropyminimization. This issue is particularly pronounced for highly confident ED samples, which exhibit low entropy despite being incorrect.

To analyze the impact of different types of samples on the adaptation process, we conduct experiments under a typical continual TTA scenario on the CIFAR100C and ImageNetC datasets with Gaussian corruption at severity-5. The samples are sorted by ERS, and pretrained ResNeXt-29 and ResNet50 from [51] are adapted using entropy-minimization with different subsets of these samples. Among the samples ranked by ERS, the top 58.17% on CIFAR100C and 11.59% on Ima-

60%

30%

50%

25%

40%

2

20%

Accura

30%

Top40%ERS

Top70%ERS

15%

Top30%ERS

Top70%ERS

20%

Top75%ERS

Top50%ERS

Top80%ERS

Top65%ERS

Top80%ERS

10%

Top90%ERS

10%

5%

0%

0

5

10

1520253035

40

45

50

0

5

10

152025303

35

40

45

50

Epochs

Epochs

(a) CIFAR100C (ResNeXt-29)

(b) ImageNetC (ResNet-50)

geNetC are identified as ET samples with S (ˆ p ) > 0 . Figure 1 shows the performance of continual adaptation over 50 epochs.

The results demonstrate the detrimental impact of ED samples on continual adaptation. For example, on CIFAR100C, when adaptation is performed exclusively on ET samples (e.g., the top 40% of samples), the model does not exhibit degeneration during continual adaptation. In contrast, expanding the sample set to include ED samples (e.g., the top 75% or 80% of samples) leads to significant degeneration. A similar trend is observed on ImageNetC.

4 Method

The above analysis suggests that the degeneration of entropy-minimization can be avoided by using only ET samples for adaptation. However, identifying ET samples without supervision is challenging. In this section, we reveal that the impact of ED samples can be suppressed by leveraging the lowdimensional structure of ET gradients. Then, we propose a robust lifelong continual TTA method.

4.1 Correlated ET Gradients Forming A Low-Dimensional Structure

As discussed above, we identified ED samples as the underlying cause of model degeneration. Here, we show that gradients from ET samples are highly correlated, which form a low-dimensional subspace. In contrast, gradients from ED samples are scattered and less-correlated.

Figure 2 (a) and (c) shows the correlation between the gradients of different batches on CIFAR100C and ImageNetC (Gaussian noise with severity-5) for ResNeXt-29 and ResNet50, respectively. The samples are sorted by their ERS values from the most entropy-truthful to the most entropy-deceptive. It can be seen that gradients of ET samples are highly correlated, while gradients of ED samples are less correlated.

This phenomenon arises as gradients of ET samples tend to share similar update directions in parameter space, thus are correlated. In contrast, the gradients of ED samples are lesscorrelated, as their update directions are more scattered and do not have consistent directions. Based on this analysis, we can expect that gradients of entropy-

(b) PCA (CIFAR100C)

(d) PCA (ImageNetC)

100%

30%

Cumulative EV (%)

0.4

Contribution

80%

Index

Batch

0.3

20%

The first 10

principal

with

components

60%

ERS=0

0.2

10%

account for.

2

95%

- 0.1

40%

0%

1510

152025

0.0

Principal Component Index

30%

100%

0.30

0.25

Contribution（

20%

80%

with ERS=0

The first 10

-0.20

principal

60%

components

- 0.15

10%

accountfor

-0.10

2

90%

40%

0.05

0%

0.00

1510

1520

25

Principal Component Index

Cumulative EV (%)

minimization in TTA have a low-dimensional structure, which is primarily formed by the principal update directions shared by ET sample gradients.

Let g t = ∇ θ L e ( B t ; θ ) denote the gradient of the entropy loss L e defined in (1) with respect to the model parameters θ on a batch of samples B t . To investigate the low-dimensional structure of stochastic gradients, we collect T backward gradients G = [ g 1 , g 2 , · · · , g T ] ∈ R n × T over T sequential batches sampled during the adaptation process, where n is the number of model parameters. Principal component analysis (PCA) of G is then performed using SVD.

Figure 2 shows typical PCA results in the TTA experiments on CIFAR100C (ResNeXt-29) and ImageNetC (ResNet-50). Gradients are sampled every 50 iterations with T = 50 . The bar plot shows the explained variance of individual principal components, while the red curve represents the cumulative explained variance. Notably, the first few principal components dominate, with the top 10 components capturing more than 90% of the total variance. This result demonstrates that parameter gradients during TTA reside in a low-dimensional subspace, formed by ET sample gradients. In contrast, ED gradients do not have such a structure, as shown in Figure 9 in Appendix J.

4.2 Suppressing Entropy-Deceptive Samples via Low-Dimensional Gradient Structure

We reveal that ET gradients are predominantly concentrated within a low-dimensional principal subspace, whereas ED gradients exhibit a weaker alignment with this subspace. To show this, we first construct a r -dimensional principal subspace of the gradients. Specifically, given a collected

gradient matrix G ∈ R n × T , we extract a r -dimension subspace of it via the following formulation

where U r ∈ R n × r contains the orthonormal bases corresponding to the largest r eigenvalues of GG ⊤ , which spans the r -dimension principal subspace of G , with r ≪ n . Formulation (3) is a standard PCA problem that can be solved by eigen-decomposition of GG ⊤ .

Given an extracted subspace spanned by U r , we analyze the impact of gradient projection into this subspace. Specifically, for a gradient g t , its lowdimensional representation is obtained via projection as U ⊤ r g t , and then backprojected to the original parameter space as

We then examine the effect of this principal subspace representation on ET and

(a) CIFAR100C (ResNeXt-29)

(b) ImageNetC (ResNet-50)

0.8

一

Sorted ERS

1.00

Rank=25

Rank=100

1.00

0.7

0.75

0.5

Rank=75

Rank=50

Rank=125

0.75

0.6

0.50

Score

0.50

Score

1161/126111

0.5

0.25

Reliability

Ilgel/llgll

0.4

0.25

Reliability

0.4

0.00

0.3

0.00

0.3

0.25

Entropy

-0.25

Entropy

0.2

Rank=3

Rank=5

-0.50

0.2

-0.50

Rank=10

0.1

Rank=20

-0.75

0.1

-0.75

0.0

Rank=50

-1.00

Sorted ERS

-1.00

08162432404856647280

0

2

4

6

8

8101214161820

Most truthful

Most deceptive

Most truthful

Most deceptive

samples

→samples

samples

→samples

ED samples. Using the same experimental setting as in Figure 1 with the CIFAR100C and ImageNetC, we sort the test samples by their ERS values. With the sorted samples, we compute the ℓ 2 -norm ratio ∥ ˜ g t ∥ / ∥ g t ∥ , from the most truthful to the most deceptive samples. Figure 3 presents the results for different subspace dimensions, where the ratio ∥ ˜ g t ∥ / ∥ g t ∥ is computed on batched samples.

Interestingly, as shown in Figure 3, the ratio ∥ ˜ g t ∥ / ∥ g t ∥ generally decreases from the most truthful to the most deceptive samples. This implies that the principal subspace representation (4) can effectively differentiate between samples to some extent. Notably, the gradients of ED samples exhibit a more pronounced decay in this representation, due to the weak correlation with correlated ET gradients.

4.3 Adaptation in Tracked Low-Dimensional Subspace

The above findings offers a way to mitigate the impact of ED samples by leveraging the lowdimensional structure of ET gradients. We utilize the low-dimensional representation ˜ g t of the gradient g t for parameter update. Furthermore, considering that the test data distribution may vary continuously in continual adaptation scenarios, we dynamically track a low-dimensional principal subspace of the gradients in an online manner. As evidenced by experiments (Table 4 in Appendix), using a fixed subspace would degrade performance under varying test data distributions.

Specifically, during adaptation, we compute the gradient of the entropy loss with respect to θ for the t -th batch as g t = ∇ θ L e ( B t ; θ ) . To capture the evolving gradient structure, we maintain a queue of the most recent k gradients G t = [ g t -k +1 , g t -k +2 , . . . , g t ] ∈ R n × k where n is the dimensionality of the parameter space. Subsequently, PCA (3) is applied to G t to extract a r -dimensional principal subspace represented by the projection matrix U r,t ∈ R n × r , where r < k ≪ n .

The projection matrix U r,t defines the subspace used for parameter update during adaptation, based on which the component of g t residing within this subspace is obtained by projecting g t into the subspace and then back-projecting it into the original parameter space: ˜ g t = U r,t U ⊤ r,t g t . Then, the model parameters are updated based on ˜ g t as

where η is the learning rate. This approach ensures that the parameter updates are informed by the salient components of ET gradients, captured by the dynamically tracked low-dimensional subspace.

However, applying this method to update all model parameters is memory-intensive, as it requires maintaining the gradient queue G t of size n × k and the projection matrix U r,t of size n × r . This

significantly increases memory consumption, particularly for modern neural networks with large parameter dimensionality n , which makes it impractical for some on-device adaptation applications.

To address this issue, we restrict updates to the affine parameters of the normalization layers rather than the entire model. Previous studies [1, 2, 7] have demonstrated that updating normalization layers alone is sufficient to achieve strong performance in TTA, as they play a crucial role in controlling the feature distribution and are sensitive to distributional shifts. Notably, the BN and LN parameters typically constitute less than 1% of the total model parameters. Hence, tracking the subspace of them introduces minimal computational and memory overhead.

Moreover, we also employ the entropy-based sample filtering (ESF) strategy [15, 7] before subspace tracking. Note that, ESF alone cannot distinguish between ET and ED samples, as shown in Figure 7 in Appendix G. As shown in Section 4.1, ET sample gradients form a more prominent low-dimensional structure, while ED sample gradients are scattered and less-correlated. Thus, removing high-entropy (ambiguous) samples (Figure 7) enables the subspace to better capture reliable update directions, which makes ED gradients more orthogonal to it and easier to suppress.

5 Analysis on the Enhanced Stability of Subspace Projected TTA

We present stability analysis to show that the proposed subspace projection method enhances TTA robustness against detrimental ED samples. Note that TTA is essentially a local fine-tuning of a source pretrained model. The analysis is restricted to a neighbourhood of any local equilibrium, and relies on assumptions that ET gradients concentrate in a low-dimensional principal subspace, while ED gradients have weak inter-sample correlation and approximately isotropic dispersion.

Let θ t ∈ R n be the model parameters updated by mini-batch SGD during TTA. For each step t , the mini-batch B t = { ˆ B t , ˘ B t } contains ET samples ˆ B t and ED samples ˘ B t . Denote the gradients on B t as

where ˆ g t := ∇ θ L ( θ t ; ˆ B t ) and ˘ g t := ∇ θ L ( θ t ; ˘ B t ) denote the gradient components of ET and ED samples, respectively. From the results in Section 4.1, we make assumptions: a) Mean and variance models of ET and ED gradients: E B [ˆ g ] = ¯ g ET , Cov[ˆ g ] = Σ ET , E B [˘ g ] = ¯ g ED , Cov[˘ g ] = Σ ED = σ 2 ED I n +∆ , where Σ ET and Σ ED are batch-normalized covariance of ET and ED gradients, respectively, with ∥ ∆ ∥ ≪ σ 2 ED . Weak correlation between ET and ED gradients: Cov(ˆ g -¯ g ET , ˘ g -¯ g ED ) = 0 . b) Low-rank structure of ET gradients: Σ ET has effective rank r ≪ n , let Σ ET = U Λ U T be any eigen-decomposition with U r = [ u 1 , . . . , u r ] being the eigenvectors corresponding to the largest eigenvalues, and denote P r := U r U T r . c) Small-step limit: As η → 0 the discrete dynamics of SGD θ t +1 = θ t -ηg t converge to the stochastic differential equation (SDE) [52]

Theorem 1 (Local stability of full-space and subspace-projected TTA) . Suppose that the above assumptions hold. Let θ · be an equilibrium satisfying ¯ g ( θ · ) = 0 (i.e. with zero gradient expectation). For the full-space SGD, if, for all θ in a neighbourhood of θ · ,

then the SDE (5) is mean-square stable at θ · . Further, for rankr subspace projection based learning, i.e., update in the principal subspace of ET gradients via replacing g t by P r g t , the corresponding continuous-time SDE

is mean-square stable at θ · if

The proof is given in Appendix A. Theorem 1 provides balance-of-forces conditions for the stability of both full-space SGD and subspace-projected SGD. Stable local convergence is guaranteed when the corrective drift from ET gradients exceeds the combined ED bias and diffusion noise. Conversely,

(a) ResNet-50 on ImageNetC.

60%

Tent

CoTTA

Rotta

SAR

PETAL

50%

BN

ETA

DEYO

Ours

40%

2

30%

Accurac

20%

10%

0%

1

5

10

15

20

25

30

35

40

45

50

Adaptation Cycles

80%

Tent

CoTTA

Rotta

DEYO

Ours

BN

ETA

SAR

60%

40%

20%

0%

1

5

10

15

20

25

30

35

40

45

50

Adaptation Cycles

(b) ViT-B/16 on ImageNetC

if the ED contribution outweighs corrective ET drift, the dynamics may diverge or converge to an inferior minimum, which leads to model degeneration. Comparing the conditions (6) and (8), the subspace projection method relaxes the stability requirement and thus is more robust.

Specifically, when the ET gradient mean ¯ g ET lies mostly in the subspace U r , we have P r ¯ g ET ≈ ¯ g ET. Meanwhile, projection onto this subspace removes most out-of-subspace component of the EDgradient bias such that P r ¯ g ED ≈ 0 for r ≪ n . Then, the stability condition (8) simplifies to

where ∑ r i =1 λ i = Tr ( P r Σ ET P T r ) with λ i being the i -th eigenvalue of Σ ET. Compared with the full-space case with tr(Σ) = ∑ n i =1 λ i + nσ 2 ED , subspace projection largely reduces the diffusion energy by a factor ∑ r i =1 λ i + rσ 2 ED ∑ n i =1 λ i + nσ 2 ED ≈ r n ≪ 1 . This significantly enlarges the admissible range of ED bias and noise that still satisfies the stability condition. Consequently, the subspace-projected method is substantially more robust to detrimental ED samples.

6 Experiments

We conduct experiments on the ImageNetC dataset [53], which consists of 15 corruptions each with 5 severity levels. We experiment with two representative model architectures: ResNet-50 with batch normalization [54] and the ViT-B/16 with layer normalization. Results on CIFAR100C and semantic segmentation are provided in Appendix K and L.

We evaluate our method under a challenging long-term continual TTA setting, where the model adapts continuously over 50 cycles of 15 corruption types ( severity=5 ), a total of 37.5 million test samples. The model performs unsupervised continual adaptation without any external intervention from the very first beginning, such as domain-specific information, model resetting, or warm-up. We compare our method ( r = 25 ) with several SOTA continual TTA methods, including AdaContrast [5], BN [1], TENT [2], CoTTA [8], SAR [7], RoTTA [4], ETA [15], PETAL [55] and DeYO [56]. Notably, CoTTA, and RoTTA employ teacher-student networks, while SAR adopts a model resetting strategy.

6.1 Results on ImageNet-to-ImageNetC

Figure 4 presents the results on the ImageNet-to-ImageNetC task over 50 continual adaptation cycles. Clearly, our method demonstrates robust and superior long-term adaptation performance on ImageNetC. Most of the compared methods suffer significant degradation during long-term adaptation. For instance, the accuracy of Tent drops below 10% within the first 10 adaptation cycles, whilst that of CoTTA gradually declines over continual adaptation. In contrast, our method consistently achieves high performance throughout the entire adaptation process, attributed to its ability to effectively suppress the detrimental impact of ED samples. These results demonstrate the robustness and adaptability of our approach in challenging long-term adaptation scenarios.

Results under a standard TTA setting, where a model adapts to one corruption at a time for a single epoch, are provided in Tables 2 and 3 in Appendix C, which demonstrate that our method can also achieve competitive performance in short-term adaptation scenarios.

6.2 Analysis and Ablations

Ablation of subspace projection. We evaluate the effectiveness of the subspace projection method. As shown in Table 1, the method without using any strategy quickly collapse under entropy-based continual adaptation. Entropy filtering improves initial performance but still suffers from ED samples due to the nature of entropy, leading to degeneration in long-term. Our method achieves consistent robustness over long-term adaptation.

Effect of subspace dimension. Figure 5 shows the performance of our method with different dimensions of the subspace, r ∈ { 10 , 25 , 50 , 100 } , in continual adaptation on ImagenetC over 50 cycles. Using a reasonably low dimension (e.g., r = 25 ) can achieve satisfactory performance during long-term continual adaptation. However, as the rank increases, the restricting effect of the subspace diminishes, leading to performance degradation in later cycles. On the other hand, too low a rank would limit the adaptation performance.

Effect of hyperparameters. We conduct experiments on the sampling interval and the length of the gradient queue k . As shown in Figure 6, when the queue length k for storing gradients exceeds a certain threshold (e.g., 25), it captures sufficient gradient information to compute a stable subspace, thus maintaining stable performance. Similarly, the model exhibits robust long-term adaptation performance across a wide range of sampling intervals. Table 5 in Appendix E further shows that proposed subspace method exhibits strong robustness to a wide range of learning rate choices.

43%

42%

41%

40%

rank=10

rank=50

39%

rank=25

rank=100

0

5

10

1520

25

30

35

40

45

50

AdaptationCycles

46%

Accuracy (%)

k=75

44%

k=100

42%

0

5

10

15

20

AdaptationCycles

46%

step=1

step=50

Accuracy(%

step=100

44%

42%

40%

0

5

10

15

20

AdaptationCycles

Ablation on subspace tracking. Ablation study on the proposed subspace tracking approach is provided in Table 4 in Appendix D, which demonstrates the effectiveness and indispensability of it in achieving robust continual adaptation under continually varying data distributions.

Computational Complexity and Efficiency. Our method only introduces additional memory to store a gradient queue to compute subspace (Section 4.3), which is typically less than 0 . 01 kn ≈ 0 . 5 n for k = 50 . Thus, it does not incur significant memory and computational costs. Memory and runtime comparison on ImageNetC with ResNet-50 is given in Table 7 in Appendix I.

7 Conclusion

To address the critical challenge of performance degeneration in unsupervised continual TTA, this work identified ED samples as an underlying cause of the degeneration in entropy-minimization methods. Furthermore, we revealed that the backward gradients of entropy-minimization exhibit an intrinsic low-dimensional structure, and demonstrated that constraining weight updates within a low-dimensional principal subspace can effectively suppress the detrimental impact of ED samples. Then, we proposed a novel subspace-based continual TTA method and proved its enhanced robustness against detrimental ED samples. Extensive experiments demonstrated that LCoTTA can effectively overcome degeneration and maintain robust performance in long-term adaptation scenarios.

Limitations. In this work, we only consider a fixed subspace dimension r , while an adaptive selection of r may further improve performance, e.g., larger at the beginning and lower later, or adjusting it based on the degree of distribution shift and severity changes. Moreover, our theoretical analysis of the stability of the subspace-based method holds only locally.

8 Acknowledgment

This work was supported in part by the Science and Technology Innovation 2030-Major Project under Grant 2022ZD0208701, and National Natural Science Foundation of China (NSFC) under Grant 62271314. The authors also express special thanks to Yepeng Yang , for the helpful discussion related to this work.

References

S. Schneider, E. Rusak, L. Eck, O. Bringmann, W. Brendel, and M. Bethge, 'Improving robustness against common corruptions by covariate shift adaptation,' Advances in Neural Information Processing Systems , vol. 33, pp. 11539-11551, 2020.

D. Wang, E. Shelhamer, S. Liu, B. Olshausen, and T. Darrell, 'Tent: Fully test-time adaptation by entropy minimization,' arXiv preprint arXiv:2006.10726 , 2020.

J. Liang, D. Hu, and J. Feng, 'Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation,' in International Conference on Machine Learning , pp. 6028-6039, 2020.

L. Yuan, B. Xie, and S. Li, 'Robust test-time adaptation in dynamic scenarios,' in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 15922-15932, 2023.

D. Chen, D. Wang, T. Darrell, and S. Ebrahimi, 'Contrastive test-time adaptation,' in CVPR , 2022.

J. Song, J. Lee, I. S. Kweon, and S. Choi, 'Ecotta: Memory-efficient continual test-time adaptation via self-distilled regularization,' in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 11920-11929, 2023.

S. Niu, J. Wu, Y. Zhang, Z. Wen, Y. Chen, P. Zhao, and M. Tan, 'Towards stable test-time adaptation in dynamic wild world,' in Internetional Conference on Learning Representations , 2023.

Q. Wang, O. Fink, L. Van Gool, and D. Dai, 'Continual test-time domain adaptation,' in Proceedings of Conference on Computer Vision and Pattern Recognition , 2022.

Y. Yuan, Y. Wu, X. Fan, M. Gong, W. Ma, and Q. Miao, 'Egst: Enhanced geometric structure transformer for point cloud registration,' IEEE Transactions on Visualization and Computer Graphics , vol. 30, no. 9, pp. 6222-6234, 2024.

Y. Sun, H. Zhang, H. Ding, T. Zhang, X. Ma, and Y.-G. Jiang, 'Sama: Towards multi-turn referential grounded video chat with large language models,' arXiv preprint arXiv:2505.18812 , 2025.

L. Zhao and Y. Shen, 'Proactive model adaptation against concept drift for online time series forecasting,' in Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining , ACM, feb 2025.

S. Wang, W. Liu, J. Chen, Y. Zhou, W. Gan, X. Zeng, Y. Che, S. Yu, X. Hao, K. Shao, et al. , 'Gui agents with foundation models: A comprehensive survey,' arXiv preprint arXiv:2411.04890 , 2024.

L. Zhao, S. Kong, and Y. Shen, 'DoubleAdapt: A meta-learning approach to incremental learning for stock trend forecasting,' in Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , ACM, aug 2023.

J. Lee, D. Das, J. Choo, and S. Choi, 'Towards open-set test-time adaptation utilizing the wisdom of crowds in entropy minimization,' in Proceedings of the IEEE/CVF International Conference on Computer Vision , pp. 16380-16389, 2023.

S. Niu, J. Wu, Y. Zhang, Y. Chen, S. Zheng, P. Zhao, and M. Tan, 'Efficient test-time model adaptation without forgetting,' in International Conference on Machine Learning , pp. 1688816905, 2022.

O. Press, S. Schneider, M. Kümmerer, and M. Bethge, 'Rdumb: A simple approach that questions our progress in continual test-time adaptation,' Advances in Neural Information Processing Systems , vol. 36, 2024.

J. Kang, N. Kim, J. Ok, and S. Kwak, 'Membn: Robust test-time adaptation via batch norm with statistics memory,' in European Conference on Computer Vision , pp. 467-483, Springer, 2025.

Y. Gao, X. Shi, Y. Zhu, H. Wang, Z. Tang, X. Zhou, M. Li, and D. N. Metaxas, 'Visual prompt tuning for test-time domain adaptation,' arXiv preprint arXiv:2210.04831 , 2022.

A. Khurana, S. Paul, P. Rai, S. Biswas, and G. Aggarwal, 'Single image test-time adaptation,' arXiv:2112.02355 , 2021.

M. Zhang, S. Levine, and C. Finn, 'Memo: Test time robustness via adaptation and augmentation,' Advances in Neural Information Processing Systems , vol. 35, pp. 38629-38642, 2022.

M. Boudiaf, R. Mueller, I. Ben Ayed, and L. Bertinetto, 'Parameter-free online test-time adaptation,' in IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 83448353, 2022.

Y. Zhang, X. Wang, K. Jin, K. Yuan, Z. Zhang, L. Wang, R. Jin, and T. Tan, 'Adanpc: Exploring non-parametric classifier for test-time adaptation,' in International Conference on Machine Learning , pp. 41647-41676, 2023.

Y. Tang, C. Zhang, H. Xu, S. Chen, J. Cheng, L. Leng, Q. Guo, and Z. He, 'Neuro-modulated hebbian learning for fully test-time adaptation,' in IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 3728-3738, 2023.

Y. Yuan, B. Xu, L. Hou, F. Sun, H. Shen, and X. Cheng, 'Tea: Test-time energy adaptation,' in IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 23901-23911, 2024.

T. Lee, J. Tremblay, V. Blukis, B. Wen, B.-U. Lee, I. Shin, S. Birchfield, I. S. Kweon, and K.-J. Yoon, 'Tta-cope: Test-time adaptation for category-level object pose estimation,' in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 21285-21295, 2023.

D. Lee, J. Yoon, and S. J. Hwang, 'Becotta: Input-dependent online blending of experts for continual test-time adaptation,' in International Conference on Machine Learning , 2024.

H. Lim, B. Kim, J. Choo, and S. Choi, 'Ttn: A domain-shift aware batch normalization in test-time adaptation,' arXiv preprint arXiv:2302.05155 , 2023.

S. Choi, S. Yang, S. Choi, and S. Yun, 'Improving test-time adaptation via shift-agnostic weight regularization and nearest source prototypes,' in European Conference on Computer Vision , pp. 440-458, Springer, 2022.

Y. Liu, P. Kothari, B. Van Delft, B. Bellot-Gurlet, T. Mordan, and A. Alahi, 'Ttt++: When does self-supervised test-time training fail or thrive?,' Advances in Neural Information Processing Systems , vol. 34, pp. 21808-21820, 2021.

K. Adachi, S. Yamaguchi, and A. Kumagai, 'Covariance-aware feature alignment with precomputed source statistics for test-time adaptation to multiple image corruptions,' in IEEE International Conference on Image Processing (ICIP) , pp. 800-804, 2023.

S. Jung, J. Lee, N. Kim, A. Shaban, B. Boots, and J. Choo, 'Cafa: Class-aware feature alignment for test-time adaptation,' in IEEE/CVF International Conference on Computer Vision , pp. 19060-19071, 2023.

T. Gong, Y. Kim, T. Lee, S. Chottananurak, and S.-J. Lee, 'Sotta: Robust test-time adaptation on noisy data streams,' Advances in Neural Information Processing Systems , vol. 36, 2024.

Z. Wang, Z. Chi, Y. Wu, L. Gu, Z. Liu, K. Plataniotis, and Y. Wang, 'Distribution alignment for fully test-time adaptation with dynamic online data streams,' in European Conference on Computer Vision , pp. 332-349, Springer, 2025.

X. Ruan and W. Tang, 'Fully test-time adaptation for object detection,' in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 1038-1047, 2024.

R. A. Marsden, M. Döbler, and B. Yang, 'Universal test-time adaptation through weight ensembling, diversity weighting, and prior correction,' in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , pp. 2555-2565, 2024.

Y. Zhao, T. Zhang, and H. Ji, 'Test-time model adaptation for image reconstruction using self-supervised adaptive layers,' in European Conference on Computer Vision , pp. 111-128, Springer, 2025.

X. Ma, J. Zhang, S. Guo, and W. Xu, 'Swapprompt: Test-time prompt adaptation for visionlanguage models,' Advances in Neural Information Processing Systems , vol. 36, 2024.

Y. Wang, J. Hong, A. Cheraghian, S. Rahman, D. Ahmedt-Aristizabal, L. Petersson, and M. Harandi, 'Continual test-time domain adaptation via dynamic sample selection,' in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , pp. 1701-1710, 2024.

Y. Zhang, A. Mehra, S. Niu, and J. Hamm, 'Dpcore: Dynamic prompt coreset for continual test-time adaptation,' in Forty-second International Conference on Machine Learning , 2024.

Y. Zhang, A. Mehra, and J. Hamm, 'Ot-vp: Optimal transport-guided visual prompting for test-time adaptation,' in 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) , pp. 1122-1132, IEEE, 2025.

C. Li, H. Farkhoor, R. Liu, and J. Yosinski, 'Measuring the intrinsic dimension of objective landscapes,' in International Conference on Learning Representations , 2018.

B. W. Larsen, S. Fort, N. Becker, and S. Ganguli, 'How many degrees of freedom do we need to train deep networks: a loss landscape perspective,' in International Conference on Learning Representations , 2021.

G. Gur-Ari, D. A. Roberts, and E. Dyer, 'Gradient descent happens in a tiny subspace,' arXiv preprint arXiv:1812.04754 , 2018.

F. Gressmann, Z. Eaton-Rosen, and C. Luschi, 'Improving neural network training in low dimensional random bases,' Advances in Neural Information Processing Systems , vol. 33, pp. 12140-12150, 2020.

T. Li, L. Tan, Z. Huang, Q. Tao, Y. Liu, and X. Huang, 'Low dimensional trajectory hypothesis is true: Dnns can be trained in tiny subspaces,' IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 45, no. 3, pp. 3411-3420, 2022.

T. Li, Z. Huang, Q. Tao, Y. Wu, and X. Huang, 'Trainable weight averaging: Efficient training by optimizing historical solutions,' in International Conference on Learning Representations , 2023.

Z. Li, S. Sajadmanesh, J. Li, and L. Lyu, 'Stella: Subspace learning in low-rank adaptation using stiefel manifold,' arXiv preprint arXiv:2510.01938 , 2025.

T. Gong, J. Jeong, T. Kim, Y. Kim, J. Shin, and S.-J. Lee, 'Note: Robust continual test-time adaptation against temporal correlation,' Advances in Neural Information Processing Systems , vol. 35, pp. 27253-27266, 2022.

M. Döbler, R. A. Marsden, and B. Yang, 'Robust mean teacher for continual and gradual test-time adaptation,' in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 7704-7714, 2023.

O. Press, R. Shwartz-Ziv, Y. LeCun, and M. Bethge, 'The entropy enigma: success and failure of entropy minimization,' in Proceedings of the 41st International Conference on Machine Learning , pp. 41064-41085, 2024.

D. Hendrycks, N. Mu, E. D. Cubuk, B. Zoph, J. Gilmer, and B. Lakshminarayanan, 'Augmix: A simple data processing method to improve robustness and uncertainty,' arXiv preprint arXiv:1912.02781 , 2019.

M. Stephan, M. D. Hoffman, D. M. Blei, et al. , 'Stochastic gradient descent as approximate bayesian inference,' Journal of Machine Learning Research , vol. 18, no. 134, pp. 1-35, 2017.

D. Hendrycks and T. Dietterich, 'Benchmarking neural network robustness to common corruptions and perturbations,' arXiv preprint arXiv:1903.12261 , 2019.

F. Croce, M. Andriushchenko, V. Sehwag, E. Debenedetti, N. Flammarion, M. Chiang, P. Mittal, and M. Hein, 'Robustbench: a standardized adversarial robustness benchmark,' arXiv preprint arXiv:2010.09670 , 2020.

D. Brahma and P. Rai, 'A probabilistic framework for lifelong test-time adaptation,' in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 3582-3591, 2023.

J. Lee, D. Jung, S. Lee, J. Park, J. Shin, U. Hwang, and S. Yoon, 'Entropy is not enough for test-time adaptation: From the perspective of disentangled factors,' International Conference on Learning Representations , 2024.

W. Choi, D.-Y. Kim, J. Park, J. Lee, Y . Park, D.-J. Han, and J. Moon, 'Adaptive energy alignment for accelerating test-time adaptation,' in International Conference on Learning Representations , 2025.

C. Ni, F. Lyu, J. Tan, F. Hu, R. Yao, and T. Zhou, 'Maintainingconsistentinterclasstopologyincontinualtest-timeadaptation,' in Proceedings of Conference on Computer Vision and Pattern Recognition , 2025.

A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V. Koltun, 'CARLA: An open urban driving simulator,' in Proceedings of the 1st Annual Conference on Robot Learning , pp. 1-16, 2017.

M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele, 'The cityscapes dataset for semantic urban scene understanding,' in Proceedings of the IEEE conference on computer vision and pattern recognition , pp. 3213-3223, 2016.

A Proof of Theorem 1

Consider the SGD update rule employed during TTA as

where g ( θ t ; B t ) = ∇ θ L ( θ t ; B t ) is gradient on the batch B t . Assume the batch B t contains both ET and ED samples, where the ET sample subset is denoted by ˆ B t whilst the ED sample subset is denoted by ˘ B t such that B t = [ ˆ B t , ˘ B t ] . Denote the gradients on ˆ B t and ˘ B t are given by

Denote the mean and covariance of ET and ED gradients by

where Σ ET and Σ ED are the batch-normalized covariance of ET and ED gradients, respectively. Empirical results in Section 4.1 indicate weak inter-sample correlation among ED-sample gradients.

We therefore model

with E [ ξ ED ] = 0 , and Cov( ξ ED ) = Σ ED, where ¯ g ED is the bias of ED gradients. ξ ED is scattered noise with covariance Σ ED = σ 2 ED I n +∆ with ∥ ∆ ∥ ≪ σ 2 ED .

TTA operates as a local fine-tuning of a pretrained model. The expected ET gradient ¯ g ET points toward a nearby local minimum, whereas its covariance Σ ET is low-rank (Figure 2), with effective rank r ≪ n . In contrast, the ED gradient dispersion is almost isotropic, Σ ED ≈ σ 2 ED I n . Moreover, assuming E B [(ˆ g -¯ g ET )(˘ g -¯ g ED ) T ] = 0 , the mean and covariance of the mini-batch gradient g ( θ t ; B t ) can be expressed as

When the learning rate η → 0 , the discrete dynamics (10) can be approximated by the continuous-time stochastic differential equation (SDE) [52]

with drift ¯ g ( θ ) = ¯ g ET +¯ g ED and diffusion Σ( θ ) = Σ ET +Σ ED.

Define a local expected-gradient equilibrium θ · that

at which ET corrective force balances ED bias. Choose the quadratic Lyapunov function V ( θ ) = 1 2 ∥ θ -θ · ∥ 2 , its infinitesimal generator is

Then, a sufficient condition for mean-square stability is given by

In the worst case that the ED bias is opposite to ( θ -θ · ) , it follows from (15) that

If either the bias magnitude ∥ ¯ g ED ∥ or the noise level Tr (Σ) is too large that exceeds a critical threshold, stable convergence cannot be guaranteed.

Next, we consider subspace projection based SGD to show its advantage. Retain only the first principal ET directions U r and let P r = U r U T r be the projection matrix. With P r Σ P T r = P r Σ 1 / 2 ( P r Σ 1 / 2 ) T , the subspace projected SDE can be expressed as

Similarly, we can derive the corresponding stability condition as

As ET directions lie almost entirely in U r , we have P r ¯ g ET ≈ ¯ g ET. Meanwhile, the subspace projection eliminates most ED bias outside the subspace, hence for r ≪ n we have P r ¯ g ED ≈ 0 . Moreover, diffusion noise is greatly reduced by subspace projection as

where λ i is the i -th eigenvalue of Σ ET, with Tr ( P r Σ ET P T r ) = ∑ r i =1 λ i . Then, under these assumptions and with r ≪ n , the sufficient condition (18) can be simplified as

Compared with the condition (16) for full-space SGD, the noise term Tr (Σ) = ∑ n i =1 λ i + nσ 2 ED is largely reduced to ∑ r i =1 λ i + rσ 2 ED . In the setting of the proposed subspace projection method with r ≪ n , we have

Consequently, it is easy to see that, the projection onto the subspace U r can substantially enhance the robustness of entropy minimization based TTA against the detrimental effect of ED samples.

B Updating in Low-Dimensional Subspaces Constrains the Adaptation

Adapting model parameters within a low-dimensional subspace offers a controlled mechanism for constraining weight changes during adaptation, which in turn enhances the stability of the process. Let θ s ∈ R n represent the parameter vector of the pre-trained source model, and U r ∈ R n × r denote an orthonormal basis spanning a r -dimensional subspace, where r ≪ n . Weight updates with and without subspace constraints can be compared as follows.

When the updates are restricted to the subspace defined by U r , the updated weights are expressed as ˜ θ t = θ s + U r U T r ∆ θ, whereas without the subspace constraint, the updates are given by θ t = θ s +∆ θ. Let U = [ U r , U ⊥ r ] ∈ R n × n represent the full orthonormal basis spanning the original parameter space, where U r ∈ R n × r and U ⊥ r ∈ R n × ( n -r ) span two orthogonal complementary subspaces. Denote

Using the orthonormality of U , the weight updates with and without subspace representations can be expressed as

Clearly, in the space defined by U , the update ˜ θ t involves only a much smaller r -dimensional subspace, as r ≪ n , in contrast to the full update θ t . In our experiments, we use r = 5 , resulting in an extremely low-dimensional subspace for adaptation.

Given that U r has orthonormal columns, satisfying U T r U r = I r × r , for any ∆ θ ∈ R n , we have ∥ U r U T r ∆ θ ∥ 2 2 = (∆ θ T U r )( U T r ∆ θ ) = ∥ U T r ∆ θ ∥ 2 2 ≤ ∥ ∆ θ ∥ 2 2 . This implies that the magnitude of the weight updates is reduced when using subspace projection as

Thus, adapting weights within a low-dimensional subspace reduces the magnitude of changes and ensures that the updated weights retain higher dependency on the source model. This property, together with the suppression effect of low-dimensional principal subspace on deceptive samples (Section 4.2), benefits the stability in long-term continual adaptation, with improved robustness to hyperparameters (see Table 5 in the ablation study).

C Results of Single-Epoch Adaptation on ImageNetC

Table 2 and Table 3 present the performance under the standard TTA setting over a single cycle, demonstrating that our method can still achieve competitive results in short-term adaptation scenarios. We also report the performance of several more recent methods including BeCoTTA [26], AEA [57], and TCA [58]. Since some of these methods do not have publicly available code for reproduction, we directly cite their reported results from their respective papers for comparison.

D Effectiveness of the Proposed Subspace Tracking Approach

We conduct ablation experiments to evaluate the effect of the proposed subspace tracking approach on continual adaptation on ImageNetC with ResNet-50. We consider four variants: 1) our method without using subspace projection of the gradients; 2) our method using a fixed 5-dimensional subspace extracted from source data, denoted as 'from source domain'; 3) our method using a 5-dimensional subspace tracked online but delayed for one corruption type, denoted as 'tracked with delay'; 4) our method using the proposed subspace tracking approach, denoted as 'tracked online'. Table 4 shows the results of these variants for continual adaptation over one-cycle of 15 corruptions,

which demonstrates the effectiveness and indispensability of the proposed dynamic subspace tacking approach in achieving robust continual adaptation.

E Robustness to Learning Rate

Table 5 presents a comparison between Tent and our method across different learning rates in continual adaptation on ImageNet-C. Consistent with results from previous studies [2, 8], entropyminimization-based TTA methods, such as Tent, are highly sensitive to the choice of learning rate. In contrast, our method is much more robust to variation in learning rate, attributed to its ability to suppress deceptive samples and constrain updates within a principal subspace. This robustness substantially alleviates the need for meticulous hyperparameter tuning, offering a more practical and reliable solution for continual adaptation.

F Standard Deviation of Our Method

Table 6 presents the standard deviations of our method with different random seeds on CIFAR100-C with ResNext-29, ImageNetC with ResNet-50, and ImageNetC with ViT-B/16. We use five different random seeds and report the mean and standard deviation.

G Sample Entropy Visualization of Model Prediction

Figure 7 shows the distribution of prediction entropy from the network when the test inputs are sorted by ERS. It can be observed that a naive entropy filtering strategy can remove samples with high entropy values in the middle range, thereby robustifying adaptation performance. However, since many entropy-deceptive (ED) samples can also exhibit low entropy values, they cannot be filtered out by simple entropy filtering strategy . This phenomenon is more pronounced on the more challenging ImageNet dataset, which explains why the filtering strategy still fails in long-term adaptation scenarios.

It can be seen from Figure 7 that, using entropy-based sample filtering (ESF) alone cannot distinguish between ET and ED samples. Thus, using ESF alone still suffers from degeneration in continual TTA.

SampleEntropy

AverageEntropyperBatch

4

entropy Value

2

0

0

2000

4000

6000

8000

10000

Most truthful

SampleIndex(SortedByERS)

Mostdeceptive

samples

→samples

6

entropy Value

2

1

Sample Entropy

AverageEntropyperBatch

0

0

10000

20000

30000

40000

50000

Most truthful

Sample Index (Sorted By ERS)

Most deceptive

samples

→samples

(b) ResNet-50 on ImageNetC (Gaussian noise corruption with severity-5)

H The Reduction of Entropy-Deceptive Samples During Continual Adaptation

We conduct experiment to record the number changes of ED samples during the adaptation process. As shown in Figure 8, during continual adaptation, the number of ED samples gradually reduces.

ChangesofDeceptiveSamplesinadaptationcycles

48800

48700

48600

Total Count

48500

48400

48300

48200

2

7

Cycles

I Comparison of Computational Complexity and Efficiency

As discussed in Section 6.2, our method only introduces an additional gradient queue and negligible PCA computation overhead. Table 7 presents a comparison of memory consumption and runtime among different methods. As can be seen, the computational and time complexity of our method are comparable to those of Tent, and our approach remains competitive among the compared methods.

J Low-Dimension Structure of Gradients for ED and ET Samples

As discussed in Section 4.1, ET sample gradients are highly correlated and align well in parameter space, forming a clear low-dimensional structure. In contrast, ED gradients are scattered and lack such alignment. To illustrate this, we sort samples by ERS and analyze the top, middle, and last 20% using PCA. As shown in Figure 9, only ET samples reveal a pronounced low-dimensional subspace.

35%

35%

%

35%

30%

30%

30%

25%

25%

25%

Explained

Explained

Explained

20%

20%

20%

15%

15%

15%

ative

ative

Cumulative

mulat

10%

Cumulat

10%

10%

cu

5%

5%

5%

U

0%

0%

0%

0

2

4

6

8

0

2

4

6

8

0

2

4

6

8

Principal ComponentIndex

Principal ComponentIndex

Principal Component Index

K Results on CIFAR100-to- CIFAR100C

K.1 Results of Continual Adaptation on CIFAR100C

Figure 10 presents the performance comparison in the considered long-term adaptation setting on the CIFAR100-to- CIFAR100C task over 100 adaptation cycles. Using teacher-student networks and partial model resetting, CoTTA and ECoTTA mitigate the degeneration but still degrade over time due to the inherent instability of unsupervised adaptation. AdaContrast achieves strong initial performance but also degenerates over prolonged adaptation. In contrast, our method maintains robust performance throughout all adaptation cycles, effectively overcoming the degeneration issue in long-term continual adaptation.

K.2 Results of Single Epoch on CIFAR100C

Table 8 presents the results of single epoch adaptation on CIFAR100C with ResNeXt-29.

80%

Source

BN

SAR

Adacontrast

ETA

ECOTTA

Tent

CoTTA

RoTTA

70%

60%

%

50%

40%

30%

20%

10%

0%

1

5

10

15

20

25

30

35

40

45

50

55

60

65

70

75

80

85

06

95

100

Adaptation Cycles

L Results of Semantic Segmentation

L.1 Results on the Segmentation Task on the CarlaTTA Dataset

In this section, we conduct online continual test-time adaptation experiments using the CARLA simulator [59] across three domain-shift scenarios with varying weather and visual conditions: dayto-night, clean-to-fog, and clean-to-rain. As shown in Table 9a to Table 9c, our method not only adapts well to semantic segmentation tasks, but also consistently outperforms baseline methods across different scenarios.

L.2 Visualization Results on the Segmentation Task on the Cityscapes Dataset

We further evaluate our method on segmentation tasks using the real-world Cityscapes [60] dataset under corrupted target domains for a more intuitive demonstration of its effectiveness. As shown in Figure 11, while Tent performs well after one epoch of adaptation, its performance deteriorates with prolonged continual adaptation. In contrast, our method maintains stable and accurate segmentation results even after 10 epochs, demonstrating superior robustness in long-term adaptation scenarios.

(a) Results on day2night setting.

(b) Results on clear2fog setting.

(c) Results on clean2rain setting.

(d) Results on dynamic setting.

Source

Tent, 1 epoch

Ours, 1 epoch

Corruption

Tent, 5 epoch

Ours, 5 epoch

Ground Truth

Tent, 10 epoch

Ours, 10 epoch

Source only

Tent, 15 epoch

Ours, 15 epoch

M Experimental Setup and Hardware Configuration

We conduct the main experiments of 50 cycles TTA in Section 6.1 on a Linux server equipped with 8 NVIDIA V100 GPUs with 32GB memory each, and an Intel(R) Xeon(R) Platinum 8280 CPU @ 2.70GHz. All other experiments in Section 6.2 are performed on a PC platform equipped with a single Nvidia RTX 3090 GPU with 24GB memory, including the efficiency analysis in Table 7.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The paper's main claims, as stated in the abstract and introduction, are clearly articulated and are well-supported by both the theoretical analysis and experimental results. Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: The paper explicitly discusses its limitations in Section 7, including assumptions made in the methodology and factors that may affect generalizability, providing a transparent assessment of where the results may not fully extend.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: The main theorem in our paper is accompanied by clearly stated assumptions and is supported by a complete proof provided in Section A.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We have released the full implementation of our algorithm, and all details necessary for reproduction are provided. The code and instructions for reproducing the main experimental results are available via the anonymous repository linked in abstract.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We provide an anonymous repository link in the abstract containing the full code and instructions needed to reproduce all main experimental results. All datasets used in our experiments are open source and clearly referenced at the corresponding points in the paper.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: All experimental settings, including data splits, hyperparameters, and optimizer details, are thoroughly described in the main text and appendix to ensure clarity and reproducibility.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We report the standard deviations across multiple random seeds for our main experiments, as shown in Section F.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We provide an analysis of computational complexity and efficiency for our proposed method in Section I, and detailed information about the experimental hardware configuration, including CPU and GPU specifications, in Section M.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: Our research fully adheres to the NeurIPS Code of Ethics in all respect.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: Our work focuses on foundational research in test-time adaptation algorithm and is not tied to specific applications or deployments, and does not have direct societal impacts.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: This paper poses no such risks

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: All assets used in the paper are properly cited and appropriately credited, with licenses and terms of use correctly respected in the manuscript.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: This paper does not release new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: This paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: This paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.

Answer: [NA]

Justification: The core method development in this research does not involve LLMs as any important, original, or non-standard components.

Guidelines:

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

Please refer to our LLM policy ( https://neurips.cc/Conferences/2025/LLM ) for what should or should not be described.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nExisting TTA methods primarily rely on unsupervised entropy minimization, which can lead to trivial low-entropy yet inaccurate predictions—a failure mode attributed to entropy-deceptive (ED) samples. To address this issue, the authors observe that ED samples have gradient directions misaligned with a low-dimensional subspace formed by the highly correlated gradients of entropy-truthful (ET) samples. Leveraging this insight, they propose LCoTTA, a method that constrains model updates within the ET-driven subspace to suppress the influence of ED samples. Both theoretical analysis and comprehensive experiments validate the robustness and effectiveness of the proposed approach.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n- The proposed method offers a novel and meaningful perspective on addressing performance degradation in test-time adaptation by analyzing the role of misleading samples and utilizing the low-dimensional structure of gradients.\n- The experiments are comprehensive and convincing, demonstrating the method's effectiveness across various benchmarks, model architectures, and adaptation scenarios.\n- The inclusion of theoretical analysis strengthens the work, providing additional justification for the method's design and helping to enhance its overall credibility.\n\n**Weaknesses**\n\n- The baselines used in the experiments are somewhat outdated. It would be better to include one or two more recent baselines for comparison.\n- The authors claim that \"The proposed method is complementary to existing approaches for robust TTA.\" It would be better if this claim were supported by some experiments.\n- Lines 22 and 92 contain excessive, unstructured citations that hinder clarity. The Introduction overreferences a well-known definition, while the Related Work section lists too many papers without context. Using fewer, representative works would improve focus and readability.\n\nQUESTIONS:\nED samples also appear in standard TTA scenarios, so why don’t existing TTA methods suffer from significant performance degradation in those settings?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-19"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies the impact of correctly classified and incorrectly classified samples on the stability of entropy minimization. It argues that (i) incorrectly classified samples negatively impact the stability of entropy minimization during long-term adaptation and that (ii) the gradients of wrongly predicted samples are not correlated, compared to correctly predicted ones. Based on this observation, the paper proposes to “denoise” the gradient by using a projection onto the $K$ principal components instead. The paper shows that entropy minimization can be stabilized with this approach, despite adapting on millions of samples.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n- The experimental results of the paper are promising: the adaptation accuracy remains high despite long-term adaptation on 37 million samples.\n- The analysis on the correlation of gradients is interesting and provides new insights into why and how entropy minimization can fail.\n- The paper provides theoretical justification for the stability of gradient descent with subspace gradients.\n\n**Weaknesses:**\n\n- The analysis in Section 3.3 is interesting and the results are intuitive. However, the observation that a large share of incorrectly predicted samples leads to degeneration in entropy minimization has also been made by [1]. This connection could be discussed more explicitly.\n\n**Minor comments:**\n\n- It would be helpful to use consistent color coding for different methods in Figure 4.\n- Line 308: Should it be c) instead of d)?\n\n\n[1] Press, Ori, et al. \"The entropy enigma: Success and failure of entropy minimization.\" ICML (2024).\n\nQUESTIONS:\n- I understand there is no resetting mechanism that resets the model to the source model. Is that correct? If not, when is it triggered?\n- How does the method scale with the number of adapted parameters? Would it be feasible to use the method for adapting more than just BN parameters, i.e., computing PCs on a larger set of model parameters?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the problem of continual test-time adaptation (TTA). The authors identify a key failure mode of entropy-minimization–based TTA: entropy-deceptive (ED) samples, on which the model is over-confident but wrong, causing collapse over long adaptation. They further observe that gradients from entropy-truthful (ET) samples lie in a correlated, low-dimensional subspace, whereas ED gradients are scattered. Building on this, they propose LCoTTA, which (1) tracks the principal subspace of recent gradients online, (2) projects batch gradients into that subspace before updating only the normalization-layer parameters, and (3) optionally filters out high-entropy samples. Theoretical analysis via SDE stability conditions shows that subspace projection dramatically reduces noise from ED samples.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The identification of entropy-deceptive samples as the root cause of degeneration in long-term TTA is interesting and well motivated by empirical observations (Fig. 1) and formalized via an Entropy Reliability Score (ERS).\n- The discovery that ET gradients concentrate in a low-dimensional principal subspace (PCA explained variance > 90 % in first 10 PCs) is insightful, and it provides a clear mechanism to suppress ED noise.\n- LCoTTA’s online subspace tracking and projection are simple to implement and incur minimal overhead by updating only BN/LN parameters (< 1 % of weights). The SDE-based stability analysis (Theorem 1) rigorously explains why subspace constraint improves robustness.\n\nWeakness:\n\nLack of some related work, including VDP [1], CTTDA [2] and BECoTTA [3].\n\n\n[1] Gan, Yulu, et al. \"Decorate the newcomers: Visual domain prompt for continual test time adaptation.\" Proceedings of the AAAI conference on artificial intelligence. Vol. 37. No. 6. 2023.\n\n[2] Wang, Yanshuo, et al. \"Continual test-time domain adaptation via dynamic sample selection.\" Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 2024.\n\n[3] Lee, Daeun, Jaehong Yoon, and Sung Ju Hwang. \"Becotta: Input-dependent online blending of experts for continual test-time adaptation.\" arXiv preprint arXiv:2402.08712 (2024).\n\nQUESTIONS:\n- How did you choose the fixed subspace rank r (e.g. 25)? Have you tried adapting r dynamically based on explained-variance thresholds, and if so, how does that affect performance and stability?\n- Your entropy-sample filter (ESF) uses a threshold to discard “high-entropy” inputs before projection. How sensitive is LCoTTA to that threshold choice, and what rules of thumb can practitioners use to set it?\n- The SDE analysis assumes gradients decompose into low-rank signal plus isotropic noise. In practice, do you observe violations of these assumptions, and how robust is LCoTTA when noise is not uniform?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the challenge of Lifelong Test-Time Adaptation, where models must adapt to evolving data streams over long periods. The authors identify the presence of entropy-deceptive, instances where the model makes highly confident but incorrect predictions. These samples provide misleading gradients that degrade performance over time. The paper proposes LCoTTA that constrains model updates to the principal gradient subspace. By tracking the subspace online using PCA on a queue of recent gradients and projecting new gradients onto it, LCoTTA effectively filters out the harmful influence of entropy-deceptive sample. Experiments on several challenging benchmarks show that LCoTTA outperforms strong baselines.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n1. The paper addresses the Lifelong Test-Time Adaptation, which is a challenging and practical scenario.\n2. The paper is well written and easy to follow.\n3. The paper provides extensive experiments, showing the effectiveness and versatility of the proposed LCoTTA.\n4. The analysis of the degeneration problem is insightful.\n\n\nWeaknesses\n1. LCoTTA introduces several hyperparameters (e.g., subspace dimension r and the gradient queue length k), which are quite sensitive (fig 5, 6) and may require careful tuning for new datasets.\n2. The exact values in fig 4 is not available and it is hard the get the real improvements compared to the baselines.\n3. The improvement on Single-Epoch Adaptation is limited (tab 3, 4, 8).\n4. The improvement on Segmentation Task is also quite limited (tab 9).\n\nQUESTIONS:\n1. There are too many hyperparameters. How are they selected?\n2. What are the exact values in fig 4?\n3. The paper states that LCoTTA is complementary to other TTA techniques. Have you performed any experiments combining LCoTTA with a method that uses a different mechanism, such as the teacher-student framework in CoTTA?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}