Title: ReservoirTTA: Prolonged Test-time Adaptation for Evolving and Recurring Domains
Venue: NeurIPS 2025
Accepted: yes
Keywords: Test-Time Adaptation

Domain Shift

Catastrophic Forgetting

Recurring Domains
Abstract: This paper introduces **ReservoirTTA**, a novel plug–in framework designed for prolonged test–time adaptation (TTA) in scenarios where the test domain continuously shifts over time, including cases where domains recur or evolve gradually.
At its core, ReservoirTTA maintains a reservoir of domain-specialized models—an adaptive test-time model ensemble—that both detects new domains via online clustering over style features of incoming samples and routes each sample to the appropriate specialized model, and thereby enables domain-specific adaptation.
This multi-model strategy overcomes key limitations of single model adaptation, such as catastrophic forgetting, inter-domain interference, and error accumulation, ensuring robust and stable performance on sustained non-stationary test distributions.
Our theoretical analysis reveals key components that bound parameter variance and prevent model collapse, while our plug–in TTA module mitigates catastrophic forgetting of previously encountered domains. Extensive experiments on scene-level corruption benchmarks (ImageNet-C, CIFAR-10/100-C), object-level style shifts (DomainNet-126, PACS), and semantic segmentation (Cityscapes→ACDC) — covering recurring and continuously evolving domain shifts — show that ReservoirTTA substantially improves adaptation accuracy and maintains stable performance across prolonged, recurring shifts, outperforming state-of-the-art methods. Our code is publicly available at https://github.com/LTS5/ReservoirTTA.

ReservoirTTA: Prolonged Test-time Adaptation for Evolving and Recurring Domains

Abstract

This paper introduces ReservoirTTA , a novel plug-in framework designed for prolonged test-time adaptation (TTA) in scenarios where the test domain continuously shifts over time, including cases where domains recur or evolve gradually. At its core, ReservoirTTA maintains a reservoir of domain-specialized models-an adaptive test-time model ensemble-that both detects new domains via online clustering over style features of incoming samples and routes each sample to the appropriate specialized model, and thereby enables domain-specific adaptation. This multi-model strategy overcomes key limitations of single model adaptation, such as catastrophic forgetting, inter-domain interference, and error accumulation, ensuring robust and stable performance on sustained non-stationary test distributions. Our theoretical analysis reveals key components that bound parameter variance and prevent model collapse, while our plug-in TTA module mitigates catastrophic forgetting of previously encountered domains. Extensive experiments on scenelevel corruption benchmarks (ImageNet-C, CIFAR-10/100-C), object-level style shifts (DomainNet-126, PACS), and semantic segmentation (Cityscapes → ACDC) -covering recurring and continuously evolving domain shifts - show that ReservoirTTA substantially improves adaptation accuracy and maintains stable performance across prolonged, recurring shifts, outperforming state-of-the-art methods. Our code is publicly available at https://github.com/LTS5/ReservoirTTA.

1 Introduction

Deep networks have achieved state-of-the-art performance across many tasks, but their reliability degrades when test-time data deviates from the training distribution. Real-world deployment scenarios such as autonomous driving or surveillance often involve dynamic shifts caused by changing weather, sensor degradation, or environmental variation. These settings call for robust test-time adaptation (TTA) methods [26, 47, 21, 40, 37] that enable pre-trained models to adapt on-the-fly, ideally over prolonged periods, without catastrophic forgetting or model collapse. Most existing TTA methods, e.g., efficient TTA (ETA) [27], assume each domain appears only once in the test stream. In realworld long-term deployments, however, domain conditions often recur. As Figure 1 (left) shows, visual distributions may shift and later reappear. Empirically, this recurring behavior destabilizes ETA [27], which lacks explicit regularization, whereas anti-forgetting TTA (EATA) [27] maintains greater long-term stability by constraining parameter drift. Even so, when domains re-emerge, these regularized methods remain vulnerable to catastrophic forgetting , as illustrated in Figure 1 (right).

To address this challenge, we clarify two related yet distinct axes of variationstyle and domain -and revisit how their boundaries are detected. Prior work typically treats domains as discrete, sourceannotated groups (e.g., different sensors or collection conditions). Styles and domains may not map

Time

Sequence of 15 visual domains on CIFAR100-C

Recurrence on CIFAR100-C Snow visual domain

1

2

3

4

5

6

7

8

9 1011121314151617181920

ETA

EATA

EATA+ReservoirTTA

40

35

Error Rate

35

Rate

30

30

Error

25

25

1

5

10

15

20

O

200

400

600

800

1000

Recurrence on CIFAR1o0-C

Adaptation Step on CIFAR1oO-C Snow

one-to-one: a single domain can exhibit multiple styles due to intra-domain variability, and distinct domains can share similar style signatures (for example, zoom blur and defocus blur yield nearly identical style statistics). Nevertheless, in our framework, a domain is represented by a group of similar styles, where style is computed as the set of low-level appearance attributes captured via channel-wise feature statistics from early layers of a pre-trained VGG network [35]. This allows us to leverage the style embeddings to detect new domains or recognize the recurring ones via online clustering without over-fragmentation-an issue observed in the recent methods lacking robust style representations (see Figure 17) [47, 3]. Based on the style of the current test samples, we route them to the corresponding domain model in our reservoir, where specialized adaptation is performed.

In summary, we introduce ReservoirTTA , which maintains a pool of domain-specialized models, adapts each one independently with its corresponding TTA updates, and combines their parameters in a weighted ensemble for final predictions. Our contributions are as follows:

Multi-model TTA with domain-aware specialization. ReservoirTTA explicitly decouples adaptation across domains using a pool of models. This plug-in design supports diverse architectures and lightweight adapters, including normalization statistics, prompts, and LoRA [13] modules.

Style-driven clustering for online domain discovery. We propose an online clustering algorithm based on deep style features. By quantifying style at test time, our method can detect and reuse previously adapted models, enabling continual and efficient re-adaptation.

Theoretical insights into long-term stability. We provide theoretical bounds showing how parameter regularization curbs collapse in single-domain TTA-clarifying the stability of methods such as EATA [27]-and motivate the modular design of ReservoirTTA for maintaining stability under recurring domain shifts.

Extensive empirical evaluation. We test ReservoirTTA on long-term/recurring TTA: classification on ImageNet-C , CIFAR-10/100-C , and segmentation on Cityscapes → ACDC . We also evaluate object-level style shifts ( DomainNet-126 , PACS ), consistently surpassing PeTTA [12], RDumb [31], and CoTTA [42] across datasets/backbones.

2 Related Work

Prior works on prolonged, recurring test-time adaptation fall into two main strands. Continual TTA methods [42, 45, 36] continually update a single model to track evolving domains but suffer from drift and forgetting when domains recur, while robust/persistent TTA methods [31, 12, 27, 24] employ techniques such as variance constraints or periodic resets to preserve stability yet lack efficient re-adaptation to previously seen shifts. For a concise survey of representative algorithms and their limitations in recurring and evolving settings, see Appendix A.

3 Recurring Continual TTA and Theoretical Analysis

3.1 Background

Notation and Setting. We consider a deep network f θ : X → Y , pre-trained on an inaccessible source dataset. At test time, the model parameters θ are updated using an unsupervised TTA objective L TTA ( x , θ ) on incoming test images x . In practice, these updates typically affect only a subset of parameters. Test data is received sequentially in batches, and we assume that a batch of test images B t := { x 1 t , . . . , x b t } at time t is drawn from a test domain distribution D t , where D t ∈ {D 1 , . . . , D K } . These distributions are unknown during training and evolve dynamically after deployment. We distinguish three primary scenarios for prolonged domain evolution at test time:

Recurring Continual Structure Change (CSC): Domains change in a predictable order but may reappear (e.g., day-night cycles), denoted as D 1 →D 2 →D 3 ↬ D 1 .

Recurring Continual Dynamic Change (CDC): Domains shift unpredictably, though some conditions recur (e.g., abrupt weather changes), represented as D 1 glyph[axisshort]glyph[axisshort]glyph[arrowaxisright] D 4 glyph[axisshort]glyph[axisshort]glyph[arrowaxisright] D 5 ↬ D 4 .

Continuously Changing Corruptions (CCC): Domains evolve gradually via incremental changes (e.g., weather or degradation), so that D 1 →D ′ 1 →D ′′ 1 →D 2 , where D ′ i and D ′′ i denote successive variations of D i before transitioning to D i +1 .

For notation, we use D i →D j for structured shifts in CSC and CCC, D i glyph[axisshort]glyph[axisshort]glyph[arrowaxisright] D j for unstructured shifts in CDC, and D i ↬ D j to denote recurring domains.

3.2 Test-Time Adaptation Trajectory

Stability Regions for Individual Domains. We begin by analyzing how standard single-model TTA updates via stochastic gradient descent (SGD) can cause parameter variance to grow linearly over time, increasing the risk of drifting outside the stability region.

Assumption 1. At test-time, the model is updated using an unsupervised TTA objective on the target domain, L TTA ( θ, x ) , which serves as a surrogate for the true task loss L Task ( θ ) = E ( x , y ) [ L sup ( x , y , θ )] , where L sup measures model performance using ground truth labels. The optimal parameters for the given task are given as: θ ∗ Task = arg min θ L Task ( θ ) .

Definition 1 (Stability Region) . For a task with optimal parameters θ ∗ Task , the stability region is defined as the set of parameters { θ : | θ -θ ∗ Task | ≤ β } , where β is the stability radius beyond which model performance collapses when updated using the TTA objective L TTA ( θ, x ) .

Lemma 1 (Parameter Variance Growth) . Under standard SGD-based adaptation, the variance of the updated parameters grows linearly with adaptation steps:

where η is the learning rate and ¯ V is the average gradient variance.

Theorem 1 (Bound on Divergence Probability) . Let θ t denote the model parameters at time t , and let θ ∗ Task be the task-specific optimum. Suppose that E [ θ t ] → θ ∗ Task and ∥ E [ θ t ] -θ ∗ Task ∥ < ∥ θ 0 -θ ∗ Task ∥ . Then for any threshold β > ∥ θ 0 -θ ∗ Task ∥ , the probability of divergence from the stability region is bounded by:

where θ 0 represents the initial model parameters.

The above analysis reveals why conventional TTA approaches often suffer from model collapse during prolonged adaptation: as the number of update steps increases, so does the variance of model parameters , eventually causing them to drift beyond the stability radius. The proof of Theorem 1 is provided in Appendix B.1. Next, we examine strategies to mitigate this variance growth.

Variance Reduction Strategies. To address the parameter variance growth problem, we explore two key strategies utilized in [27, 24]:

Proposition 1 (Sample Filtering) . Using an active sample selection function S ( x ) that filters out unreliable samples with high entropy reduces the effective gradient variance.

In practice, not all test samples contribute equally to model adaptation. One remedy would be to employ an active sample selection function S ( x ) that filters out unreliable samples-those with high entropy or that are redundant (see [27, 24]). Thus, the effective gradient update becomes:

thereby reducing effective gradient variance to V eff < ¯ V while preserving linear dependence on t .

Proposition 2 (Weight Ensembling) . Interpolating the updated parameters with the source model parameters constrains the adaptation trajectory and bounds the overall variance as:

where α ∈ (0 , 1] controls the contribution of source model parameters.

Another common strategy to bound the variance is to update parameters by interpolating with the source model θ 0 [24], thereby anchoring the adaptation steps:

This 'weight ensembling' bounds the overall variance, as each gradient update is geometrically damped by a factor of α 2( t -i ) . A similar strategy [27] applies Fisher regularization [18] with respect to the source parameters θ 0 in the TTA loss:

where, Ω = diag ([ ω 1 , . . . , ω n ]) is the Fisher coefficient matrix that weights each parameters θ = [ θ 1 , . . . , θ n ] based on its Fisher Information, and λ sets the regularization strength. This formulation keeps the updated parameters close to the source and is equivalent to 'weight ensembling.'

A detailed proof by induction and the complete variance analysis are provided in Appendix B.1.

Parameter Drift in Recurring Continual TTA. Even with these variance control strategies (e.g., sample filtering and weight ensembling), a single adapting model remains vulnerable to parameter drift when the shift between domain-optimal parameters exceeds the stability radius, i.e., ∥ θ ∗ Task i -θ ∗ Task i +1 ∥ > β i +1 , for some i. As illustrated in Figure 5 of Appendix B.2, such shifts cause the model's parameter trajectory { θ t } to deviate from the optimal region for a given domain, leading to catastrophic forgetting and negative transfer.

ReservoirTTA: Decoupled Adaptation Across Domains. To mitigate catastrophic forgetting and inter-domain interference in recurring continual TTA, we propose ReservoirTTA, a novel framework that partitions adaptation across domains by maintaining up to K domain-specialized models. Each reservoir component is updated exclusively when its corresponding domain is active, thereby isolating the test-time objective E x ∼D i [ L TTA ( θ, x )] for i ∈ { 1 , . . . , K } . While existing strategies-such as sample filtering [27], weight ensembling and Fisher regularization [24] help control the variance of updates and prevent model collapse, they do not fully address catastrophic forgetting when domains reoccur. In contrast, our plug-in ReservoirTTA module disentangles domain-specific adaptation and further mitigates unnecessary re-adaptation when previously encountered domains return.

4 Methodology

In this section, we introduce our framework, ReservoirTTA , for prolonged TTA in environments with recurring and evolving domains. As illustrated in Figure 2, our framework comprises four stages: (1) Style Characterization and Domain Identification -leveraging style-based online clustering to determine the domain characteristics of the current test batch; (2) Model Reservoir Initialization -allocating a new domain-specific model when a novel domain is detected in the domain identification step; (3) Model Reservoir Adaptation -selectively updating only the model associated with the current domain using state-of-the-art TTA techniques; and (4) Model Prediction -Predictions are then obtained via the ensemble's parameters from all domain-specific models (see pseudocode in Appendix C).

Style Quantification

Feature Maps

(2)Model Reservoir Initialization

OnLine Clustering

Style Reservoir

Domain

Exists

New

Domain

(3)Model Reservoir Adaptation

(4)Model Prediction

ArgMax(.)

clone(.)

ArgMin(.)

Model Reservoir

Domain

Assignment

(1)Style Characterization & Domain Identification

Legend

forward

frozen weights

trainable weights

model selection

backward

style centroids

VGG-19

Test

Batch

Style

LogVar(.)

yes

no

4.1 Style Characterization and Domain Identification

Style Characterization. To distinguish domains at test time, following prior style-characterization work [9, 15, 41, 35], we encode image style as batch-wise log-variances from the first L layers of a frozen, ImageNet-trained VGG-19 g style. Given a batch B t ∼ D t and feature maps { z 1 , . . . , z L } with z l ∈ R b × h l × w l × c l , the style vector for layer l is computed as s l ( B t ) = logvar ( z l ) where logvar computes the natural logarithm of the variance over the batch, height, and width. The overall style descriptor is formed by concatenating these vectors: s t = [ s 1 ( B t ) , . . . , s L ( B t ) ] ∈ R d . This architecture-agnostic statistic captures robust texture cues while remaining independent of the source model; Appendix D analyzes VGG configurations and shows that VGG-based style vectors yield more stable, higher-quality style clustering than source-model and ViT-based alternatives.

Domain Identification via Online Clustering. At test time, the number of visual domains is unknown. We introduce an online clustering algorithm inspired by DP-Means [19] to dynamically identify new domains using style vectors s t . At each timestep t , we maintain centroids C t = [ c 1 t , . . . , c K t t ] ∈ R d × K t , where K t is the number of domains identified up to time t . C t is initialized with a single centroid as the mean style feature from the source domain ( K 0 = 1 ). A new cluster is created if the current style vector is sufficiently distant from all existing centroids:

where τ is set to the q -quantile of pairwise distances among style vectors from the source domain, ensuring that new clusters are formed only under substantial domain shifts.

To adapt the centroids over time, we optimize a mutual information loss L MI between past style vectors and the current centroids. Storing all past styles is infeasible, so we maintain a fixed-size Style Reservoir R t := [ s t 1 , . . . , s t M ] , updated via Reservoir Sampling [39, 4] to approximate uniform sampling over all previously seen s t . At each step, s t is added to R t -1 if t ≤ M ; otherwise, it replaces a randomly selected vector with probability M t , ensuring unbiased coverage.

We define the soft-assignment for each style vector in the style reservoir to the centroids as follows:

The centroids C t are updated by gradient descent on the mutual information loss:

where

Here, L ent ( Q ) = -1 M ∑ M i =1 ∑ K j =1 q ij log q ij encourages confident assignments of style vectors in R t to the nearest style centroids, while L cm ( Q ) = ∑ K j =1 ¯ q j log ¯ q j (with ¯ q j = 1 M ∑ M i =1 q ij ) acts to prevent centroid collapse by promoting assignment diversity. Appendix E presents the sensitivity of all introduced hyperparameters, alternative online clustering strategies, and alternative distance metrics, including cosine similarity and Kullback-Leibler (KL) divergence, as well as the style reservoir update.

4.2 Model Reservoir: Initialization, Adaptation, and Prediction

Building on the theoretical analysis of parameter drift (Section 3.2), we maintain a Model Reservoir comprising K t domain-specialized models { θ 1 t , . . . , θ K t t } , with one model per discovered domain up to time t . To ensure computational efficiency, we store only the trainable parameters of each model, rather than full model instances. The total number of domains is bounded by a fixed constant K max to prevent memory exhaustion. At initialization, the reservoir contains a single model { θ 1 0 } corresponding to the source domain. When a new domain is detected, we instantiate a new model by cloning the parameters of an existing reservoir model that yields the most confident and diverse predictions on the current test batch B t . This is formalized using mutual information as follows:

where f θ ( B t ) ∈ R b ×| Y | is the softmax output of the model. This criterion favors models whose predictions are simultaneously confident and diverse, reducing the risk of collapse when adapting to a novel domain (see alternative initialization in Appendix F).

Model Adaptation. Given a test batch B t , we compute a soft assignment vector q t ∈ R K t by comparing the current style vector s t to domain centroids { c 1 t , . . . , c K t t } using scaled negative squared Euclidean distances followed by a softmax :

The most relevant model is then selected as k ∗ = arg max 1 ≤ j ≤ K [ q t ] j . The selected model θ k ∗ t is adapted using a test-time adaptation objective L TTA:

Model Prediction. For inference, the trainable parameters from all reservoir models are ensembled according to their soft assignment weights, yielding the ensemble parameters:

The final prediction is then computed as ˆ Y t = f ¯ θ t ( B t ) . Notably, ¯ θ t is used solely for prediction and is not updated, ensuring that all specialized models contribute to the output without being overwritten.

5 Experiments

Datasets and Evaluation Metrics. We evaluate on standard scene-level corruption benchmarks-CIFAR-10 → CIFAR-10-C, CIFAR-100 → CIFAR-100-C, and ImageNet → ImageNetC-using three CNN backbones with only batch/group norms updated; for ImageNet-C, we also test ViT-B/16 (see Appendix H). To demonstrate shift-agnosticism, we further assess object-level style-shift benchmarks, DomainNet-126 [30, 33] and PACS [22], using a ResNet-50 following [2]. For segmentation, we use Segformer-B5 as in CoTTA [42]. Classification is tested under CCC [31], CSC, and CDC settings over 20 rounds (averaging error rates, %; a subset is shown for clarity). For segmentation, we follow the Cityscapes → ACDC protocol [42], where ACDC presents four weather conditions (Fog, Night, Rain, Snow) sequentially. We report the mean IoU (%) averaged over 10 repetitions.

Baselines. We evaluate our method against several state-of-the-art TTA baselines (for more details, see the Appendix G). For single-target TTA , we compare with TENT [40]. For continual TTA , we consider CoTTA [42] (using the affine-parameter variant, CoTTA * ), RoTTA [45], ETA [27], and SAR [28]. To assess long-term stability, we include persistent TTA methods such as RDumb [31], PeTTA [12], EATA [27] (with Fisher-based regularization), and ROID [24] (reported as ROID * -a version omitting the augmentation consistency loss). For ViT-based models, we also compare with domain-disentanglement approaches like CoLA [3] and DPCore [47]. In segmentation experiments, we evaluate segmentation variants of TENT, CoTTA * , and BECoTTA [20]. For fair comparison, all methods update only the backbone's affine parameters-except when we compare ReservoirTTA and BECoTTA (using LoRA) and DPcore (using visual prompts).

Methods Tested for ReservoirTTA Plug-in. Based on our theoretical analysis, we integrate ReservoirTTA only into TTA methods that employ variance reduction via sample filtering along with variance control through weight ensembling and Fisher regularization. Accordingly, we apply ReservoirTTA to EATA and ROID ∗ , which incorporate both components, as well as to ETA and SAR, which use sample filtering alone, and to TENT even though it lacks both. For segmentation tasks, we plug ReservoirTTA into CoTTA ∗ and BECoTTA to demonstrate its generality. Although methods such as RoTTA and PeTTA are compatible in principle, they require a separate memory bank for each reservoir model which makes their practical integration computationally prohibitive.

Implementation Details. All methods are re-implemented in PyTorch [29] within a unified TTA repository [24] for fair comparison, using pre-trained source models from RobustBench [11]. See Appendix H for further implementation details.

5.1 Main Results

Classification. Table 1 highlights the limitations of existing TTA methods when repeatedly adapted under recurring domain shifts and prolonged testing with three CNN-based backbones. (1) Singletarget TTA : Methods like TENT initially adapt well but suffer from severe error accumulation over multiple cycles-for example, on CIFAR-10-C, error rises from 19.3% at visit 1 to 87.8% at visit 20, with similar trends on CIFAR-100-C and ImageNet-C. Plugging in ReservoirTTA significantly improves performance, though error accumulation still occurs, highlighting the need for both variance control modules from our theoretical analysis. (2) Continual TTA : Approaches like CoTTA ∗ and RoTTA mitigate short-term instability yet still accumulate errors with repeated adaptation. ReservoirTTA decouples domain-specific adaptation to prevent parameter drift-reducing, for instance, ETA's CIFAR-10-C error from 30.9% to 16.4% at the 20th visit (Recurring CSC) and similarly improving ImageNet-C performance. (3) Persistent TTA : Although methods such as EATA and ROID ∗ are designed to prevent catastrophic forgetting, they exhibit limited re-adaptation (Figure 1), with minimal improvements over repeated visits. ReservoirTTA overcomes this by enabling controlled adaptation without excessive re-learning-reducing EATA's ImageNet-C error from 55.9% to 51.0% in Recurring CSC and ROID's from 55.5% to 52.1%.

As shown in Table 2, using a ViT-B-16 backbone on ImageNet-C further underscores ReservoirTTA's advantages in recurrent continual TTA. Domain-disentangled methods like DPCore improve early adaptation but lack long-term stability. For instance, in Recurring CSC, CoLA reduces ETA's 20thvisit error to 33.8%, whereas ReservoirTTA lowers it further to 31.9%. In CCC, CoLA stabilizes ETA at 40.2%, whereas ReservoirTTA achieves 38.5%. DPCore also struggles: in CCC, its error rises from 42.2% to 43.1%, whereas combining ReservoirTTA with the same prompt-tuning approach

(VPT) [14]-used by DPCore for fair comparison-keeps the error at 39.2%. This gap shows that the domain identification mechanism in ReservoirTTA outperforms those in CoLA and DPCore by avoiding gradual degradation and accelerating performance improvement. As shown in Appendix D, our style features form higher-quality clusters than the ViT-based features used in CoLA and DPCore. Moreover, our model discovery mechanism is less sensitive to test batch order and estimates the number of domains more accurately (see Figure 4 and Appendix I), resulting in fewer trainable parameters compared to DPCore and COLA. See Tables 12-25 in Appendix K for full results across all 20 recurrences, CNN backbones (group norm), DomainNet-126, and PACS.

Segmentation. Table 3 shows segmentation results on Cityscapes → ACDC under recurring CSC. Methods such as TENT, CoTTA ∗ , and BECoTTA suffer mIoU declines when re-encountering each domain, indicating limited re-adaptation. By contrast, plugging in ReservoirTTA consistently boosts mIoU and limits performance drift. For example, TENT + ReservoirTTA gains over two percentage points on Snow by the 10th revisit, while CoTTA ∗ + ReservoirTTA and LORA + ReservoirTTA show steady improvements across all conditions compared to BECoTTA. This demonstrates that our domain-specific reservoir effectively preserves and reapplies learned knowledge over multiple visits (visit Appendix J, Figure 18 for segmentation visualizations).

2

4

8

16 32 64

K

28

29

30

Error Rate (%)

Recurring CSC

2

4

8

16 32 64

K

28

29

30

Recurring CDC

ROID

*

+ReservoirTTA

EATA+ReservoirTTA

5.2 Analyses and Ablations

Component-wise Ablation Analysis and Runtime. Table 4 quantifies the impact of ReservoirTTA's components on CIFAR-100-C (CSC, CDC) and ImageNet-C (CCC), along with runtime relative to EATA. Incorporating the Model Reservoir (MR) enables domain-specific adaptation, yielding a 1.9% gain at visit 20 in CSC. The Style Reservoir (SR) has limited effect on CSC/CDC-frequent, abrupt domain resets make styles easily separable, so a single embedding suffices-but helps in CCC, where gradual drift benefits from accumulated history (+1.2%). EMR ensembling is marginal on CSC/CDC (assignments are near one-hot) yet gives +1.5% on CCC by leveraging multiple models. As shown in Appendix I, reusing specialists via MR outperforms simple weight resets: RDumb's blind reset 32.2% → domain-aware reset 31.2% → MR28.4% (CSC, visit 20). Overall, ReservoirTTA improves accuracy by +2.1% (CSC), +2.7% (CDC), and +1.5% (CCC) at only 1 . 3 × EATA's runtime.

Sensitivity Analysis of ReservoirTTA Hyperparameters. Figure 3 highlights the importance of setting K max , with classification error stabilizing for K max ≥ 16 on CIFAR-100-C consistent with its domain structure. Overestimating the number of domains helps avoid premature merging and improves specialization. Moreover, as shown in Figure 4, ReservoirTTA is robust to batchorder variability, reliably estimating the number of domains after the first recurrence in the CSC setting even without constraining K max . Across subsequent recurrences, the detected domain count increases marginally, consistent with the stable performance reported above. Additional ablations on CIFAR-100-C and CCC (see Appendix E) show that ReservoirTTA is robust to key hyperparameters: performance remains stable across style reservoir sizes M , and variations in source sample count or quantile q th for threshold τ in Equation (7) affect error by less than 1%. As shown in Appendix I, larger batch sizes improve performance, but ReservoirTTA consistently outperforms baselines across

Gauss

Shot

Impulse

Defocus

Motion

Zoom

Snow

Frost

Fog

Bright

Contrast

Elastic

JPEG

2

4

6

8

10

12

14

Number of Detected Domains

Noise

Blur

Weather

Digital

ReservoirTTA: # of detected domains

during the 1st visit

1 2 3 4 5 6 7 8 91011121314151617181920

12

13

14

15

16

17

18

19

Number of Detected Domains

ReservoirTTA: # of detected domains

across 20 visits

Seed 1

Seed 100

Seed 1000

Seed 200

Seed 2000

settings. Notably, it requires less reliance on weight ensembling or Fisher regularization, indicating strong inherent variance control.

Additional Analysis. We further test two challenging TTA settings: (i) gradual shifts with smoothly varying severity, where ReservoirTTA adapts well to subtle transitions, and (ii) temporally correlated streams with category bias, where pairing with sampling/stabilization mitigates bias. Results (Appendix K) show effectiveness across diverse, realistic test-time scenarios.

6 Conclusion

We present ReservoirTTA as a novel framework to extend test-time adaptation from single model to multiple model adaptation by decoupling updates across a reservoir of domain-specialized models. Rather than forcing a single model to adapt continuously, our approach selectively updates the most relevant component based on dynamic clustering of style features. Furthermore our approach identifies, assigns, and updates its specialists fully at test time without needing multiple source domains for training. This design not only stabilizes the adaptation process but also curtails the accumulation of errors and catastrophic forgetting that typically plague single-model methods. Our theoretical analysis and comprehensive experiments underscore the framework's ability to maintain robust performance even under prolonged and unpredictable domain shifts.

Limitations. Our plug-in approach improves adaptation by decoupling domain-specific updates but introduces additional computational overhead due to the Model Reservoir, online clustering, and refinement using a Style Reservoir. This can increase computation by up to 30% on top of lightweight methods such as EATA and ETA. However, the memory overhead remains low, as only trainable parameters and not full models are duplicated for the Model Reservoir.

A further limitation-common to most TTA methods-is updating parameters on every incoming batch regardless of convergence. Introducing an adaptive update trigger (to switch optimization on/off) could markedly reduce runtime overhead and improve practicality in resource-constrained settings.

References

Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsupervised learning of visual features by contrasting cluster assignments. Advances in neural information processing systems, 33:9912-9924, 2020.

Dian Chen, Dequan Wang, Trevor Darrell, and Sayna Ebrahimi. Contrastive test-time adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 295-305, 2022.

Guohao Chen, Shuaicheng Niu, Deyu Chen, Shuhai Zhang, Changsheng Li, Yuanqing Li, and Mingkui Tan. Cross-device collaborative test-time adaptation. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.

Aristotelis Chrysakis and Marie-Francine Moens. Online continual learning from imbalanced data. In International Conference on Machine Learning, pages 1952-1961. PMLR, 2020.

Chaoran Cui, Yongrui Zhen, Shuai Gong, Chunyun Zhang, Hui Liu, and Yilong Yin. Dynamic prompt allocation and tuning for continual test-time adaptation. arXiv preprint arXiv:2412.09308, 2024.

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. ICLR, 2021.

Yulu Gan, Yan Bai, Yihang Lou, Xianzheng Ma, Renrui Zhang, Nian Shi, and Lin Luo. Decorate the newcomers: Visual domain prompt for continual test time adaptation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 7595-7603, 2023.

Yunhe Gao, Xingjian Shi, Yi Zhu, Hao Wang, Zhiqiang Tang, Xiong Zhou, Mu Li, and Dimitris N Metaxas. Visual prompt tuning for test-time domain adaptation. arXiv preprint arXiv:2210.04831, 2022.

Leon A Gatys, Alexander S Ecker, and Matthias Bethge. Image style transfer using convolutional neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2414-2423, 2016.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.

Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. Proceedings of the International Conference on Learning Representations, 2019.

Trung-Hieu Hoang, Duc Minh Vo, and Minh N. Do. Persistent test-time adaptation in recurring testing scenarios. In Thirty-eighth Conference on Neural Information Processing Systems (NeurIPS), 2024.

Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022.

Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim. Visual prompt tuning. In European conference on computer vision, pages 709-727. Springer, 2022.

Justin Johnson, Alexandre Alahi, and Li Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. In Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14, pages 694-711. Springer, 2016.

Byungjai Kim, Chanho Ahn, Wissam J Baddar, Kikyung Kim, Huijin Lee, Saehyun Ahn, Seungju Han, Sungjoo Suh, and Eunho Yang. Test-time ensemble via linear mode connectivity: A path to better adaptation. In The Thirteenth International Conference on Learning Representations, 2025.

Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.

James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521-3526, 2017.

Brian Kulis and Michael Jordan. Revisiting k-means: New algorithms via bayesian nonparametrics. In John Langford and Joelle Pineau, editors, Proceedings of the 29th International Conference on Machine Learning (ICML-12), ICML '12, pages 513-520, New York, NY, USA, July 2012. Omnipress.

Daeun Lee, Jaehong Yoon, and Sung Ju Hwang. Becotta: Input-dependent online blending of experts for continual test-time adaptation. In International Conference on Machine Learning, 2024.

Jungsoo Lee, Debasmit Das, Jaegul Choo, and Sungha Choi. Towards open-set test-time adaptation utilizing the wisdom of crowds in entropy minimization. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 16380-16389, 2023.

Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M Hospedales. Deeper, broader and artier domain generalization. In Proceedings of the IEEE international conference on computer vision, pages 5542-5550, 2017.

Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations, 2019.

Robert A Marsden, Mario Döbler, and Bin Yang. Universal test-time adaptation through weight ensembling, diversity weighting, and prior correction. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 2555-2565, 2024.

Chaithanya Kumar Mummadi, Robin Hutmacher, Kilian Rambach, Evgeny Levinkov, Thomas Brox, and Jan Hendrik Metzen. Test-time adaptation to distribution shift by confidence maximization and input transformation. arXiv preprint arXiv:2106.14999, 2021.

Shuaicheng Niu, Chunyan Miao, Guohao Chen, Pengcheng Wu, and Peilin Zhao. Test-time model adaptation with only forward passes. In The International Conference on Machine Learning, 2024.

Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Yaofo Chen, Shijian Zheng, Peilin Zhao, and Mingkui Tan. Efficient test-time model adaptation without forgetting. In The International Conference on Machine Learning, 2022.

Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Zhiquan Wen, Yaofo Chen, Peilin Zhao, and Mingkui Tan. Towards stable test-time adaptation in dynamic wild world. In International Conference on Learning Representations, 2023.

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.

Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. Moment matching for multi-source domain adaptation. In Proceedings of the IEEE/CVF international conference on computer vision, pages 1406-1415, 2019.

Ori Press, Steffen Schneider, Matthias Kümmerer, and Matthias Bethge. Rdumb: A simple approach that questions our progress in continual test-time adaptation. Advances in Neural Information Processing Systems, 36, 2024.

Herbert Robbins and Sutton Monro. A stochastic approximation method. The annals of mathematical statistics, pages 400-407, 1951.

Kuniaki Saito, Donghyun Kim, Stan Sclaroff, Trevor Darrell, and Kate Saenko. Semi-supervised domain adaptation via minimax entropy. In Proceedings of the IEEE/CVF international conference on computer vision, pages 8050-8058, 2019.

Christos Sakaridis, Dengxin Dai, and Luc Van Gool. Acdc: The adverse conditions dataset with correspondences for semantic driving scene understanding. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10765-10775, 2021.

Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.

Junha Song, Jungsoo Lee, In So Kweon, and Sungha Choi. Ecotta: Memory-efficient continual test-time adaptation via self-distilled regularization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11920-11929, 2023.

Devavrat Tomar, Guillaume Vray, Behzad Bozorgtabar, and Jean-Philippe Thiran. Tesla: Testtime self-learning with automatic adversarial augmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 20341-20350, 2023.

Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008.

Jeffrey S Vitter. Random sampling with a reservoir. ACM Transactions on Mathematical Software (TOMS), 11(1):37-57, 1985.

Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. In International Conference on Learning Representations, 2021.

Pei Wang, Yijun Li, and Nuno Vasconcelos. Rethinking and improving the robustness of image style transfer. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 124-133, 2021.

Qin Wang, Olga Fink, Luc Van Gool, and Dengxin Dai. Continual test-time domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7201-7211, 2022.

Yuxin Wu and Kaiming He. Group normalization. In Proceedings of the European conference on computer vision (ECCV), pages 3-19, 2018.

Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. Aggregated residual transformations for deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1492-1500, 2017.

Longhui Yuan, Binhui Xie, and Shuang Li. Robust test-time adaptation in dynamic scenarios. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15922-15932, 2023.

Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In Edwin R. Hancock Richard C. Wilson and William A. P. Smith, editors, Proceedings of the British Machine Vision Conference (BMVC), pages 87.1-87.12. BMVA Press, September 2016.

Yunbei Zhang, Akshay Mehra, Shuaicheng Niu, and Jihun Hamm. Dpcore: Dynamic prompt coreset for continual test-time adaptation. arXiv preprint arXiv:2406.10737, 2024.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: Our abstract and introduction summarize our contributions with experimental results for persistent test time adaptation. All the claims are provided in detail in the papers' corresponding sections, including the appendix.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We discuss the limitations of our approach in the Conclusion section. The primary limitation lies in the increased complexity introduced by our method, which is analyzed in the corresponding section.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: Each theorem introduced in Section 3.2 is proved in Appendix B. All assumptions are also provided in Section 3.2. Each assumption, theorem, lemma, and proposition in Section 3.2 and Appendix B is numbered for clarity.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: The method and all associated hyperparameters are thoroughly described in the Method section. We also include a pseudocode in Appendix C, which accurately reflects our algorithm. The specific hyperparameter values used in our experiments are provided in Appendix H.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [No]

Justification: The data is publicly available (see Appendix H). However, the code will be released upon acceptance.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: All hyperparameters, optimizers, and model architectures used are described in detail in Section 5 and Appendix H.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: All reported results are averaged over 5 random seeds. Our main claims are supported by Figure 1, which includes confidence intervals.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: Information on computing resources are provided in Appendix H.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: Our work relies exclusively on publicly available image benchmarks, involves no human subjects or sensitive data, and presents no foreseeable misuse; it fully adheres to the NeurIPS ethical guidelines.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: This work focuses on a general formulation of recurring and persistent testtime adaptation, without targeting any specific downstream application. As such, we do not anticipate any direct societal impact resulting from this research beyond those shared by machine learning in general.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: We do not believe out paper poses such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We properly cited the papers that provided some of the code in Section 5. The corresponding licenses and links to used assets are also provided in Appendix G and Appendix H.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: This work does not release new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: This work does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: This work does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.

Answer: [NA]

Justification: The core method development in this research does not involve LLMs.

Guidelines:

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

Please refer to our LLM policy ( https://neurips.cc/Conferences/2025/LLM ) for what should or should not be described.

Appendix

In this appendix, we provide further details on related work (Appendix A), our theoretical analysis (Appendix B), our algorithm (Appendix C), style features quality (Appendix D), domain identification via online clustering (Appendix E), model reservoir initialization (Appendix F), baselines (Appendix G), implementation details (Appendix H), additional ablation studies (Appendix I), as well as extra qualitative (Appendix J) and quantitative results (Appendix K). Note: The numbering of tables, figures, and equations in this appendix continues from the main document.

A Related Work

Continual Test-Time Adaptation. Early TTA approaches adapt pre-trained models to a fixed, stationary target domain by updating only a small subset of parameters-typically the affine parameters of BatchNorm-using techniques such as pseudolabeling or entropy minimization [40, 25]. However, these single-target TTA methods are prone to error accumulation when faced with prolonged or recurring continual domain shifts. To address these limitations, continual TTA (CTTA) methods (e.g., CoTTA [42], RoTTA [45], and EcoTTA [36]) employ teacher-student frameworks, self-distillation, and regularization techniques to improve robustness in adapting pre-trained models across evolving target domains. Nevertheless, most CTTA methods update a single shared model across domains, leading to slow convergence on brief exposures, catastrophic forgetting, and negative transfer when domains differ. Although they can reduce short-term performance fluctuations, these approaches remain prone to cumulative errors over time and assume each environment appears only once-a condition that rarely holds in practice. Recent works propose distinct paradigms. SAR [28] uses sharpness-aware entropy minimization to stabilize adaptation by suppressing noisy gradients. BECoTTA [20] updates domain-specific low-rank experts through adaptive routing, reducing interference but relies on access to multi-source data before TTA deployment. CoLA [3] enables collaborative test-time adaptation by sharing domain knowledge vectors across devices to improve efficiency, though its success depends on stable inter-device communication in resource-constrained environments. An alternative approach, known as prompt-based TTA , leverages visual prompt-based learning [47, 5] to enable domain-specific adaptation without altering the core network. Methods such as DPCore [47], VDP [7], DePT [8], and PAINT [5] employ learnable tokens or pixel-level prompts, though they typically require a warm-up phase to compute alignment statistics and reliably initialize the prompts.

Robust and Persistent Test-Time Adaptation. Persistent TTA approaches are designed to sustain robust performance over extended periods. Techniques such as active sample selection, Fisher-based regularization, and weight ensembling have been explored in methods like EATA [27] and ROID [24] to prevent catastrophic forgetting and model collapse. Other approaches, such as reset-based methods like RDumb [31], use teacher-student updates or periodic resets to counteract adaptation error accumulation. Similarly, PeTTA [12] dynamically adjusts its adaptation strategy to prevent model collapse without frequent resets. In parallel, Test-Time Ensemble (TTE) [16] leverages linear mode connectivity to form an adaptive weight-space ensemble (EMA modulated by a divergence term) together with a dropout ensemble, and applies de-biased reverse-KL distillation to stabilize updates; TTE plugs into standard TTA objectives and improves robustness on continual, non-i.i.d. streams. However, these strategies often exhibit limited re-adaptation capability when encountering previously seen distributions and require precise hyperparameter tuning or source data for initialization. In contrast, our ReservoirTTA is designed to maintain consistent long-term adaptation performance without these constraints.

B Details on Theoretical Analysis

B.1 TTA Variance Bound via Source Weighted Ensembling and Fisher Regularization

Theorem 2 (Recursive Weight Ensembling Update) . Let the update rule be defined as

with initial parameter θ 0 ∈ R d , step size η > 0 , and ensembling parameter α ∈ [0 , 1] . Then for all t ≥ 1 , the iterates θ t admit the closed-form recursion

Proof. We proceed by induction on t .

Base Case ( t = 1 ). From (15), we have:

Simplifying:

which matches (17) for t = 1 .

Inductive Step. Assume the result holds for some t ≥ 1 , i.e.,

Using (16):

Then by (15) and (16):

Substituting:

Now apply the inductive hypothesis:

which completes the inductive step.

Conclusion. By mathematical induction, Equation (17) holds for all t ≥ 1 .

Proposition 3 (Bounded Variance under Source-Weighted Ensembling) . Assume that the per-step gradient noise has average variance ¯ V , i.e.,

Then, under the weight ensembling update in (15) -(16) , the variance of θ t satisfies

Proof. From Equation (17), the update rule for θ t is:

Assuming independent gradient noise across time steps with constant average variance ¯ V , the variance of the sum is:

Finally, taking the limit as t → ∞ (and assuming α < 1 ), we observe that α 2 t → 0 , yielding the upper bound:

Thus, the variance remains bounded uniformly in t .

Additionally, consider the limit α → 1 . In this case, the denominator approaches 0 and we can expand the geometric sum:

which recovers the standard linear-int variance growth:

This confirms that weight ensembling curtails variance growth over time compared to unregularized adaptation.

Proposition 4 (Fisher Regularization as Weighted Ensembling) . Let the TTA objective be augmented with Fisher regularization as follows:

where λ > 0 is the regularization coefficient and ω is the (diagonal) Fisher information weight. Then the gradient descent update becomes:

Define α = 1 -2 λωη . Then this update is equivalent to the weighted ensembling rule:

Proof. Starting from the Fisher-regularized objective, the gradient is:

Substituting into the gradient update rule:

which simplifies to:

Letting α = 1 -2 λωη , we get:

Now define ˆ θ t = θ t -1 -η ∇L TTA ( θ t -1 , x t -1 ) , and substitute:

which matches the ensembling formulation in Equation (16).

Remark 1 (Bias-Variance Tradeoff) . The source-weighted ensembling update controls the variance of θ t over time, as shown in Equation (18) , but introduces bias toward the source model θ 0 . In the extreme case α = 0 , no adaptation occurs, resulting in maximal bias. As α → 1 , the method recovers standard SGD, minimizing bias but allowing unbounded variance. In practice, setting α close to 1 balances low variance with limited bias.

Proof of Theorem 1 (Bound on Divergence Probability). Let θ t denote the model parameters at time t , and let θ ∗ Task be the task-specific optimum. Suppose that E [ θ t ] → θ ∗ Task and ∥ E [ θ t ] -θ ∗ Task ∥ < ∥ θ 0 -θ ∗ Task ∥ . Then for any threshold β > ∥ θ 0 -θ ∗ Task ∥ , the probability of divergence from the stability region is bounded by:

Proof. We begin by decomposing the distance between θ t and θ ∗ Task :

By the assumption that ∥ E [ θ t ] -θ ∗ Task ∥ < ∥ θ 0 -θ ∗ Task ∥ , we have:

Now apply Chebyshev's inequality:

This completes the proof.

B.2 Comparison of Single Model TTA and Model Reservoir TTA

In Figure 5, we compare the adaptation trajectories of a single model TTA approach and our proposed Model Reservoir TTA framework under recurring continual domain shifts. Consider an example with three domains, D 1 , D 2 , and D 3 , each associated with stability radii β 1 , β 2 , and β 3 , respectively. Let θ ∗ Task i = arg min θ L Task i ( θ ) for i = 1 , 2 , 3 denote the task-optimal parameters that minimize the latent task loss in each domain. If the shift between optimal parameters exceeds the stability radius for a transition,

then adaptation on D 1 yields parameters near θ ∗ Task 1 , but a subsequent shift to D 2 may cause the adapted parameters to drift outside the stability region of D 2 . This drift is compounded in a recurring continual TTA setting, where the test stream follows D 1 → D 2 → D 3 ↬ D 1 . The left panel of Figure 5 illustrates that even with sample filtering and weight ensembling, a single model TTA approach accumulates error at each domain transition and ultimately fails to re-adapt properly (i.e., the trajectory drifts away from the D 1 optimum, converging instead near the source model). In contrast, the right panel demonstrates that our Model Reservoir TTA framework maintains separate, domain-specific trajectories that remain bounded within their respective stability regions, thereby enabling efficient re-adaptation when a previously encountered domain reoccurs.

C Algorithm of ReservoirTTA

For clarity, we detail the complete ReservoirTTA workflow in Algorithm 1.

D Style Features Quality

In this section, we comprehensively evaluate the quality of style feature representations for ReservoirTTA by conducting quantitative ablation studies on VGG19 layer configurations and various style feature operations, and by presenting qualitative t-SNE [38] visualizations of the extracted features.

VGG-19 Configuration: Shallow vs. Deep. Table 5 reports an ablation study on the impact of various VGG19 layer configurations for the extraction of style features. VGG19 is organized into four hierarchical blocks from which we extract style features at specific layers: Shallow ( [2 , 5 , 7] ), Middle1 ( [10 , 12 , 14 , 16] ), Middle-2 ( [19 , 21 , 23 , 25] ), and Deep ( [28 , 30 , 32 , 34] ), where the indices denote

*

T

1

*

T

2

*

T

3

1

2

3

Drift starts

Accumulated Error

Re­adaptation Failure

Single Model TTA

*

T

1

*

T

2

*

T

3

1

2

3

Switch: D1

D2

Switch: D2

D3

Switch: D3

D1

Model Reservoir TTA

Source Model

Single Model Trajectory

Res. Traj. D1 (Recur)

the corresponding layers in the network. Additionally, we consider mixed configurations that integrate layers across different depths: Mixed-1 (Shallow + Middle-1), Mixed-2 (Deep + Middle-2), Mixed-3 (Shallow + Middle-1 + Middle-2 + Deep), and Mixed-4 (Shallow + Deep). We evaluated the four main configurations above (Shallow, Middle-1, Middle-2, Deep) along with several mixed configurations on three scenarios: recurring CSC and CDC on CIFAR100-C and CCC on ImageNet-C. The low standard deviations in the average error rates confirm that ReservoirTTA is robust to the choice of intermediate VGG19 layers. Table 6 reports the ablation study on style feature representations in ReservoirTTA. The average classification error rates (%) for five different operationsmean , var , logvar , [mean, var] , and gram -are evaluated. Notably, our logvar -based style representation delivers superior performance compared to the other operations.

Style Extractor Choice: VGG-19 vs. Source. We investigate whether using the source model for style feature extraction could replace the ImageNet-trained VGG. Following this suggestion, we replaced VGG with early layers of the source backbone for style calculation. On CIFAR100-C

Algorithm 1 ReservoirTTA: Prolonged Test-Time Adaptation

Require: Pre-trained source model f θ 0 ; maximum domains K max ; style reservoir size M ; new-domain threshold τ (initialized using source examples); TTA objective L TTA ( · )

1 K max

(CSC) with a ResNeXt-29 backbone, this swap increases mean error by +2.12% (Table 7). Source backbones lack the broad texture priors learned from ImageNet and would make our approach architecture-specific. By contrast, VGG delivers consistent gains across ResNeXt-29, ViT-B-16, and ResNet-50 (GN) (see Appendix K). These results highlight that the VGG-19 style extractor with the automatically chosen threshold τ introduces minimal, dataset-agnostic overhead while providing substantially better domain detection and adaptation performance.

Style Features Quality. Figure 6a presents the t-SNE visualization of style features extracted from three corruption benchmarks-CIFAR-10-C, CIFAR-100-C, and ImageNet-C-using our ReservoirTTA method. The datasets cover 15 distinct domains, which we organize into four categories: Noise (Gaussian Noise, Shot Noise, Impulse Noise), Blur (Defocus Blur, Glass Blur, Motion Blur, Zoom Blur), Weather (Snow, Frost, Fog, Brightness), and Digital (Contrast, Elastic, Pixelate, JPEG). As observed, samples from the same domain form well-defined clusters, while features from different domains remain clearly separated. This result confirms that style features effectively capture the inherent domain differences in these challenging corruption scenarios. In Figure 6b, we compare

the feature distributions produced by our method, ReservoirTTA, against two baseline approaches, DPCore and CoLA, under the recurring CSC setting in the CIFAR-100-C dataset. We show the evolution of the t-SNE plots at the first, 10th, and 20th visits, illustrating how the features adapt as domain shifts accumulate over time. Our method consistently maintains well-separated and dense clusters throughout the adaptation process, while the baselines exhibit less distinct clustering. This comparison further demonstrates the effectiveness of ReservoirTTA in maintaining domain distinctions even in evolving environments.

E Domain Identification via Online Clustering

Effect of Style Reservoir Capacity. The capacity of style reservoir M determines how many style features are used to optimize the style centroids in Equation (9). As shown in Figure 7, in both recurring CSC and CDC settings, performance remains stable across different values of M , with low sensitivity to changes in this parameter. This suggests that even with an extremely small style reservoir size (e.g., M = 1 ), our style-based domain assignment optimization remains robust and efficient, without incurring significant memory overhead. In CCC, increasing M leads to a reduction

Domain Labels

Defocus Blur

Fog

Contrast

Pixelate

Snow

Brightness

Elastic

JPEG

Impulse Noise

Frost

CIFAR-10-C

CIFAR-100-C

ImageNet-C

DPCore

DPCore: after 1st visit

DPCore: after 10th visit

DPCore: after 20th visit

CoLA

CoLA: after 1st visit

CoLA: after 10th visit

CoLA: after 20th visit

ReservoirTTA

ReservoirTTA: after 1st visit

ReservoirTTA: after 10th visit

ReservoirTTA: after 20th visit

Domain Labels

Impulse Noise

Defocus Blur

Snow

Frost

Fog

Brightness

Contrast

Elastic

Pixelate

JPEG

2 0

2 3

2 4

2 5

2 6

2 8

2 10

2 12

M

27.8

28.0

28.2

28.4

Error Rate (%)

Recurring CSC

2 0

2 3

2 4

2 5

2 6

2 8

2 10

2 12

M

27.8

28.0

28.2

28.4

Recurring CDC

2 0

2 3

2 4

2 5

2 6

2 8

2 10

2 12

M

58.8

59.0

59.2

59.4

59.6

59.8

60.0

CCC

ROID*+ReservoirTTA

EATA+ReservoirTTA

300

500

1000 2000 500010000

Number of Source Samples

27.8

28.0

28.2

28.4

28.6

Error Rate (%)

Recurring CSC

300

500

1000 2000 500010000

Number of Source Samples

27.8

28.0

28.2

28.4

28.6

Recurring CDC

300

500

1000 2000 500010000

Number of Source Samples

59.0

59.2

59.4

59.6

59.8

60.0

60.2

CCC

ROID*+ReservoirTTA

EATA+ReservoirTTA

in errors, confirming our ablation study analysis that a larger style reservoir allows for a more comprehensive observation of domain shifts, thereby improving performance. We set M = 1024 for all main experiments.

Effect of Number of Source Samples and Quantile Value. In Figure 8, we vary the number of unlabeled source samples from 300 to 10,000. For ROID * +ReservoirTTA, the error remains nearly constant, while for EATA+ReservoirTTA a slight decrease in error is observed with more source samples. In addition, minor fluctuations-especially in the recurring CSC and CDC settings.

On the choice of the quantile threshold q th: lowering q th increases sensitivity-detecting smaller shifts-but also raises false positives and spawns unnecessary domains; higher q th is more conservative and can delay detection of genuinely new domains. In a single sweep over the 15 CIFAR-100-C corruptions, q th = 0 . 5 severely overestimates (detects > 100 domains), whereas q th = 1 . 0 merges similar corruptions-[zoom, motion, glass blur] and [Gaussian, shot, impulse noise]-yielding 9 domains. Intermediate thresholds balance sensitivity and precision: q th = 0 . 99 identifies 17 domains and q th = 0 . 999 identifies 14. Importantly, Figure 9 shows that varying q th leads to only minor changes in error. Overall, our style-based domain identifier is robust to both the quantile setting and the number of source samples, with minimal impact on performance.

Ablation on Online Clustering Strategies. Table 8 compares different online clustering strategies (Sinkhorn-Knopp [1], Online K-Means, and our adaptive clustering scheme used for reservoirTTA) when integrated into ROID ∗ as the baseline. We evaluate recurring CSC and CDC on CIFAR-100-C,

0.5

0.9

0.99

0.999

1.0

qth

quantile

27.8

28.0

28.2

28.4

28.6

28.8

Error Rate (%)

Recurring CSC

0.5

0.9

0.99

0.999

1.0

qth

quantile

27.8

28.0

28.2

28.4

28.6

28.8

29.0

Recurring CDC

0.5

0.9

0.99

0.999

1.0

qth

quantile

59.0

59.2

59.4

59.6

59.8

CCC

ROID*+ReservoirTTA

EATA+ReservoirTTA

and CCC on ImageNet-C, reporting average classification error rates. Across all settings, our method consistently achieves the lowest error. In particular, for recurring CSC, our approach shows the best performance at visits 10 and 20, highlighting accurate domain partitioning and model assignment over multiple rounds. Sinkhorn-Knopp and Online K-Means provide moderate improvements but still trail behind our reservoir-based strategy. These findings confirm that our online clustering mechanism effectively separates target domains and assigns them to specialized models, leading to superior long-term adaptation.

Ablation on Distance Metrics. We investigate alternative metrics for centroid assignment within EATA+ReservoirTTA. Euclidean distance aligns naturally with our quantile-based new-domain detector τ , yet cosine similarity and KL divergence might, in principle, offer advantages in highdimensional or streaming regimes. To assess this, we compare soft assignments using Euclidean distance, cosine similarity, and KL divergence on CIFAR-100-C (CSC) over 20 recurrences. As summarized in Table 9, KL divergence outperforms cosine similarity-supporting its suitability for online adaptation-while Euclidean distance computed on log-variance features consistently achieves the lowest error across recurrences. These findings validate our choice to retain Euclidean distance and provide further empirical evidence of the method's robustness, alongside a clear head-to-head comparison of distance metrics.

Model Assignments per Domain. Figure 10 displays a heatmap illustrating how each target domain is assigned to different models in ReservoirTTA. The rows represent 15 corruption types (e.g., Gauss, Shot, Impulse, Defocus), while the columns correspond to 15 model indices. Each cell [ i, j ] denotes the proportion of samples from the i -th domain that are routed to the j -th model, with row sums normalized to 1. We observe a clear one-to-one correspondence for most corruptions, though some exhibit similar style cues. Overall, the concentration along the diagonal reflects domain-specialized

adaptation, whereas smaller off-diagonal entries highlight moderate sharing among visually related domains. This confirms that ReservoirTTA's style-based clustering reliably assigns test samples to the most relevant domain-specific model.

0

1

2

3

4

5

6

7

8

9

10

11

12

13

14

Model Index

Gauss

Shot

Impulse

Defocus

Motion

Zoom

Snow

Frost

Fog

Bright

Contrast

Elastic

JPEG

Target Domain

0

1

0

0

0

0

0

0

0

0

0

0

0

0

0

0

1

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

1

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

1

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

1

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

1

0

0

0

0

0

0

0

0

0

0

0

0

1

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

1

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

1

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

1

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

1

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0.9

0.1

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

1

0

0

0

0

0

0

0.04

0

0

0

0

0

0

0

0

0.96

0

0.02

0

0

0

0

0

0

0

0

0

0

0

0

0

0.98

Model Assignments per Domain (Normalized)

0.0

0.2

0.4

0.6

0.8

1.0

Style Cluster Centroid Trajectories During Adaptation. Figure 11 shows the evolution (via trajectories) of style cluster centroids during adaptation. Each centroid is assigned a unique ID, which corresponds to the model index presented in Figure 10. As shown, each centroid remains closely linked to its assigned domain throughout adaptation, indicating that each model adapts exclusively to that domain.

FIFO vs. Reservoir Sampling for Domain Balancing. Figure 12 compares two sampling strategies-FIFO and Reservoir sampling-for maintaining a domain-balanced Style Reservoir in recurring CSC on CIFAR-100-C. It shows the distribution of different domains over time for two reservoir sizes ( M =256 and M =1024). The results reveal that Reservoir sampling produces a more uniform and stable domain distribution than FIFO, even as the reservoir size varies. These observations underscore the robustness of our approach and support the use of Reservoir sampling as the preferred update strategy.

0

1

2

3

4

5

6

7

8

9

10

11

12

13

14

Gauss

Shot

Impulse

Defocus

Motion

Zoom

Snow

Frost

Fog

Bright

Contrast

Elastic

JPEG

Start Position

End Position

0

2000

4000

6000

8000

10000

12000

14000

Adaptation Step

0.0

0.2

0.4

0.6

0.8

1.0

Domain Distribution

FIFO,

M

=1024

0

2000

4000

6000

8000

10000

12000

14000

Adaptation Step

0.0

0.2

0.4

0.6

0.8

1.0

Domain Distribution

FIFO,

M

=256

0

2000

4000

6000

8000

10000

12000

14000

Adaptation Step

0.0

0.2

0.4

0.6

0.8

1.0

Domain Distribution

Reservoir Sampling,

M

=1024

0

2000

4000

6000

8000

10000

12000

14000

Adaptation Step

0.0

0.2

0.4

0.6

0.8

1.0

Domain Distribution

Reservoir Sampling,

M

=256

Gauss

Shot

Impulse

Defocus

Motion

Zoom

Snow

Frost

Fog

Bright

Contrast

Elastic

JPEG

F Model Reservoir Initialization

In this section, we explore strategies for initializing a new model when a new domain is detected.

MI vs. Source Weight Initialization. As shown in Figure 13, we evaluate two initialization approaches: (1) initializing with source model parameters and (2) initializing with reservoir parameters that maximize mutual information (Equation (11)) on the current batch. We evaluate these on CIFAR100-C under recurring CSC and CDC settings (20 recurrences) using ResNeXt-29, and on CCC with ResNet-50 and ImageNet-C under recurring CSC and CDC settings (20 recurrences) using

Model Initialization

TENT

+Res.TTA

ETA

+Res.TTA

EATA

+Res.TTA

30.0

32.5

35.0

37.5

40.0

42.5

45.0

Error Rate (%)

CIFAR100-C CSC

(ResNeXt-29)

TENT

+Res.TTA

ETA

+Res.TTA

EATA

+Res.TTA

30.0

32.5

35.0

37.5

40.0

42.5

45.0

CIFAR100-C CDC

(ResNeXt-29)

TENT

+Res.TTA

ETA

+Res.TTA

EATA

+Res.TTA

55

60

65

70

75

80

85

CCC

(ResNet-50)

TENT

+Res.TTA

ETA

+Res.TTA

EATA

+Res.TTA

30.0

32.5

35.0

37.5

40.0

42.5

45.0

ImageNet-C CSC

(ViT-B-16)

TENT

+Res.TTA

ETA

+Res.TTA

EATA

+Res.TTA

30.0

32.5

35.0

37.5

40.0

42.5

45.0

ImageNet-C CDC

(ViT-B-16)

Source

Eq. (11)

a ViT-B-16. Our results show that the optimal strategy depends on both the method and the backbone. For convolutional networks, TENT and ETA benefit from source initialization-likely because, without a regularization module, drifting too far from the source degrades performance-while EATA is largely unaffected. Conversely, for ViT-B-16, initializing via Equation (11) significantly reduces error rates after 20 recurrences for TENT, ETA, and EATA, suggesting that the transformer is less prone to parameter drift and can better leverage the most similar model in the reservoir, whereas source initialization may overwrite valuable acquired knowledge.

MI vs Top-k Model Weight Ensembling Initialization. We assess the effect of topk ( k = 3 ) ensemble initialization strategy for EATA+ReservoirTTA. On CIFAR-100-C (CSC) with a CNN backbone, we observe that EATA remains unaffected by the choice of initialization, yielding nearly identical performance across source, topk , and MI initialization, consistent with our previous analysis. In contrast, on ImageNet-C with a ViT backbone, topk initialization improves performance by 3.37% compared to source initialization, but degrades performance over MI initialization (-0.27%).

G Baselines

Below is a summary of how the baseline TTA methods in our paper are categorized, along with a very short description of each method. Every baseline has been tested in a unified TTA repository 2 [24] (MIT license):

Single-Target TTA. TENT minimizes the entropy of prediction online.

Continual TTA. These methods are robust to continual TTA settings but may suffer performance degradation when run for a long time.

CoTTA * [42]: A variant of CoTTA that updates only affine parameters for a fair comparison. The CoTTA method combined weight averaging, predictions averaged over augmentations, and stochastic restoration within a mean-teacher framework. We employ the official mmsegmentation code provided by the authors 3 (MIT license).

RoTTA [45]: Incorporates robust adaptation and reinitialization mechanisms to counteract the negative effects of prolonged adaptation.

ETA [27]: Uses a sample-adaptive entropy minimization strategy that filters out test samples that are unreliable or redundant. This approach reduces the number of backward passes and error accumulation during test-time adaptation.

SAR [28]: Employs sharpness-aware entropy minimization to improve reliability by mitigating the influence of noisy gradients.

3. Persistent TTA. These methods are designed for long-term stability, preventing model collapse and catastrophic forgetting across repeated domain shifts.

RDumb [31]: Periodically resets model parameters to counteract error accumulation and avoid catastrophic forgetting.

PeTTA [12]: Explicitly controls parameter drift during test-time adaptation by dynamically adjusting the update and regularization parameters, thereby preventing model collapse and sustaining performance over extended, recurring testing scenarios.

EATA [27]: Extends ETA by incorporating a Fisher-based anti-forgetting regularizer [18] to prevent forgetting and ensure robust adaptation.

ROID * : A modified version of ROID [24] that omits the augmentation consistency loss, enabling a fair comparison with other baselines which do not incorporate this extra regularization. Removing it isolates the impact of key components like weight ensembling and diversity weighting, which are common to the compared methods.

4. Domain-Disentangled / Prompt-based TTA. These methods incorporate mechanisms to disentangle domain-specific features or use visual prompts for more effective adaptation to domain shifts.

CoLA [3]: Leverages collaborative adaptation by sharing domain knowledge vectors across devices for improved efficiency.

DPCore [47]: Utilizes prompt-based learning by incorporating visual prompts to guide domainspecific adaptation.

BECoTTA [20]: Uses input-dependent blending of experts to adapt to domain shifts, and is evaluated especially in the context of semantic segmentation.

H Implementation Details

Following the RobustBench 4 [11] (MIT license), we employ WideResNet-28 [46] on CIFAR-10-C, ResNeXt-29 [44] on CIFAR-100-C, and ResNet-50 [10] on ImageNet-C. We also evaluate on ViT-B16 [6] and ResNet-50 with GroupNorm [43] for ImageNet-C to further assess generalizability. For CIFAR-10-C and CIFAR-100-C, TTA baselines (except SAR [28]) are optimized with the Adam optimizer [17] using a learning rate of 1 × 10 -3 , a universal β = (0 . 9 , 0 . 999) , and a batch size of 200, whereas SAR employs SGD [32]. For ImageNet-C, models are adapted with SGD at a batch

size of 64 and a learning rate of 2 . 5 × 10 -4 (adjusted to 1 × 10 -4 for ViT-B-16 in the CCC setting). For ReservoirTTA, we configure the system with a maximum of K max = 16 reservoirs, determine the threshold τ using 2000 source examples, and update centroids with AdamW [23] at a learning rate of 1 × 10 -4 . Table 10 summarizes these settings. Concerning DomainNet-126, and PACS, we optimize the models with SGD at 2 . 5 × 10 -4 learning rate, and Adam at 1 × 10 -3 learning rate, respectively. Note CIFAR10-C, CIFAR100-C, and ImageNet-C are publicly available online 5 (Apache-2.0 license). CCC is also provided by Rdumb paper 6 [31] (MIT license). Both DomainNet-126 7 and PACS 8 are publicly available. All experiments were run on a single NVIDIA A100 Tensor Core GPU (80 GB VRAM) on our internal cluster.

I Additional Ablation Studies

0.9

0.99

0.995

0.999 0.9995

1.0

28

30

32

34

36

38

Error Rate (%)

0.0

1

x

10 1

1

x

10 2

1

x

10 3

2

x

10 3

Fisher weight

ROID* (CSC)

+ReservoirTTA

ROID* (CDC)

+ReservoirTTA

EATA (CSC)

+ReservoirTTA

EATA (CDC)

+ReservoirTTA

Impact of ReservoirTTA on Regularization Weight Sensitivity of EATA and ROID ∗ . We analyze how ReservoirTTA affects the optimal hyperparameter α of ROID ∗ , which controls the momentum in weight ensembling with the source model. We also examine its impact on Fisher weight λ , which regulates the distance to the source model parameters in EATA. As shown in Figure 14, ReservoirTTA reduces the sensitivity of both EATA and ROID ∗ to these hyperparameters on CIFAR100-C recurring CSC and CDC settings. Furthermore, we observe that the best results with ReservoirTTA are achieved at a higher α for ROID ∗ , shifting from 0.99 in ROID ∗ to 0.995 in ROID ∗ +ReservoirTTA. A similar trend is observed for EATA, where the optimal Fisher weight λ decreases from 2000 in EATA to 1000 in EATA+ReservoirTTA. This effect can be attributed to the Model Reservoir, which exposes each model to a more stable distribution of domains over time, thereby reducing the need for strong regularization.

This effect is even more pronounced in ViTs. As shown in Figure 15, EATA achieves its best performance at K max =1 with a regularization weight of 2000.0, whereas ReservoirTTA ( K max =16) reaches optimal performance with a regularization weight of 0. Interestingly, as K max increases, the need for regularization diminishes. For ROID ∗ , the trend differs. The momentum parameter α

0.9

0.99

0.995

0.999

0.9995

1.0

1

2

4

8

16

K

47.5 44.0 43.1 41.0 40.8 40.8

47.5 44.1 43.1 41.0 40.7 40.1

47.6 44.2 43.2 41.1 40.6 39.6

47.7 44.2 43.2 41.0 40.4 39.6

47.7 44.3 43.2 40.8 40.2 39.5

ROID

*

+ReservoirTTA

0.0

1.0

100.0

1000.0

2000.0

Fisher weight

K

41.4

41.2

41.2

40.4

40.1

40.3

40.3

40.2

39.9

39.9

39.7

39.7

39.7

39.7

39.8

39.6

39.8

39.9

40.1

40.2

39.5

39.7

40.0

40.3

40.4

EATA+ReservoirTTA

40

42

44

46

39.5

40.0

40.5

41.0

8

16

32

64

128

200

B

27.5

30.0

32.5

35.0

37.5

40.0

Error Rate (%)

Recurring CSC

8

16

32

64

128

200

B

Error Rate (%)

Recurring CDC

ROID*

EATA

+ReservoirTTA

+ReservoirTTA

should be set to 1.0 for ViT to achieve the best results, suggesting that regularization is unnecessary for this architecture. Moreover, further increasing K max reduces the error rate by 0.6% on CCC, demonstrating the effectiveness of ReservoirTTA in this setting.

Effect of Batch Size. As illustrated in Figure 16, we explore batch sizes from 8 to 200 for both EATA + ReservoirTTA and ROID * + ReservoirTTA on CIFAR-100-C under recurring CSC and CDC settings. We observe a consistent decrease in error rate with larger batches, which likely stems from more stable gradient estimates, reduced variance in style features, and more accurate domain assignments. Conversely, very small batches can lead to noisy updates and suboptimal domain clustering. Although a large batch is important for reliable performance, memory constraints and real-time needs may limit its size in practice.

Sensitivity to Various Sequence Orders. We examine how DPCore, CoLA, and ReservoirTTA respond to different test batch orders during domain discovery. As shown in Figure 17, we measure the

Gauss

Shot

Impulse

Defocus

Motion

Zoom

Snow

Frost

Fog

Bright

Contrast

Elastic

JPEG

0

10

20

30

40

50

60

70

Number of Detected Domains

Noise

Blur

Weather

Digital

DPCore: # of detected domains during the 1st visit

1

2

3

4

5

6

7

8

9

10 11 12 13 14 15 16 17 18 19 20

0

100

200

300

400

500

Number of Detected Domains

DPCore: # of detected domains across 20 visits

Gauss

Shot

Impulse

Defocus

Motion

Zoom

Snow

Frost

Fog

Bright

Contrast

Elastic

JPEG

1

2

3

4

5

6

7

8

Number of Detected Domains

Noise

Blur

Weather

Digital

CoLA: # of detected domains during the 1st visit

1

2

3

4

5

6

7

8

9

10 11 12 13 14 15 16 17 18 19 20

0

20

40

60

80

100

120

140

160

Number of Detected Domains

CoLA: # of detected domains across 20 visits

Gauss

Shot

Impulse

Defocus

Motion

Zoom

Snow

Frost

Fog

Bright

Contrast

Elastic

JPEG

2

4

6

8

10

12

14

Number of Detected Domains

Noise

Blur

Weather

Digital

ReservoirTTA: # of detected domains during the 1st visit

1

2

3

4

5

6

7

8

9

10 11 12 13 14 15 16 17 18 19 20

12

13

14

15

16

17

18

19

Number of Detected Domains

ReservoirTTA: # of detected domains across 20 visits

Seed 1

Seed 100

Seed 1000

Seed 200

Seed 2000

number of detected domains after the first visit and across 20 visits in recurring CSC on CIFAR100-C using five seeds-each corresponding to a distinct batch order while keeping the domain sequence fixed. Our findings reveal that DPCore is highly sensitive to batch order, undermining its reliability. Although CoLA yields consistent detection across seeds, its number of detected domains continuously increases, reaching 160 models after 20 visits. In contrast, ReservoirTTA shows minimal sensitivity, with detected domains stabilizing near the ground truth of 15 after 20 visits. These results underscore the superiority of our style quantifier, online clustering, and model discovery mechanisms over those used in CoLA and DPCore.

Resetting to Source vs. Reusing Specialists. We analyze reset-based strategies to isolate the benefit of reusing adapted specialists versus simply resetting to source parameters. On CIFAR-100-C (CSC), blind periodic resets every 1,000 iterations (RDumb) yield an average error of 32.17%. Domain-aware resets that trigger only when the VGG-based detector flags a domain change (RDumb w/ VGG) improve to 31.17% ( -1 . 0% ). By contrast, ReservoirTTA+EATA reactivates the specialist previously adapted to a recurring domain-avoiding costly re-adaptation and forgetting-and achieves 28.57% ( -2 . 60% vs. domain-aware, -3 . 60% vs. blind). See Table 11 for the comparison results.

J Additional Qualitative Results

Figure 18 shows pseudo segmentation label visualization on the ACDC dataset [34]. The first two columns display outputs from TENT and BECoTTA, respectively, while the third column presents refined labels from BECoTTA combined with ReservoirTTA. The inclusion of ReservoirTTA yields more detailed and accurate segmentation labels, improving overall prediction quality.

sidewalk

wall

traffic sign

vegetation

terrain

person

car

TENT

ReservoirTTA

K Additional Quantitative Results

Additional Evaluations on Recurring Domain Shifts. In Tables 12-23, we present extensive quantitative evaluations on classification benchmarks under various recurring domain shift scenarios (CSC, CDC, and CCC) and across different architectures. Overall, the results confirm that ReservoirTTA significantly reduces error rates and prevents catastrophic forgetting compared to existing TTA methods. These evaluations reinforce our main findings, demonstrating that ReservoirTTA's architecture-agnostic design allows seamless adoption in both CNN- and transformer-based networks.

Additional Evaluation on Object-Level Style Shifts. To further assess the versatility of our framework beyond scene-level corruptions, we evaluate on datasets with pronounced object-level domain gaps, namely DomainNet-126 (126 classes) and PACS (7 classes). On DomainNet-126, we follow a CSC protocol by training a ResNet-50 on each source domain (real, painting, clipart, sketch), then adapt over a 20-cycle recurring sequence of the remaining three target domains. A similar protocol is applied to PACS, using photo, art painting, cartoon, and sketch as sources. As reported in Tables 24 and 25, integrating ReservoirTTA with EATA consistently reduces error across both datasets, yielding average improvements of 0.79% on DomainNet-126 and 0.70% on PACS, with the largest gains observed when adapting from Real on DomainNet-126 (-1.60%) and from Cartoon on PACS (-1.93%). These results show that a framework motivated by scene-level shifts transfers robustly to object-level distribution changes.

Evaluation Under Gradual Shift. To assess the ability of our framework to handle gradual changes in corruption severity, we evaluate on a gradual domain shift stream constructed from CIFAR100-C (CSC) using a ResNeXt-29 backbone. Each corruption type cycles through severity levels 1 → 2 → 3 → 4 → 5 → 4 → 3 → 2 → 1 over 20 recurrences. Even under these subtle transitions, ReservoirTTA consistently improves over EATA. As shown in Table 26, EATA averages 25 . 41% error, while ReservoirTTA achieves 25 . 06% with K max = 16 ( -0 . 35% ) and 24 . 95% with K max = 64 (additional -0 . 10% ). Thus, our method adapts to gradually evolving severities, not only abrupt domain changes.

Can ReservoirTTA Help Tackle Temporally Correlated Test Streams? ReservoirTTA raises the question of whether it can tackle the challenge of a temporally correlated testing stream. As observed in Table 27-on CIFAR-10 → CIFAR-10-C under recurring continual CSC, with temporal correlation of image categories (Dirichlet coefficient = 0.1)- where we integrate ReservoirTTA with state-of-the-art TTA method, RoTTA [45], which is designed to address temporal correlations-yields similar gains. RoTTA tackles the instability caused by temporally correlated test samples by introducing mechanisms that stabilize the online adaptation process via smoothing the updates over time. In particular, RoTTA uses an exponential moving average (EMA)-based update to accumulate robust, long-term statistics that mitigate the bias introduced by correlated mini-batches. This principle is further enforced by using robust batch normalization (RBN) and by maintaining a memory bank through category-balanced sampling, which ensures a representative snapshot of the test distribution.

Although tackling temporally correlated test streams is not the primary focus of ReservoirTTA , it is important to note that during test time, when images are highly correlated, extracting style information using channel-wise feature statistics (logvar) can become biased toward the majority category. To address this issue, we adopt RoTTA's Category-Balanced Sampling with Timeliness and Uncertainty (CSTU) strategy, which resamples images from the online stream in a way that maintains category balance. By integrating this strategy with our method, we enhance stability over an extended, temporally correlated test stream, and instead of using the ensemble output from our reservoir models, we make predictions using the mean-teacher approach as introduced in RoTTA. While methods such as RoTTA are potential candidates for integrating ReservoirTTA, their need for separate memory banks per model makes integration computationally prohibitive and beyond the scope of our paper, which focuses on prolonged domain recurrence rather than temporally correlated test streams.

are averaged over 5 seeds.

are averaged over 5 seeds.

are averaged over 5 seeds.

are averaged over 5 seeds.

are averaged over 5 seeds.

are averaged over 5 seeds.

are averaged over 5 seeds.

Recurring TTA visit

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - →

Method

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

Avg

Source

48.75

48.75

Single-Target TTA

Tent

42.35

38.91

38.25

38.05

37.82

37.74

37.69

37.64

37.58

37.55

37.53

37.53

37.53

37.53

37.48

37.55

37.59

37.60

37.60

37.64

37.96

+

ReservoirTTA

42.87

40.76

39.58

38.75

38.26

37.81

37.40

37.07

36.79

36.53

36.29

36.10

35.91

35.75

35.60

35.46

35.34

35.22

35.13

35.04

37.08

Continual TTA

CoTTA

∗

48.55

48.06

47.69

47.26

46.94

46.64

46.35

46.12

46.00

46.03

46.08

46.12

46.32

46.30

46.31

46.34

46.45

46.40

46.38

46.46

46.64

ETA

38.91

36.69

36.22

35.93

35.87

35.69

35.71

35.67

35.47

35.55

35.36

35.32

38.54

44.64

38.43

36.21

38.69

48.53

48.34

48.43

38.71

+

CoLA

40.98

38.47

36.74

35.87

35.44

34.96

34.76

34.58

34.44

34.41

34.27

34.21

34.13

34.04

33.93

34.04

33.89

33.90

33.90

33.82

35.04

+

ReservoirTTA

39.40

36.46

34.96

34.13

33.56

33.19

32.87

32.62

32.47

32.35

32.26

32.19

32.08

32.02

31.96

31.97

31.93

31.89

31.89

31.86

33.10

SAR

44.52

41.85

40.84

40.37

40.14

40.02

39.89

39.83

39.76

39.72

39.69

39.67

39.67

39.65

39.62

39.58

39.57

39.54

39.52

39.50

40.15

+

ReservoirTTA

43.44

41.56

40.54

39.85

39.38

38.93

38.57

38.28

38.00

37.78

37.56

37.37

37.20

37.05

36.93

36.79

36.68

36.55

36.45

36.33

38.26

Persistent TTA

RDumb

39.32

39.23

38.88

38.69

38.64

39.07

41.49

40.62

38.65

38.98

38.69

39.46

39.19

38.85

38.47

38.66

39.06

42.09

38.95

38.68

39.28

EATA

39.11

37.07

36.15

35.70

35.37

35.10

34.96

34.88

34.76

34.58

34.63

34.51

34.69

34.62

34.40

34.53

35.30

34.37

34.39

34.60

35.19

+

ReservoirTTA

39.76

37.23

35.98

35.11

34.63

34.18

33.81

33.54

33.33

33.19

33.03

32.86

32.77

32.65

32.55

32.44

32.40

32.33

32.28

32.24

33.81

ROID

∗

40.14

40.04

40.02

40.06

40.07

39.98

40.03

40.01

40.07

40.12

40.06

40.04

40.00

40.07

40.04

40.04

40.04

40.76

40.06

40.03

40.08

+

ReservoirTTA

40.09

38.30

38.04

37.94

37.94

37.90

37.87

37.90

37.85

37.88

37.83

37.83

37.80

37.77

37.78

37.78

37.77

37.77

37.77

37.77

37.98

Prompt-based TTA

DPCore

40.21

43.25

44.12

44.40

44.50

44.94

44.84

44.83

44.79

44.90

44.89

44.99

45.06

45.07

45.47

45.33

45.45

45.84

45.79

45.95

44.73

37.98

36.05

35.43

35.22

34.96

34.83

34.63

34.51

34.40

34.31

34.38

34.25

34.12

34.04

34.00

33.90

33.85

33.77

33.77

33.72

34.61

Recurring TTA visit

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - →

Method

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

Avg

Source

48.75

48.75

Single-Target TTA

Tent

44.21

39.96

39.28

39.03

38.93

38.82

38.75

38.67

38.63

38.59

38.58

38.58

38.59

38.58

38.59

38.59

38.61

38.59

38.66

38.68

39.05

+

ReservoirTTA

46.64

44.93

43.90

43.34

42.84

42.51

42.02

41.84

41.54

41.33

41.11

40.94

40.77

40.68

40.49

40.44

40.30

40.28

40.23

40.19

41.82

Continual TTA

CoTTA

∗

48.37

47.90

47.43

46.97

46.63

46.34

45.91

45.69

45.49

45.44

45.51

45.62

45.54

45.67

45.61

45.68

45.70

45.66

45.90

45.94

46.15

ETA

42.89

37.66

36.99

36.74

36.48

36.25

36.20

36.24

35.96

36.08

36.08

35.94

35.93

36.91

36.40

36.46

36.29

35.87

36.06

35.87

36.66

+

CoLA

40.90

37.61

36.67

36.10

35.86

35.64

35.47

35.33

35.25

35.13

35.20

35.11

35.12

35.02

35.10

35.10

35.11

34.97

35.03

34.87

35.73

+

ReservoirTTA

41.55

38.17

36.62

35.60

35.03

34.51

34.14

33.96

33.79

33.60

33.46

33.55

33.42

33.33

33.18

33.10

33.07

33.07

32.96

32.92

34.45

SAR

45.86

42.16

41.12

40.60

40.32

40.14

40.03

39.96

39.91

39.80

39.77

39.73

39.71

39.66

39.68

39.64

39.60

39.60

39.57

39.57

40.32

+

ReservoirTTA

45.47

43.00

41.89

41.27

40.75

40.28

39.96

39.70

39.46

39.25

39.09

38.95

38.80

38.55

38.43

38.26

38.13

38.04

37.88

37.91

39.75

Persistent TTA

RDumb

43.91

40.27

39.32

39.32

39.74

40.37

40.43

39.53

39.16

39.03

38.88

39.53

39.55

39.26

40.30

39.15

39.51

40.07

39.74

39.20

39.81

EATA

39.89

37.20

36.34

35.76

35.62

35.28

35.20

35.00

34.87

34.80

35.24

34.66

34.62

34.97

34.54

34.58

34.55

34.51

34.46

34.53

35.33

+

ReservoirTTA

42.08

39.54

38.14

37.29

36.62

36.08

35.79

35.56

35.24

35.14

35.04

34.99

34.80

34.66

34.49

34.37

34.30

34.26

34.16

34.19

35.84

ROID

∗

42.09

41.12

41.73

46.97

46.24

41.18

42.15

45.99

44.52

41.81

41.99

41.12

40.91

41.28

42.01

42.81

41.45

41.43

41.21

41.12

42.46

+

ReservoirTTA

40.35

38.47

38.04

38.01

38.00

37.92

37.86

37.89

37.90

37.92

37.89

37.85

37.83

37.83

37.85

37.81

37.78

37.75

37.77

37.82

38.03

Prompt-based TTA

DPCore

42.79

45.68

46.94

47.14

44.86

42.40

42.25

41.90

43.09

43.42

44.07

45.85

45.53

45.40

46.93

46.43

47.41

47.46

47.33

47.58

45.22

38.01

36.30

35.87

35.75

35.56

35.34

35.35

35.17

34.92

34.78

34.78

34.74

34.72

34.55

34.47

34.45

34.48

34.32

34.32

34.32

35.11

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn light of the risk of catastrophic forgetting in the TTA model under prolonged TTA settings, the authors propose ReservoirTTA, which maintains a pool of models consisting of different parameters from various domains and styles. In methodology section, the authors detail how ReservoirTTA initialize, adapt and predict with this model pool. The experiments are mainly conducted on CIFAR10/100-C, ImageNet-C and Cityscapes to ACDC. The experimental results show that the module has strong plug-and-play capability and is compatible with many TTA methods such as TENT, EATA and SAR.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n1. The motivation for this paper sounds excellent, and I recognize that current TTAs are not robust over long periods of adaptation.\n2. Rather than imposing numerous constraints on the input, parameter or output spaces, such as sample selection, weight ensemble or anchor loss, this paper proposes maintaining a pool of models and actively switching to the most suitable model according to domain awareness.\n3. The experiments demonstrate that the proposed method has strong compatibility with the existing TTA methods.\n\n**Weaknesses:**\n\n1. This paper introduces a VGG network and a hyper-parameter, $\\tau $, to identify changes in the domain. It makes stronger assumptions than conventional TTA methods: 1) A pre-trained VGG network that can discriminate between different test domains (which are unknown in advance). 2) a dataset-specific hyperparameter, $\\tau $, which requires source data to be obtained.\n2. The paper's focus is on mitigating the risk of catastrophic forgetting in long-term TTA. The authors argue that a model pool would enable the use of different model parameters in different domains. However, earlier TTA work has shown that adapting a model for a long time within a single domain can still cause catastrophic forgetting problems. To address this, they introduced tools such as Fisher information and weight ensembling, which constrain the updated parameters to be as close as possible to the source model. Does this conflict with your starting point, where you want to update to the domain optimum within each domain? They constrain the parameters of each domain to be as close as possible to the source parameters.\n3. Since you could leverage the VGG network to identify domain changes, what would be the performance of simply resetting the adapted model parameters to the source parameters when the domain changes?\n\nQUESTIONS:\nSee the weaknesses above.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper focuses on overcoming the limitations of Test-Time Adaptation (TTA) methods designed to address the domain shift problem, which degrades the performance of deep learning models in real-world deployments. The authors identify a critical challenge in existing TTA approaches: performance instability caused by catastrophic forgetting and error accumulation when faced with recurring domain shifts. To tackle this, the paper introduces a novel framework that leverages a VGG-19 auxiliary network to quantitatively extract the style of input domains and dynamically identifies them through online clustering of style vectors. This strategy effectively resolves fundamental TTA issues by preserving and reusing parameters learned from previously observed domains while training separate expert models for newly identified ones. Furthermore, for inference, the framework employs an ensemble strategy in the parameter space, dynamically creating a weighted average of all expert models. This approach delivers predictions optimized for the current target domain without the significant computational overhead associated with multiple forward passes.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths \n1. A key strength of the proposed ReservoirTTA is its architecture-agnostic design. Since the style extraction is decoupled from the main model and handled by a VGG-19 auxiliary network, the framework exhibits high compatibility, enabling its application to a wide range of architectures, including diverse CNNs and ViTs.\n2. The empirical evaluation rigorously validates the method's effectiveness by testing it under various scenarios designed to mimic real-world domain shifts (*e.g.*, CSC, CDC, and CCC). The results consistently show that ReservoirTTA outperforms previous state-of-the-art continual TTA methods in inference accuracy, highlighting its robustness and superior adaptation capabilities.\n3. The method's robustness is further underscored by its strong performance even when $K^{max}$ exceeds the number of predefined corruption types. This indicates a capacity to manage unexpected domain shifts, making it a viable TTA algorithm for real-world applications.\n\n### Weaknesses\n1. The proposed method introduces notable computational overhead due to the additional VGG-19 network and the online clustering mechanism, resulting in up to a 30% increase in computation compared to lightweight TTA methods. This can lead to a significant accumulation of inference latency. Furthermore, if the clustering process is executed on the host CPU rather than a parallel processor like a GPU, it could cause substantial data movement between the host processor and external memory (e.g., DRAM). This, in turn, may lead to a considerable increase in power consumption, raising concerns about its efficiency and practicality in resource-constrained mobile environments such as autonomous driving scenario.\n2. The empirical evaluation primarily focuses on scene-level domain shifts, such as weather variations. Consequently, it remains unclear whether the proposed methodology would be effective for object-level style changes, where textures and contours are fundamentally altered (*e.g.*, 3D Rendering-to-Real). Since baseline methods like TENT are evaluated on such benchmarks (*e.g.*, VisDA-C, PACS), demonstrating performance improvements on these datasets would have substantially strengthened the paper's contribution and validated the generalizability of the approach.\n3. Minor comment: There is a duplicate citation on line 65, which lists \"methods [37, 40, 31, 31]\".\n\nQUESTIONS:\n1. The evaluation focuses on scene-level corruptions. Have the authors considered evaluating ReservoirTTA on benchmarks characterized by object-level domain shifts (*e.g.*, VisDA, PACS, DomainNet, and etc.) to further assess its generalization capabilities?\n\n2. The experiments are primarily conducted at a fixed severity level of 5. Can the proposed method effectively identify and adapt to instances of the same corruption type but with varying levels of severity?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper tackles the challenge of prolonged test-time adaptation (TTA) under recurring and evolving domain shifts. Unlike prior single-model TTA methods that update one model continuously and suffer from catastrophic forgetting or error accumulation, the authors propose ReservoirTTA, which maintains a pool (“reservoir”) of domain-specialized models. Incoming test batches are routed via online clustering over low-level style features extracted from a frozen VGG-19 backbone; new clusters spawn new specialist models, while recurring domains reuse existing ones. Only the selected specialist is updated per batch, and final predictions ensemble all specialists weighted by soft assignments. Theoretical analysis bounds parameter variance growth and explains how decoupled adaptation mitigates drift. Empirical results on ImageNet-C, CIFAR-10/100-C, and Cityscapes→ACDC segmentation demonstrate up to 2–7% error reduction and stable performance over 20+ recurrences, outperforming baselines like CoTTA, EATA, and prompt-based methods\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n- This  paper Introduces a reservoir of specialist models is an elegant solution to catastrophic forgetting in recurring domains; it decouples updates and naturally leverages past adaptation \n\n- Online clustering on VGG-19 style log-variance features, combined with reservoir sampling and mutual-information regularization, yields reliable domain detection without source annotations \n\n- Thorough evaluation under structured (CSC), unstructured (CDC), and continuously changing (CCC) shifts, with both CNN and ViT backbones, plus segmentation tasks. Ablations on reservoir size, Kmax, and component contributions bolster reproducibility\n\nWeaknesses\n\n- New models are cloned from the specialist maximizing mutual information on the current batch. It might be beneficial to compare this to initialization from the source model or ensemble of top-k specialists, to verify optimality.\n\nQUESTIONS:\n- Sect. 4.2: Soft assignment qt uses Euclidean distances in d-dim space—have you tried cosine similarity?\n- Eq. (7): Can you clarify choice of quantile for τ and discuss trade-off between sensitivity and specificity?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}

comments: SUMMARY:
This paper proposes a framework (called ReservoirTTA) for prolonged test-time adaptation in scenarios where the test domain continuously shifts over time, including recurring and evolving domains. The proposed ReservoirTTA maintains a reservoir of domain-specialized models to detect new domains via online clustering and route samples to the appropriate model for adaptation.

STRENGTHS AND WEAKNESSES:
Strengths:
1.	The proposed method introduces a multi-model strategy that decouples adaptation across domains, overcoming limitations of single-model TTA methods such as catastrophic forgetting and error accumulation.
2.	This paper provides theoretical bounds showing how parameter regularization curbs collapse in single-domain TTA and motivates the modular design of ReservoirTTA for maintaining stability under recurring domain shifts.
3.	The results demonstrate that the proposed method improves adaptation accuracy and maintains stable performance across prolonged, recurring shifts, outperforming state-of-the-art methods.

Weaknesses:
1.	In this paper, both the multi-model strategy and the online clustering technique introduce some additional computational overhead, which may limit the scalability of the proposed method to very large datasets or real-time applications.
2.	The experimental results are not convincing. The authors should compare the proposed algorithm with more recent and relevant methods for prolonged and recurring test-time adaptation.
3.	A more thorough discussion of the potential weaknesses of the proposed method would strengthen this paper.

QUESTIONS:
Please see the Strengths And Weaknesses.

LIMITATIONS:
Please see the Strengths And Weaknesses.
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 3
DATE: 2025-07-06