Class-aware Domain Knowledge Fusion and Fission for Continual Test-Time Adaptation
Authors: Jiahuan Zhou
Chao Zhu
Zhenyu Cui
Zichen Liu
Xu Zou
Gang Hua
Venue: NeurIPS 2025 poster ()
Keywords: Continual Test-Time Adaptation
Knowledge Fusion
Knowledge Fission
Primary area: applications

Abstract: Continual Test-Time Adaptation (CTTA) aims to quickly fine-tune the model during the test phase so that it can adapt to multiple unknown downstream domain distributions without pre-acquiring downstream domain data. 
To this end, existing advanced CTTA methods mainly reduce the catastrophic forgetting of historical knowledge caused by irregular switching of downstream domain data by restoring the initial model or reusing historical models. However, these methods are usually accompanied by serious insufficient learning of new knowledge and interference from potentially harmful historical knowledge, resulting in severe performance degradation. To this end, we propose a class-aware domain Knowledge Fusion and Fission method for continual test-time adaptation, called KFF, which adaptively expands and merges class-aware domain knowledge in old and new domains according to the test-time data from different domains, where discriminative historical knowledge can be dynamically accumulated. Specifically, considering the huge domain gap within streaming data, a domain Knowledge FIssion (KFI) module is designed to adaptively separate new domain knowledge from a paired class-aware domain prompt pool, alleviating the impact of negative knowledge brought by old domains that are distinct from the current domain. Besides, to avoid the cumulative computation and storage overheads from continuously fissioning new knowledge, a domain Knowledge FUsion (KFU) module is further designed to merge the fissioned new knowledge into the existing knowledge pool with minimal cost, where a greedy knowledge dynamic merging strategy is designed to improve the compatibility of new and old knowledge while keeping the computational efficiency.

Class-aware Domain Knowledge Fusion and Fission for Continual Test-Time Adaptation
Jiahuan Zhou', Chao Zhu', Zhenyu Cui', Zichen Liu', Xu Zou Gang Hua Wangxuan Institute of Computer Technology, Peking University, Beijing 100871, China 2he Huazhong University of Science and Technology, Wuhan 430074,China
Amazon.com, Inc, Bellevue, WA 98004, USA
jiahuanzhou@pku. edu. en, zhuc2022@mail sustech. edu. pku. edu. en, zxλzoux. me, ganghua@gmail com
Abstract
Continual Test-Time Adaptation (CTTA) aims to quickly fine-tune the model dur- ing the test phase that it adapt to multiple unknown downstream domain distributions without pre-acquiring downstream domain data. To this end, existing advanced CTTA methods mainly reduce the catastrophic forgetting of historical knowledge caused by irregular switching of downstream domain data by restoring the initial model reusing historical models. However, these methods are usually accompanied by serious insufficient learning of new knowledge and interference from potentially harmful historical knowledge, resulting in severe performance degradation. To this end, we propose a class-aware domain Knowledge Fusion and Fission method for continual test-time adaptation, called KFF, which adaptively expands and merges aware domain knowledge in old and new domains ac- cording the test-time data from different domains, where discriminative historical knowledge can be dynamically accumulated. Specifically, considering the huge domain gap within streaming data, domain Knowledge Flssion (KFI) module is designed to adaptively separate new domain knowledge from paired class- aware domain prompt pool, alleviating the impact of negative knowledge brought by old domains that are distinct from the current domain. Besides, avoid the cumulative computation and storage overheads from continuously fissioning new knowledge, a domain Knowledge FUsion (KFU) module is further designed to merge the fissioned new knowledge into the existing knowledge pool with minimal cost, where greedy knowledge dynamic merging strategy is designed to improve the compatibility of new and old knowledge while kceping the computational effi- ciency. Extensive experiments the ImageNet-C dataset verify the effectiveness our proposed method against other methods. The source code is available at httpr////oococoiaaaaaaa//////eae
Introduction
Recently, deep neural networks have demonstrated powerful adaptation capabilities various vision
tasks 10
14
62 50]. but still suffer from the well-known distributional shift problem [4 [5
22
51,
49
between training and test data. To address this problem, Test-Time Adaptation (TTA)
is proposed to adapt the test data in the target domain by using only unlabelled streaming test
data |6 57
29 17 Existing TTA methods have shown promising capacity to improve the
generalizability of pre-trained models through self-supervised training methods [46] 39 57 [8] 55)
Despite some progress, most TTA methods merely focus on the generalizability within single
"Corresponding author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

testing domain, ignoring the multiple test see- narios that may appear from time time in real scenarios 47
Data
Domain
To tackle the above issue, Continual Test-Time Adaptation (CTTA) aims to exploit the unla- belled test data streams with continually chang-
Fog
Bright
Fog
Stram
1-1
H+
ing testing domains for test-time adaptation [47
(a) Domain Changes in Continual Test-Time Adaptation
38.. as shown Figure Ta). The core chal-
lenge of CTTA is to adapt to the changing test data distribution by reducing error accumula- tion and preventing catastrophic forgetting to
Domana Conflici
Knowledge Office
improve the robustness of long-term adaptation. To this end, some CTTA methods mainly pre- serve historical knowledge through regulariza-
1-1
1-1
(b) Gradient Direction of Existing Methods
tion 47 38 45 32
restoration 37
38 [54]
which aim to slow down the learning new data and correct domain style bias, thereby sup- pressing the impact of distributional shift prob- lem caused by domain gaps, respectively. Un- fortunately, these methods ignore the domain conflict between streaming data collected from distinct domains, which fails to fully accumulate differential domain knowledge in various test- time data. Specifically, advanced CTTA meth- typically select and fuse parameters
Degradation
33.5
15.9
1-1
/+1
(c) Performance of Existing Methods
Figure 1: Existing methods [32 failed to accu- mulate knowledge in distinct domains, where con- flicting knowledge disrupts the gradient direction during optimizing and remains severe domain conflict problem.
ods 33 61 of historical models. Therefore, as shown in Figure the knowledge offset between two distinct domains will inevitably disrupt the gra- dient direction during TTA optimization. Consequently, Figure e illustrates, it not only brings an unlimited and continuously increasing storage overhead of historical models, but also compromises the discriminability due the inevitable mixing of conflict domain knowledge. Therefore, CTTA remains challenging issue to solve when considering the trade-off between adaptation efficiency and effectiveness.
To adapt pre-trained models to downstream tasks efficiently, prompt learning 4 41 20
proposes adjust small set of parameters while keeping the pre-trained parameters fixed. However, existing prompt learning methods typically face two key challenges. For one, existing prompt learning methods 61 I 53 fail to separate category information and domain information, resulting in mixture of discriminative information from different domains in the resulting prompt, which inhibits the discrimination of the adapted prompts. Second, existing prompt learning methods []] are difficult to dynamically adjust the prompting capacity according to different testing domains. Therefore, when the test domain switches irregularly. the adapted prompts typically weaken the robustness of discriminability in various domains.
Inspired by the above observations, proposed class-aware domain Knowledge Fusion and Fission (KFF) framework for continual test-time adaptation. To accumulate various knowledge in different domains, designed Knowledge Fission (KFI) and Knowledge FUsion (KFU) module to achieve the continual evolution of historical knowledge. Specifically, a KFI module proposed dynamically fission class-aware domain knowledge adapted the current domain by evaluating the knowledge discrepancy between the current domain and the historical domains. Sequentially, KFU module introduced merge the fissioned knowledge into the existing knowledge pool at minimal cost. Among them, a greedy-basee based knowledge fusion strategy is proposed to achieve the fusion of various knowledge with minimal risk of old knowledge loss.
We evaluate our KFF with three common CTTA benchmarks (ImageNet-to- MinageNet-to-lmageNNe-- CIFARI00-1O-CIFAT (OC and
ImageNet-C [15]
(3) under continual changing domains We further compare our KFF with state-of-the-an algorithms, including the latest advance- ments in CTTA and TTA fields. The experimental results show its effectiveness under various changes of testing domains. In particular, our KFF achieved 34.8% eтror under the ImageNet-to-Imagelee distributional shift case, which surpasses the previous SOTA method DPCore [6]] by 5.1%.
47
16 6

---

2
Related Work
Test-time Adaptation. Test-time adaptation (TTA) aims to adapt pre-trained models handle
distribution shifts during inference, without access to source data or additional supervision 7 12
18
19
36
29
44).
Some methods employ self-suuprvised losses, such as entropy minimization 446
57 58 or consistency maximization [5], to adjust the model. Some methods involve preliminary
steps to use source data: by extracting source characteristics such as statisties or features [59
35
60]. or by warming up injected parameters on source data before adaptation [12 24
However, existing TTA methods typically assume static target domain and fail to account for domain shifts
45
that evolve over time [3] 3 47]. This limitation results in challenges such as error accumulation and catastrophic forgetting, which significantly degrade model performance and adaptability during inference.
Continual Test-time Adaptation. Compared to TTA, continual test-time adaptation (CTTA)
considers more practical scenario in which the target domain continuously evolves. This setting exacerbates challenges like error accumulation [5] and catastrophic forgetting [47]. To address these
issues, some methods such as EATA [8] and TTA [32] introduce regularization strategies to mitigate error accumulation, while others like ERSK (37], RDumb [42) and CoTTA [47] utilize
weight reset mechanisms counteract catastrophic forgetting. Beyond updating the model itself, some approaches leverage a small number of parameters to incrementally learn aargt-dommai--specifn knowledge (e.g., VDP [1] SVDP [53], and ViDA (27). However, these methods struggle to retain domain-specific knowledge over time, resulting in poor performance when previously encountered domains reappear. More recently, DPCore (61] attempts preserve historical domain knowledge and dynamically compose it during inference. While it effectively retains past domain information, it applies all previously stored knowledge to each new test batch without considering potential domain conflicts, which may lead to suboptimal performance.
Prompt Learning. Prompt learning is initially introduced in natural language processing (NLP) 4 41 as a means of using learnable prompt tokens to better adapt pre-trained models to downstream tasks. Inspired by its success, researchers have extended this approach computer vision 20
48 52, 26, 31], achieving competitive results. Motivated by this, several methods have explored
the integration of prompt learning into TTA
12 [6 30 and CTTA scenarios [53]
For
instance, VDP [1I] and SVDP [53] propose self-training models that adapt learnable visual prompts to dynamically changing domains. Other methods, such as DePT [12, CPT4 (21] and DPCore [6] introduce learnable prompts into Vision Transformers (ViTs), enhancing their ability to handle complex visual inputs and improving performance in TTA and CTTA settings. However, existing prompt-based TTA/CTTA approaches focus primarily prompting for knowledge at the domain level, often overlooking the shared class-level information across domains, which could further enhance generalization.
3 Method
3.1 Problem Formulation and Notations
CTTA Problem Formulation. We focus continual test-time adaptation here, where the target distribution differs from the source distribution and is not static. The training data are from the source domain Ds [Ys,, Xs). and the test data are from different domain distributions dominated as where N represents the number of potential target domains, which is unknown and can be infinite or repeating. The model fo encounters test batches B37 = [x=)&_ Tt E X of batch size b in an online manner, which means it will only meet one batch B, test time t. The entire process cannot access any source domain data and can only access the target domain data once. With continually changing domains, our goal is to adapt the pre-trained model to target domains and maintain the ability of the model on historical domain distributions.
Dr = [Xr 1.V.
Vision Transformers ViTs). We focus ViTs for their outstanding representation learning powers. A ViTs of can be decomposed into a feature extractor o X + Z and a classifier head h : Z + },
3

---

Ratch -
Botch
Botch ++1
Knowledge Flerion
Knowledge Favion
Knowledgr Fixsion
Knowledge Favion
Knowsedgge Firdio
Knowledge Favion
Anowsedgee Faoken
Knowsedge Fasias
CLS Tiliee
Prospet
Puss
CLL Taieee
Betch
Froeeee
Froept Rlube
Heal
Pronpes
Procuge Pisten
Origleal
Updloed
Diice Enirosking
kenvingg:
Mn 23 2Mf
Dhealls Trenge
Desuia Pronegt
Insur Tokees
Takio
Figure The Overall pipeline of our KFF with Knowledge Fission(KFD) and Knowledge Fu- sion(KFU), For each test batch, the KFI module dynamically fissions domain prompt and several class prompts to adjust the source model. Sequentially, KFU merges the fissioned prompts after optimizing, achieving the balance between effectiveness and efficiency.
such that f d h. Let 20 Z represents the classification token in Z, the standard prediction process follows:
2 (xx), y l((z))
(I)
ÿ = softmax(y)
3.2 Overview of the Proposed KFF
As shown in Figure our proposed method mainly contains two key modules: Knowledge Fission and Knowledge Fusion. Given input test batch B3 from domain Dr, we choose or fission class prompts (P for every single test sample in {ar and domain prompts PJ for the whole test batch By and get prompts P {P] {P*) 1 then we apply it to the pre-trained ViT model 0, to get the predicted category i = f(B; 0., P) of input test batch B To optimize model performance, prompts are trained using a bi-level loss function that integrates both batch-level domain alignment (Ca) and instance-level classification entropy (Cc). We adopt domain alignment for batch-level loss computation, leveraging its established effectiveness and computational efficiency [2 34 Specifically, the domain alignment loss La measures the discrepancy between the source domain and the current domain by calculating the combined Fuclidean distance of their feature means and standard deviations:
- (P) + +a - -0] (P)
Notably, while this domain alignment distance calculation requires source data, labels are not needed, as we only perform marginal distribution alignment. And approximately 300 unlabelled source examples are enough for stable performance [61]. At the instance level, we minimize the prediction entropy using:
1
C.
H(De).
The model then learned P from by minimizing C La + aC, and use the learned prompts to update the two prompt pools, enabling adaptive knowledge accumulation across batches. After the update, fusion step is carried out control the size of the prompt pool, i.e. N₂ and Na. and retain historical knowledge, balancing computational efficiency with the preservation of key information for improved generalization and performance.
3.3 Knowledge Fission Module
Noticed that there might be performance degradation due to domain conflicts, shown in Figure | we proposed knowledge fission strategy to prevent the current test batch from being influenced by the conflict historical knowledge follows:

---

Class Knowledge Fission. In this submodule, we aim to handle the knowledge fission at class level. To achieve this, we use the cosine similarity st,, = sim B:) between pseudo labels Yr and prompt keys 1/ to evaluate prompt P; for every test sample in the whole batch. Specifically, we first extract pseudo-labels ie without relying on any prompt, which allows us to obtain an initial understanding of the samples' characteristics without the influence of existing prompts. Then, we evaluate P. by 81,6 which helps us to determine how well a prompt aligns with the pseudo-label of a test sample. The prompts that St,i > Ye will be selected as candidates and will be used in weighted manner:
exp(st,i/Te)
P'
Wi W.Pii Wi
(4)
No Σ === exp(st,//T)
If no candidates were selected for 2t, which means the model finds a new class which has not been seen before and is not similar to any seen classes, the model will fission a new prompt Pt for the test sample. This new prompt is designed worth mentioning that at the initial state when the pool is empty, we will fission a prompt for every single test sample. This is because there are no existing prompts to rely on, and each sample needs to have its own representation. The class prompt Pe is concatenated by p' of each test sample z; and will be used for prediction and learning:
capture the unique characteristics of the new class. is
P = [
pi........,.
,..., PP].
Domain Knowledge Fission. The module takes the statical numbers 1 i.e. mean f and standard a, of the batch Byj as input key to match the domain prompt the domain prompt pool. These statistical features can effectively represent the overall characteristics of the test batch and are used to match the domain prompt in the domain prompt pool. The prompt pool will select prompts where d; d(rT. I;) < Ya as candidates based on the Euclidean distance of input statistical numbers r and prompt keys I,;
d; = (=d(TT,1)) = || r. rell2. r -=t(u00
(6)
The selected prompts will be used by:
exp( 1 d Ta)
Pa= = PJ-
wi Pd w,
Na i=0 exp( (xxp(ddi/a)
no prompt is selected, which implics that the test batch comes from a new domain and is not similar to historical domains, or it is the first batch for testing, the model will fission new prompt for the test batch. This new fissioned prompt is added to capture the unique characteristies of the new domain.
3.4 Knowledge Fusion Module
Fission-only method will cause the prompt pool to grow up without limitation, which may result in inefficiency, inadequate understanding of historical knowledge and unnecessary retention of duplicate historical information. To address this problem, we proposed a knowledge fusion strategy to limit the growth of the prompt pool, enhance the comprehension of historical knowledge, and eliminate redundant information.
Class Knowledge Fusion. Once the model has learned the class prompts P.. we incorporate them into the prompt pool using Algorithm [ Inspired by the finding suggested in EATA (38) that adaptation on test samples with very high entropy may hurt performance, we use a threshold m to control whether the test sample should be used for updating the prompt pool. Those learned prompts with H())) > T will be used for updating the prompt pool: the learned fissioned prompts will be directly add to the original prompt pool with its pseudo label Dt. otherwise the prompts will update all the prompts that composed it in the original prompt pool with the weight of composition:
p
[wn (1--wy P...
(8)
To keep the size of the prompt pool for efficiency as well as maintain the knowledge in it as much possible, we cluster and fuse the prompts in the original prompt pool with minimum spanning
5

---

tree( We construct graph G = (V, where V is the set of all prompts in the original prompt pool and E' is the of edges connecting each pair of prompts, calculated by cosine similarity:
Ejj
11 1yll
By applying the MST algorithm, i.c. Kruskal, to this graph, we can find a sub-graph that connects all prompts with the minimum total edge weight, clustering them into Ne groups. Prompts that closely connected in the sub-graph are then fused together to reduce the size of the prompt pool.
Algorithm 1 Algorithm of Updating Class Prompt Pool
Require: Output y, learned prompts Pa and weights of prompts
1: for each i [O, len(P))) do
2: if softmax, softmaxx_entropy(/
7h then
3: continue
4: if' max(w;) is NaN then
5: add (ūt, Pt) to class prompt pool
6: else
7: for each (yi, P.) E domain prompt pool do
8:
Vr We + (1 a;W)i
9:
De 4- W/i Pr + (1 - W:)PP
10:
if class prompt pool full then
11: cluster prompts in class prompt pool into N₂ groups
merge all the prompts in each group
Domain Knowledge Fusion. After backwards propagation, the model will use learned domain prompt Pa to update the domain prompt pool with Algorithm [ We will add a new prompt to the original domain prompt pool if the prompt is fissioned prompt, otherwise, it will update all the prompts that compose it in the original prompt pool with the weight of composition:
P
Ph + (1 w.
(10)
In cases where the domain prompt pool reaches its maximum capacity, need to reduce its size while preserving the most important knowledge. To achieve this, we fuse closest pair of prompts in the pool by Euclidean distance rç).
Algorithm 2 Algorithm of Updating Domain Prompt Pool
Require: Test batch statistic rT' learned prompt Pa and weights of prompts w if mar(w) is NaN then
2: add (IT, Pā) to domain prompt pool
3: fuse the nearest pair if domain prompt pool full
4: else
5: for each (T,, Pd) E domain prompt pool do
6:
D + a T + (1 agw, Tr
7:
P:u+ + w Pi + (1 - (1-w;)P? w.
4
Experiments
4.1 Experiment Setup
Datasets. We evaluate our proposed method three classification CTTA datasets: ImageNet-to- ImageNet-C [3], CIFAR100-to 1o---000----ARI----- CIFAR100C and
(3]. Each dataset has 15 corruption types (categorized into main groups) and 5 corruption severity levels. We use the highest level of corruption severity and keep the same order as COTTA 47 CTTA settings.
Comparison Methods. We compared our proposed method with state-of-the-an CTTA and TTA methods. In detail, we investigated general TTA methods TENT [46), SAR (39] and POEM M
and CTTA methods CoTTA [7], VDP (Л RoTTA (55), C-MAE 28 ROID 33 ViDA [27]
CoLA 6 with DeYO 25), PALM and DPCore 61.

---

Table Classification error rate (%) for CoreeNeettt--mageNer online CTTA task, evaluated on
ViT-Base backbone with corruption severity level 5.
Method
Venue
Gauss
Shot
Impulse
Defocus
Glass
Motion
Zoom
Snow
Frost
Fog
Bright
Contrast
Elastic
Pixel
JPEG
Mean
Source
53.0
51.8
52.1
68.5
78.8
58.5
63.3
49.9
54.2
57.7
26.4
91.4
57.5
38.0
36.2
55.8
Tent
IC1R127
52.2
48.9
49.2
65.8
73.0
54.5
58.4
44.0
47.7
50.3
23.9
72.8
55.7
34.4
33.9
51.0
SAR
1CUR123
45.8
45.9
47.7
52.3
63.7
46.2
50.9
40.3
42.4
41.8
24.4
53.4
53.6
38.4
36.6
45.6
POEM
24
43.7
41.7
41.9
48.4
53.2
42.9
50.1
39.3
36.8
35.9
31.3
99.9
54.6
34.4
41.6
46.4
COTTA
47.7
47.0
46.4
57.5
71.2
S2.2
59.3
39.7
39.3
62.8
24.1
78.9
57.6
33.4
31.1
49.9
MAX223
52.7
51.6
50.1
70.2
56.1
42.1
46.1
45.8
23.6
70.4
54.9
34.5
50.0
RoTTA CVPR'2
51.5
50.3
51.7
60.4
58.7
52.6
54.8
47.2
43.5
42.8
25.9
49.1
48.8
46.3
39.7
48.2
C-MAE CVPW224
46.3
41.9
42.5
51.4
54.9
43.3
40.7
34.2
35.8
64.3
23.4
60.3
37.5
29.2
31.4
42.5
ROID
57.6
51.5
52.2
55.1
52.4
46.5
47.2
45.6
39.5
36.0
26.0
45.0
43.8
39.7
36.3
45.0
ViDA
47.7
42.5
42.9
52.2
56.9
45.5
48.9
38.9
42.7
40.7
24.3
52.8
49.1
33.5
33.1
43.4
PALM AAAF25
41.8
39.9
39.8
57.4
63.7
47.7
53.6
36.1
39.9
41.5
21.5
56.8
51.6
31.7
43.6
DPCore ICMU:2
42.2
38.7
39.3
47.2
51.4
47.7
46,9
39.3
36.9
37.4
22.0
44.4
45.1
30.9
29.6
39.9
Ours
40.1
36.5
36.0
44.5
45.6
39.1
39.1
32.2
31.0
30.0
20.9
38.3
34.9
26.3
27.4
34.8
40
Implementation Details. We followed the implementa- tion details specified in previous work [77 61]. We use ViT-B/16 as our backbone. We utilize the AdamW op- timizer with a learning rate 0.1 for domain prompts and 0.001 for class prompts with batch size 64. The
35.
34
32.0
Mon Binr (K)
10
26.4
27.3
22.5
length of domain prompts is set to 8, and the length of class prompts is set to Other aiper-parametes ad, Yex Ole. Tá. a, Na and No are set to 25, 0.005, 2, 0.1, 0.1, 3, 20 and 100. The nyper-parameters were deter- mined using four disjoint validation corruptions [Speckle Noise, Gaussian Blur, Spatter, Saturate] from ImageNet-C, following MEMO [57]
10
CoTTA C-MAE
Our
Figure Classification error rate(%) for -IFAR1000+--CFARRIO0 online CTTA task.
4.2 Main Results
CTTA with Non-Repeating Domains. We evaluated our proposed method across various challenging domain
Error Ratei (N6) Lower is Belta
Tex
adaptation scenarios without domain repetition. For the SoggeNettto-ImageNeee
COTTA
task, shown in Table I our method achieves a notable state-of-the-art (SOTA) im- provement of 21% 31% over the source model. Compared to second- method, DPCore, still shows a signif- icant performance improvement of 5.1 %. Additionally, we evaluate our method on the CIFAR 100-tt-CIFAR 100C and CIFAR 10-to- 1OC datasets. In the CIFAR 100- 1o-CIFARI00C task, our method outperforms DPCore by 2.6%, while in the provement reaches 3.0%. These results show that our method achieves SOTA performance both datasets, highlighting its strong generalization ability and effectiveness in handling non-repeating domain shift scenarios. Details for CIFAR10/100-1 tao--t--CAARRR-- R1R/00000CC available at Section
MMEM
DPCoze
(8.99M,
34.8%)
30
0.0
0.4
0.8
98
Leamable Parameters (M), Lawer (etter(()
task. the im-
Figure 4: Computational analysis on ImageNet-C.
CTTA with Repeating Domains. In real-world scenarios, test data domains may not only continu- ally shift but also reappear after being previously encountered. Under such conditions, CTTA methods are expected to effectively retain knowledge from seen test domains and retrieve it to assist prediction when those domains recur. As shown in Table 2 we train all 15 domains of ImageNet-C C for 10 repeated rounds and compare the mean performance of existing TTA and CTTA methods, can be seen that method achieves 34.5% mean error rate, yielding an improvement of 9.9% compared
7

---

Table Classification error rate(%) for ImageNer-to-lmageee-- online CTTA task in 10 repeated rounds (RI-R10).
Method
Venue
R4
R5
R6
R7
R8
R9
R10
Mcan
Source
55.8
55.8
55.8
55.8
55.8
55.8
55.8
55.8
55.8
55.8
55.8
Tent
21
51.0
50.6
51.0
53.1
67.9
89.7
99.9
99.9
99.9
99.9
76.3
COTTA
CVPR'22
49.9
50.8
51.5
51.5
51.7
52.2
53.0
53.2
53.3
53.5
ViDA
ICER '24
43.5
42.7
42.5
42.4
42.4
42.3
42.3
42.3
42.2
42.3
42.5
CoLA
NearlP'
40.6
39.9
38.8
38.8
38.8
38.4
38.0
38.8
38.0
38.8
38.9
DPCore
39.9
41.2
43.2
44.2
44.8
45.4
45.9
45.7
46.3
46.8
44.4
Ours
34.8
34.6
34.6
34.6
34.3
34.2
34,4
34.4
34.4
34.5
34.5
Input image
DPCore
Ours
Inpust image
DPCorre
Ours
Inpet image
DPCore
Ours
Motion Blur
The qualitative analysis of attention map on the ImageNet- CTTA task. We compared the attention map of the CLS token between DPCore and our method during CTTA process.
Figure 5:
to DPCore. This significant performance gain is primarily attributed to proposed Class- Domain Knowledge Fission module, which effectively learns and retains domain-specific knowledge, and the Knowledge Fusion module, which mitigates forgetting of previously seen domains while maintaining constant parameter overhead.
4.3 Comparison with SOTA
Computation and Memory Efficiency. We analyze the computational complexity across methods in Figure by comparing Icarnable parameters. The results show that our proposed method achieves efficiency by introducing only 0.09M parameters (~0.1% 5% of the total parameters of the model) while delivering the best performance. Further- more, we conduct a comparative analysis of our proposed method and the state-of-the-ant high-efficiency prompt- based approach DPCore in terms of learnable parameters for the CTTA with repeating domains task, as illustrated in Figure 6 The results reveal that although our method initially exhibits marginally higher number of parame- ters in the first round due to the class prompts, it maintains parameter stability throughout subsequent iterations. contrast, DPCore experiences a continuous increase in the number of parameters. In the final round, DPCore utilizes approximately five times more parameters than our method, highlighting the superior parameter efficiency of our proposed approach.
1026
Leunable Params, (K)
1000
DPCore
842
915
Ours
800
600
504
400
30
175
260
200
200
200
200
200
200
120
Round
Figure Computational analysis on ImageNet-C under 10 repeated rounds.
Visualization and Analysis. In Figure 5 we present a qualitative analysis of attention maps on the ImageNet-C CTTA task, focusing on the attention patterns of the CLS token between the previous SOTA method DPCore and our proposed method. The results reveal that our method can direct attention towards discriminative regions associated with object classes, while DPCore exhibits more diffused attention patterns, failing to concentrate on class-specific details. This disparity in attention allocation underscores efficacy of our class-specific prompt design enhancing feature extraction and adaptation performance. Furthermore, in Figure 7, we conduct a t-SNE analysis to visualize feature distributions of different domains, where distinct colours denote various domain labels. Instead, they converge towards similar feature clusters, indicating tendency to overgeneralize across

---

Tel Beldi Suciatis
Text Stsisties Pooray Keys
Buic Prougt
Prorigh
(a) DPCore
(h) Ours without KFU
(c) Ours
Figure 7: t-SNE analysis across different domains. Differnet colours represent different domains. Result shows that methods assign batches with correct prompts, compared to that DPCore and method without KFF tend to overgeneralize across domains.
domains. These findings highlight the critical role of method's prompt assignment strategy and KFF in enhancing domain discrimination during continual test-time adaptation. A comprehensive theoretical analysis of this phenomenon, including simple assumption and mathematical proofs, is presented in Section further validating the effectiveness of our proposed approach.
4.4 Ablation Study
Effect Each Component. Table evalu- ate the contributions of class prompts(P.). do- main prompts(Pa). Knowledge Fission(KFI) and Knowledge Fus))n(KFU) ImageNet-to. ImageNet-C CTTA task. Exp-1, when only domain prompts utilized along with KFI and KFU, the error rate drops significantly by 16.3% compared the source model, reaching 39.5%. However, still lags behind our pro- posed method by 4.7% in terms of error rate. Exp-2, which incorporates class prompts but omits domain prompts, shows 4.9% decrease in етrоr rate relative to the pre- tre-trained Source model, with an eтrо rate of 50.9%. In Exp-3, both Pa and Pa are used but without KFI and KFU, suffers notable performance decline, with an error rate increasing to 62.9%, highlighting the critical role of KFI. On the other hand, Exp-4, which includes both types of prompts and KFI but excludes KFU, achieves an error rate of 36.9 56.9%. Our proposed method, with all components integrated, achieves the lowest average error rate of 34.8%. The result indicates that each component plays an indispensable role in CTTA tasks.
Table 3: Effect of each components. Average error
(n) for ImageNett-o-ImageNeeeC CTTA task
Base
Pa
Pa
KFI
KFU
Mean
55.8
v
V
39.5
v
/
/
50.9
v
/
v
62.9
36.9
34.8
Influence of Hyper-parameters.
3H
The size of domain prompt pool Na and the size of class prompt pool N. are two important hyper-parameters in KFF. Therefore, we conduct extensive experiments to evaluate their influence. As illustrated in Figure g both prompt pool sizes have little impact on performance within reasonable range, showing stable error rate change. This because our KFF has comprehensively described trend of domain and class changes across the CTTA process through knowledge fission and fusion, while excessive prompts inevitably introduce unnecessary parameter redundancy. Therefore, to balance effectiveness and efficiency, our KFF sets Na and N. to and 100, respectively. We provide further discussion about the hyper-parameters in the Appendix Section B
Eroo rale (%))
Error (N6)
3.6
35
10
20
30
40
50
100
150
200
Size of Domain Prompt Pool
(b) of Class Prompt Pool
Figure 8: Ablation Study on ImageNet-to-lmageNeet online CTTA task.

---

Conclusion
In this paper, we tackle the twin challenges of catastrophic forgetting and inadequate assimilation of new knowledge in Continual Test-Time Adaptation (CTTA), where models must adapt to unknown, shifting domains without prior access downstream data. We introduce KFF, a class-aware Knowl- edge Fusion and Fission framework: the Fission module isolates discriminative, domain-specific prompts to block interference from dissimilar historical domains, while the Fusion module greedily merges new knowledge back into the existing pool to maintain efficiency. Across multiple CTTA benchmarks, KFF reduces forgetting by up 30% and improves new-domain accuracy by average of 4.2%, all with minimal extra computation and storage.
Acknowledgements
This work was supported by the National Natural Science Foundation of China (62376011) and the National Key R&D Program of China (2044YF1441000),
References
[1] Y. Bar, S. Shaer, and Y. Romano. Protected test-time adaptation via online entropy matching: A betting approach. The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL httpst::: httptttts///p//eeeeeee http:///ppeeree.eeeeee wwwp://wwwwee.ooooiiideeeie qamf jyhPeg
[2] S. Ben-David, J. Blitzer, K. Crammer, and F. Pereira. Analysis of representa- tions for domain adaptation.
In B. Schölkopf, J. Platt, and T. Hoffman, edi- tors, Advances in Neural Information Processing Systems, volume 19. MIT Press, 2006. URL ftttps://proceediing htt.:/eeeeddiinnnnnnueeueeeeeeee wwtp:ccccppppeeeeeeee 0tt0:4222aaaa0001114441661111189
pdf
[3] D. Brahma and Rai. A probabilistic framework for lifelong test-time adaptation. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3582-3591, June 2023.
[4] T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert- Voss, G. Krueger, T. Henighan, R. Child, A. A.Rmmesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B.Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei. Lan- guage models are few-shot learners. NIPS 20, Red Hook, NY, USA, 2020. Curran Associates Inc. ISBN 9781713829549
[5] D. Chen, D. Wang, T. Darrell, and S. Ebrahimi. Contrastive test-time adaptation. In CVPR, 2022.
[6] G. Chen, S. Niu, D. Chen, S. Zhang, C. Li, Y. Li, and M. Tan. Cross-device collaborative test-time adaptation. In The Thirty-eights Annual Conference on Neural Information Processing Systems, 2024.
[7] Z. Chi, Y. Wang, Y. Yu, and J. Tang. Test-time fast adaptation for dynamic scene deblurring via meta-auxiliary learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9137-9146, June 2021.
[8] S. Choi, S. Yang, S. Choi, and S. Yun. Improving test-time adaptation via shift-agnostic weight regularization and nearest source prototypes. In S. Avidan, G. J. Brostow, M. Cissé, G. M. Farinella, and T. Hassner, editors, Computer Vision ECCV 2022 17th European Conference, Tel Aviv, Israel. October 23-27. 2022, Proceedings. Part XXXIII, volume 13693 of Lecture Notes in Computer Science, pages 440-458. Springer, 2022. doi: 10. 111100/977888--1119987----888881 Wtl.phttts:///ii..ot
10.
26
[9] F. Croce, M. Andriushchenko,, V. Sehwag, E. Debenedetti, N. Flammarion, M. Chiang, P. Mittal, and M. Hein. Robustbench: standardized adversarial robustness benchmark. In Thirty- Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2021.
URL https:://pperreviie httpeenrrriiee.eee//rrrrrrt wwt.:/ee......ooo//dddd/ss......
[10] A. Dosovitskiy, I.. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, Uszkoreit, and N. Houlsby. An image worth 16x16 words: Transformers for image recognition at scale. ICLR, 2021.
10

---

[11] Y. Gan, Y. Bai, Y. Lou, X. Ma, R. Zhang, N. Shi, and L. Luo. Decorate the newcomers: Visual domain prompt for continual test time adaptation, 2023. URL https://axxv..rg///
2212. 04145.
[12] Y. Gao, X. Shi, Y. Zhu, H. Wang, Z. Tang. X. Zhou, M. Li, and D. N. Metaxas. Visual prompt tuning for test-time domain adaptation, 2023. URL URL http::://openrevvveee.etttttoood 3InIBTJIXTS
[13] S. Gui, X. Li, and S. Ji. Active test-time adaptation: Theoretical analyses and an algorithm. In The Twelfth International Conference on Learning Representations, 2024. URL https
//pperevview weet/forummdd-VHGGl....
[14] K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick. Momentum contrast for unsupervised visual representation learning. arXiv preprint arXiv: 1et::/991,,022,,, 2019.
[15] D. Hendrycks and T. Dietterich. Benchmarking neural network robustness to common corrup- tions and perturbations. Proceedings of the International Conference Learning Representa- tions, 2019.
[16] T.-H. Hoang, D. M. Vo, and M. N. Do. Persistent test-time adaptation in recurring testing scenarios. In Thirty-eighth Conference on Neural Information Processing Systems (NeurIPS), 2024.
[17] Y. Iwasawa and Y. Matsuo. Test-time classifier adjustment module for model-agnosta domain generalization. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P. Liang. and J. W. Vaughan, editors, Advances in Neural Information Processing Systems. volume 34, pages 2427-2440. Curran Associates, Inc., 2021. URL /ttpstttss:////tttsss /ttptp:::///prcceee pttpoeeepp///rroceedinn.gsss newrips.e.ppp..r... ct/ccc/c/ppo wwtppppr..........p files/ .pf
[18] V. Jain and E. Lealned-Miller. Online domain adaptation of pre-trained cascade of classifiers. In CVPR 20/ pages 577-584, 2011, doi: 10. 20.1109/CPR.2011... 1.59953177
[19] M. Jang, S.-Y. Chung, and H. W. Chung. Test-time adaptation via self-training with nearest neighbor information. In The Eleventh International Conference on Learning Representations, 2023. URL
[20] M. Jia, L. Tang, B.-C. Chen, C. Cardie, S. Belongie, B. Hariharan, and S.-N. Lim. Visual prompt tuning. S. Avidan, G. Brostow, M. Cissé, G. M. Farinella, and T. Hassner, editors, Computer Vision ECCV 2022, pages 709-727, Cham, 2022. Springer Nature Switzerland.
[21] M. Karimian and H. Beigy. Cpt4: Continual prompted transformer for test time train- ing. Information Sciences, 700:121841, 2025. ISSN 0020-0255. doi: https://doi.org/ 10.1016/j.ins.2044..
URL htLhhttp::/// Destps::/www.. http://ww..ww....e.eeee......eee www.ccciecc///eaaicccccecoocceee
p11/80020025524017559
[22] P. W. Koh, S. Sagawa, H. Marklund, S. M. Xie, M. Zhang, A. Balsubramani, W. Hu, M. Ya- sunaga, R. L. Phillips, I. Gao, T. Lee, E. David, Stavness, W. Guo, B. Earnshaw, Haque, S. M. Beery, J. Leskovec, A. Kundaje, E. Pierson, S. Levine, C. Finn, and P. Liang. Wilds: benchmark of in-the-wild distribution shifts. In M. Meila and T. Zhang, editors, Pro- ceedings of the 38th International Conference Machine Learning, volume 139 of Pro- ceedings of Machine Learning Research, pages 5637-5664. PMLR, 18-24 Jul 2021. URL https::/proceeddinne
mlr. press/ www.///o/1aa..... htmll.
[23] A. Krizhevsky, Learning multiple layers of features from tiny images. University of Toronto, 05 2012.
[24] D. Lee, J. Yoon, and S. J. Hwang. BECOTTA: Input-dependent online blending of experts for continual test-time adaptation. In Forty-first International Conference on Machine Learning, 2024. URI httttps://opereeviee net./foruumiddd5xTTXX22222
[25] J. Lee, D. Jung, S. Lee, J. Park, J. Shin, U. Hwang, and S. Yoon. Entropy is not enough for test-time adaptation:
From the perspective of disentangled factors. In The Twelfth International Conference on Learning Representations, 2024. URL htttps:://ppereeeeeeeee wttpriiwwwwwweeeeeeee form?
id=9w3iw8aDuE
[26] Q. Li, K. Xu, Y. Peng, and J. Zhou. Exemplar-free lifelong person re-identification via prompt- guided adaptive knowledge consolidation. IJCV, 132(1 32(11)):8504488 2024.
[27] Liu, S. Yang, P. Jia, M. Lu, Y. Guo, W. Xue, and Zhang. Vida: Homeostatic visual domain adapter for continual test time adaptation. arXiv preprint 4rXiv:2306.04344 2023.
11

---

[28] J. Liu, R. Xu, S. Yang. R. Zhang, Q. Zhang, Z. Chen, Y. Guo, and S. Zhang. Continual- mae: Adaptive distribution masked autoencoders for continual test-time adaptation. pages 28653-28663, 2024.
[29] Y. Liu, P. Kothari, B. G. van Delft, B. Bellot-Gurler, T. Mordan, and A. Alahi. Tt+++: When does self-supervised test-time training fail or thrive? In Thirty-Fifth Conference on Neural Information Processing Systems, 2021.
[30] Z. Liu, H. Sun, Y. Peng, and J. Zhou, Dart: Dual-modal adaptive online prompting and knowledge retention for test-time adaptation. In Proceedings of the AAAI Conference Artificial Intelligence, volume 38, pages 14106-14114, 2024.
[31] Z. Liu, K. Xu, B. Su, X. Zou, Y. Peng, and J. Zhou. Stop: Integrated apatial-tempore dynamic prompting for video understanding. In CVPR, pages 19776-137866 2025.
[32] S. K. Maharana, B. Zhang, and Y. Guo. PALM: pushing adaptive learning rate mechanisms for continual test-time adaptation. CORR, abs/2403. 10650, 2024. doi: 10.48550/ARXIV.2400
10650. URL httLhttt///deee http::///oooooo/.../oo/ 1.0.44850//ariiv
10650
[33] R. A. Marsden, M. Döbler, and B. Yang. Universal test-time adaptation through weight ensembling, diversity weighting, and prior correction. In Proceedings of the IEEE/CVF Winter Conference Applications of Computer Vision (WACV), pages 2555-2565, January 2024.
[34] A. Mehra, Y. Zhang, and J. Hamm. Understanding the transferability of representations via task-relatedness. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
[35] M. J. Mirza, P. J. Soneira, W. Lin, M. Kozinski, H. Possegger, and H. Bischof. Actmad: Activation matching to align distributions for test-time training. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
[36] J. Nath Kundu, N. Venkat, M. V. Rahul, and R. Venkatesh Babu. Universal source-free domain adaptation. 2020 IEEE/CVE Conference on Computer Vision and Puttern Recognition 2020. doi: 1101199/CYV7-200022000000444
(CVPR), pages
[37] Niloy, S. M. Ahmed, D. S. Raychaudhuri. S. Oymak, and A. K. Roy-Chowdhury, Effective restoration of source knowledge in continual test time adaptation. In Proceedings of the IEEE/CVE Winter Conference on Applications of Computer Vision (WACV), pages 2091-2100, January 2024.
[38] S. Niu, J. Wu, Y. Zhang, Y. Chen, S. Zheng, P. Zhao, and M. Tan. Efficient test-time model adaptation without forgetting. In The Internetional Conference on Machine Learning, 2022.
[39] S. Niu, J. Wa, Y. Zhang, Z. Wen, Y. Chen, P. Zhao, and M. Tan. Towards stable test-time adaptation in dynamic wild world. In Internetional Conference on Learning Representations, 2023.
[40] S. Niu, C. Miao, G. Chen, P. Wu, and P. Zhao. Test-time model adaptation with only forward passes. In The International Conference on Machine Learning. 2024.
[41] E. M. Ponti, G. Glavaš, O. Majewska, Q. Liu, I. Vulié, and A. Korhonen. XCOPA: A multilingual dataset for causal commonsense reasoning. In B. Webber, T. Cohn, Y. He, and Y. Liu, editors, Proceedings of the 2020 Conference on Empirical Methods in Nat- ural Language Processing (EMNLP), pages 2362-2376, Online, Nov. 2020. Association for Computational Linguistics. doi: 1186536
URL https
//aclanthology.... org/ 202//2220.co wwwee-maaaaeeeeeeeeee 185/
[42] O. Press, S. Schneider, M. Kuemmerer, and M. Bethge. RDumb: A simple approach that questions our progress in continual test-time adaptation. In seirth- Conference on Neural Information Processing Systems, 2023. httpht:tt//:::ntttt www.wiiwwww..eeootnu.....ae VIP6VTVsHc
[43] B. Recht, R. Roelofs, L. Schmidt, and V. Shankar. Do ImageNet classifiers generalize to ImageNet? In K. Chaudhuri and R. Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Leurning Research, pages 5389-5400. PMIR, 09-15 Jun 2019. URL URI recht 10cht19a.. 19ml
mlr. wwwps//vvvvv /
12

---

[44] A. Royer and C. H. Lampert. Classifier udaptation at prediction time. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1401-1409, 2015. doi: 10.1109) CVPR.2015.72999
[45] J. Song, J. Lee, I. S. Kweon, and S. Choi. Ecotta: Memory-efficient continual test-time adaptation via self-distilled regularization. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), pages 11920-11929 June 2023.
[46] D. Wang, E. Shelhamer, S. Liu, B. Olshausen, and T. Darrell. Tent: Fully test-time adaptation by entropy minimization. In International Conference on Learning Representations, 2021. URL https://openreviee..neet hwwp:eiieieeieeeeeeeeeeeeeeeeeee di-uumiiuddduu333
[47] Q. Wang, O. Fink, I.. Van Gool, and D. Dai. Continual test-time domain adaptation. In Proceedings of Conference Computer Vision and Pattern Recognition, 2022.
[48] Z. Wang, Z. Zhang, C.-Y. Lee, H. Zhang, R. Sun, X. Ren, G. Su, V. Perot, J. Dy, and T. Pfister. Learning to prompt for continual learning. In 2022 IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 139-149, 2022. doi: 10.1 2ttp:CCVRRR268.22222.0000
[49] K. Xu, X. Zou, Y. Peng, and J. Zhou. Distribution-awarc knowledge prototyping for non- exemplar lifelong person re-identification. In CVPR, pages 16604-16613, 2024.
[50] K. Xu, X. Zou, and J. Zhou. Lstke: Long short-term knowledge consolidation for lifelong person re-identification. In AAAI, volume 38, pages 16202-16210, 2024.
[51] K. Xu, C. Jiang, P. Xiong, Y. Peng, and J. Zhou. Dask: Distribution rehearsing via adaptive style kernel learning for exemplar-free lifelong person re-identification. In AAAI, volume 39, pages 8915-8923, 2025.
[52] K. Xu, X. Zou, G. Hua, and J. Zhou. Componential prompt-knowledge alignment for domain incremental learning. In ICML, 2025.
[53] S. Yang, Wu, J. Liu, X. Li. Q. Zhang, M. Pan. Y. Gan, Z. Chen, and Zhang. Exploring sparse visual prompt for domain adaptive dense prediction. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 16334-16342, 2024.
[54] X. Yang, X. Chen, M. Li, K. Wei, and C. Deng. A versatile framework for continual test- time domain adaptation: Balancing discriminability and generalizability. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 23731-23740, 2024. doi: 10.1 1.11009VVP22233..20 200.77332.2000022400
[55] L. Yuan, B. Xie, and S. Li. Robust test-time adaptation in dynamic scenarios. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15922 15932, 2023.
[56] C. Zhang, K. Xu, Z. Liu, Y. Peng, and J. Zhou. Scap: Transductive test-time adaptation via supportive clique-based attribute prompting. In CVPR, pages 30032-30041, 2025.
[57] M. Zhang, S. Levine, and C. Finn. Memo: Test time robustness via adaptation and augmentation. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 38629-38642. Curran Associates, 2022, URL URL httptppse//prrccoonngg...ee wwwrine.nippppppe cc hwtp:cccppappppaeeeeeeee www_fffeespppppe http://eiisseaapeeee/e////////// /ptp//////r/////1le/// pdf
[58] Q. Zhang. Y. Bian, X. Kong. P. Zhao, and C. Zhang. Come: Test-tima adaption by conservatively minimizing entropy. In Internetional Conference on Learning Representations, 2025.
[59] Y. Zhang, X. Wang, K. Jin, K. Yuan, Z. Zhang, L. Wang. R. Jin, and T. Tan. Adanpe: Exploring non poramaramt classifier for test-time adaptation. In International Conference on Machine Learning, pages 41647-41676. PMLR, 2023.
[60] Y. Zhang, A. Mehra, and J. Hamm. Ot-vp: Optimal transport- cunsport-guided visual prompting for
test-time adaptation. In 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 1122-1132, 2025. doi:
10.
[61] Y. Zhang, A. Mehra, S. Niu, and J. Hamm. Dpcore: Dynamic prompt coreset for continual
test-time adaptation, 2025. URL. https: //trri/vavvv.ee/ www./bo./////....oooo..oo..e abpp///s...22..../// 10737
[62] J. Zhou, K. Xu, F. Zhuo, X. Zou, and Y. Peng. Distribution-aware knowledge aligning and prototyping for non-exemplar lifelong person re-identification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
Our abstract and introduction clearly state the claims and contributions of our work.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer:
[Yes]
Justification:
We discussed the limitations in Section
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
14

---

Justification:
We provide provide the full set of assumptions and complete (and correct)
proof in Section
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We provide the information needed for reproducing the main results this paper in Section 4.1
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
15

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: The code will be made publicly available after the paper is accepted. At the same time, all the datasets we used are publicly available and the main experimental results can be reproduced based Section 3 and Section 4.1
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
mublic/guider SoodeSummissioooolle
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ //nips http:cccpppccccccccccoooocccccee
for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: We described the settings in the experiment in Section 4.1 including the hyperparameters and how they were chosen, as well as the type of optimizer.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Due limited resources, only extensively evaluate the performance of our proposed method on ImageNet-C across 10 independent runs with different random seeds. The detailed values provided in Section B and Table g Other experiments use fixed random seeds to ensure reproducible results within the same settings.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
16

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπτог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We provided the computer resources we used to conduct the experiments in Section 41
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer:
[Yes]
Justification: The research conducted in the paper conform with the NeurlPS Code of Ethics
https: http:c/ccpccc//cccaaiiiiiiiea
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., s if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discussed the potential impacts of the work in Section D
Guidelines:
The answer NA means that there is no societal impact of the work performed.
17

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
We do not release anything that needs safeguard.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We cited the original paper that we build upon, such as the pre-trained ViT model and the Latasets(ImageN))) ImageNet-C, CIFAR100, CIFAR 100-C, CIFAR10 and CIFAR10C) we used. The detailed information about licenses stated in Section
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
18

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the datasets.paperssiii paperswi thcode. www.coo//aaaaa datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We do not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
19

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification:
The core method development this research does not involve LLMs.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy httt::///eeerrps..ccooo//rreer/p for what should or should not be described.
20

---

A Theoretical Analysis
We further provide theoretical insight into why methods can apply test batches with correct prompts, under the assumption of well-separated clusters.
We consider streaming scenario where test batches Bi..... B₁ arrive sequentially. Each batch B, is associated with a feature representation r = (u(o(B:)), o (B))), where & is a feature extraction function, denotes the mean and a denotes the standard deviation. The distance between two batches is defined as the Euclidean distance between their feature representations:
d(B,, BB) = d(T,, - IIT; - Tall2
Assumption A.1. We assume that these batches can be naturally partitioned into N well-separated clusters {C;} based on their distances. Formally:
Definition A.1 Seeer-Sepr Clusters). A clustering {C; NN is well-separated if there exists a threshold 0 > 0 such that
Vi H max B,B'EC,
d'B, <0< min ECC,
d'B,B')
This implies that intra-cluster distances are uniformly smaller than inter-cluster distances. We set our saper-pprrmeters such that T < 0 and N₂ > N, where JIa is the distance threshold for prompt matching and Na is the maximum number of prompts allowed in the pool.
Lemma A.1. Under the Knowledge Fission mechanism, our method correctly assigns all batches B to prompts from the same cluster.
Proof. We proceed by induction the test time t:
Base Case (t. = 1): At test time f1, when there is no prompts in the prompt pool, We initialize the first prompt with r I(B,), which trivially belongs the same cluster as B,.
Inductive Step: Suppose our algorithm assigns batches Bi, B2...., B.-1 with correct prompts. Since each prompt is updated by I + (1 - a)I;; + oI (B.). for the reason that both I, and I(B,) belongs to the same cluster, 1 also belongs to the same cluster. Then, assume that a new batch B belongs cluster j. we consider two cases based on whether B, matches a prompt: (1) B, matches one or more prompts. For the reason that d(C,)) < 0 < d(C,, Cÿ) and 7a < 6, the matched prompt(s) should be in the same cluster with B, (2) B, do not match any prompt. Then B, creates new prompt I new T (B,) in cluster j. This new prompt, where T Inew = [(B,), lies exactly in the same cluster as Be. Thus, all batches will match the correct prompt(s) by induction.
Lemma A.2. Our proposed method fuses prompts from the same clusters with Knowledge Fusion.
Proof. From Lemma A we can learn that each prompt lies inside a cluster, and as No > N, we know that there exists clusters that has more than one prompts when the prompt pool is full. With assumption d(C,) < 0 < d(C,, Cý). the distance between the fused prompts must be less than 8, which implies that it comes from the same cluster. Thus, our proposed method fuses prompts from the same cluster with Knowledge Fusion, and furthermore the fused prompt still lies inside the cluster.
Proposition A.3. Our proposed method assigns all batches B, with correct prompt(s) from the same cluster with Knowledge Fission and Fusion.
Proof. By Lemma A.I we know that the Knowledge Fission mechanism ensures that each batch B initially assigned to a prompt from its correct cluster. And Lemma shows that the Knowledge Fusion mechanism only merges prompts within the same cluster. Therefore, even after fusion operations, each prompt remains associated with a single cluster. The iterative process knowledge fission and fusion maintains the invariant that all prompts accurately represent their respective clusters. Hence, all batches are consistently assigned to prompts from their correct clusters.
This theoretical framework guarantees that our method effectively organizes prompts into semantically meaningful clusters, enabling accurate and efficient retrieval during inference. The experimental results visualized in Figure Figure corroborate this theory, as the clear separation of domains in the t-SNE plot aligns with the hypothesized cluster structure, and meanwhile, our prompt keys are shown to correspond explicitly to individual domains, thus validating the correctness of our theoretical analysis.
21

---

Table Classification enror rate(%) for ImageNet-to -o-er-too--mee--CCCCCC- online CTTA task across different strategies of selecting class prompts.
Causs
Shot
Impuise
Defoeus
Glass
Motion
Zoom
Snow
Frost
Bright
Contrast
Elastic
Pixel
JPEG
Mean
Fog
Method
Source
53.0
51.8
52.1
68.5
78.8
58,5
63.3
49.9
54.2
57.7
26.4
91.4
57.5
38.0
36.2
55.8
DPCore
42.2
38.7
39.3
47.2
51.4
47.7
46.9
39.3
36.9
37.4
22.0
44.4
45.1
30.9
29.6
39.9
Hard Match
36.6
36.4
45.1
46.8
42.0
40.9
33.6
30.5
28.2
20.8
38.7
39.0
30.4
35.8
Top1 Match
40.9
37.0
36.2
44.5
45.4
40.2
39.3
33.6
31.0
33.4
20.9
39.4
34.7
26.9
28.1
35.4
Top3 Match
40.5
37.0
36.0
44.9
45.7
39.0
38.3
31.7
30.8
37.4
20.7
37.1
34.5
25.5
27.2
35.1
Top5 Match
40.0
37.5
37.8
46.1
48.6
40.5
39.5
32.6
30.4
27.9
21.2
38.7
38.7
30.7
28.7
35.9
Ours
40.1
36.5
36.0
44.5
45.6
39.1
39.1
32.2
31.0
30.0
20.9
38.3
34.9
26.3
27.4
34.8
B Additional Results
Impact of Class Prompt Selection Methods. To evaluate the effect of class prompt selection strategies, we conducted experiments comparing three approaches: using pseudo-labels directly as matching keys, selecting prompts via top-k matching (k 1,3, 5), and threshold-based method for adaptive prompt utilization. As shown Table 4 our threshold-based approach achieves the best performance, outperforming the source model by %. plausible explanation is that while pseudo-label methods fail to leverage inter-class shared knowledge, and top-k methods overlook class- specific gaps, while our threshold mechanism can dynamically identifies discriminative boundaries between classes. This allows for more effective exploitation of prior knowledge, balancing cross-class generalization and intra-clas cttas specificity.
We present the detailed results for Detailed Results for and CIFAR IEPPRRO-tO-CA O--t-CIFAARII 100C online CTTA tasks in Table and Table 6 respectively. For fair comparison, our method is benchmarked against several TTA and CTTA methods. Result shows that our method achieves SOTA performance on both datasets.
Prompt Functionality Validation via Domain- Class Information Separation. To validate the distinct roles of domain prompts and class prompts, we designed an experiment using three sequentially varying domains, where the test set comprised the first 100 classes the ini- tial domain. In the full-information setup, the model was adapted on all samples across the three domains, leveraging both domain and class prompts. the class class-excluded excluded setup, we ex- cluded the first 100 classes of the initial domain from adaptation, forcing the model to rely solely target class information from other domains and domain information without target classes. Across 10 independent tests (T). as shown in Figure 9 the class-excluded setup exhibited only a 0.2% average performance degradation compared to the full inlormafttttonnn setup, which is significantly smaller than the 4.7% and 16.1% degradations observed when isolating one the prompt components (Table These results confirm that: I) domain prompts and class prompts effectively capture domain- and class-level features. respectively; the model integrates these two types of information to maintain robust generalization in unseen scenarios, validating the complementary utility of the proposed prompt design.
40
Full information
Class-oxcluded
Error Rate (%)
30
20
7
T2
T
T2
T5
Te
T7
T
Tg
Tio
Figure 9: Error between the full-information and class-excluded settings ImageNet-C across 10 independent tests (Ti-Tio).
t-SNE Analysis across Different Classes. In Figure 10 we leverage t-SNE to visualize distribution of pseudo-labels for test images, where each class is colour-coded for clarity. Unlike standard t-SNE implementations that rely on Fuclidean distance, we adopt cosine similarity to compute pairwise distances between pscudo-labels, aligning with method's similarity metric
22

---

Table Classification error rate (%) for CIFARI00-to CIFARIOO-C
CIFARLOO--O-CCFARRIOOO online CTTA task, evaluated on
ViT-Base backbone with corruption severity level 5.
Method
Venue
Gauss
Shot
Impulse
Defocus
Glass
Motion
Zoom
Snow
Frost
Fog
Bright
Contrast
Elastic
Pixel
JPEG
Mean
Source
55.0
51.5
26.9
24.0
60.5
29.0
21.4
21.1
25.0
35.2
11.8
34.8
43.2
56.0
35.9
35.4
Tent
IC1R121
53.0
47.0
24.6
22.3
58.5
26.5
19.0
21.0
23.0
30.1
11.8
25.2
39.0
47.1
33.3
32.1
CoTTA
CVPR'22
55.0
51.3
25.8
24.1
59.2
28.9
21.4
21.0
24.7
34.9
11.7
31.7
40.4
55.7
35.6
34.8
VDP
AA4723
54.8
51.2
25.6
24.2
59.1
28.8
21.2
20.5
23.3
33.8
7.5
11.7
32.0
51.7
35.2
32.0
C-MAE
CVPR224
48.6
30.7
18.5
21.3
38.4
22.2
17.5
19.3
18.0
24.8
13.1
27.8
31.4
35.5
26.4
ViDA
JCL8'24
50.1
40.7
22.0
21.2
45.2
21.6
16.5
17.9
16.6
25.6
11.5
29.0
29.6
34.7
27.1
27.3
DPCore JCME/25
48.2
40.2
21.3
20.2
44.1
21.1
16.2
18.1
15.2
22.3
9.4
13.2
28.6
32.8
25,5
25.1
Ours
31.2
28.1
15.1
20.5
36.0
20.4
16.3
19.4
17.7
20.2
10.6
16.8
27.9
27.1
29.6
22.5
Table 6: Classification error rate (%) for -UFARIO-O-CIFARIOC
Method
Venue
Gauss
Shot
Impulse
Defocus
Glass
Motion
Zoom
Snow
Frost
Fog
Bright
Contrast
Elastic
Pixel
JPEG
Mean
Source
60.1
53.2
38.3
19.9
35.5
22.6
18.6
12.1
12.7
22.8
5.3
49.7
23.6
24.7
23.1
28.2
Tent
IC18111
57.7
56.3
29.4
16.2
35.3
16,2
12.4
11.0
11.6
14.9
4.7
22.5
15.9
29.1
19.5
23.5
CoTTA
CVPR'22
58.7
51.3
33.0
20.1
34.8
20.0
15.2
11.1
11.3
18.5
4.0
34.7
18.8
19.0
17.9
24.6
VDP
57.5
49.5
31.7
21.3
35.1
19.6
15.1
10.8
10.3
18.1
4.0
27.5
18.4
22.5
19.9
24.1
C-MAE
CVPR24
30.6
18.9
11.5
10.4
22.5
13.9
9.8
8.8
4.0
12.7
9.2
14.4
12.6
ViDA
JCLN224
52.9
47.9
19.4
11.4
31.3
13.3
7.6
7.6
9.9
12.5
3.8
26.3
14.4
33.9
18.2
20.1
DPCore JCME/25
22.0
18.2
14.9
14.3
24.4
13.9
12.0
11.6
10.7
15.0
5.7
21.8
15.6
12.7
18.0
15.4
Ours
17.8
14.4
9.4
11.4
22.8
12.9
8.8
8.6
8.1
10.5
4.5
17.9
13.3
10.3
15.4
12.4
design, Given that DPCore lacks class-specific prompts, we compare our approach against baseline method without the Knowledge Fission and Fusion (KFF) module. The results demonstrate that our method effectively maps distinct classes to corresponding prompts, thereby mitigating inter-class confusion and achieving superior class discrimination. Specifically, the t-SNE visualization reveals well-separated clusters for each class, validating that our prompt-based adaptation strategy can dynamically adjust to class-specific features during test time.
Further Comparison on Efficiency. To further validate the efficiency of the method, we sup- plemented the comparisons of the learnable parameter count, GPU memory usage, average Flops used per batch, and relative computation time with baselines, evaluated on a single NVIDIA 4090 GPU. The experiments were carried out in the repeating domain setting, with the results presented in Table The results show that the proposed method balances performance and efficiency. When compared to Tent (superior raw efficiency), it gains 15.2% performance with minimal extra cost.
Pezdi Ladd of brags Clos Prace
Frodlt Labcafibe Clas Paep
Ours without KFI KFU
Ours
Figure 10: t-SNE analysis across classes. Different colours represent different classes. Result shows that the proposed method effectively maps pseudo-labe labels of distinct classes to corresponding prompts compared to the baseline method without the KFF.
23

---

Table Computational analysis on ImageNer-to-ImageNNe with repeating domains.
Method
Venue
Params.(M)
Time
Mcm.(GB)
TFLOPs
Err Mean
Tent
ICLR211
0.03
1,0
5.5
1.08
51.0
Round I
CoTTA
86.57
4.7
16.2
3.24
49.9
ViDA
93.70
35.3
9.3
14.03
43.4
DPCore
JCME/2
0.08
1.6
5.7
1.67
39.9
Ours
0.09
1.9
6.0
1.68
34.8
Tent
ICLR'21
0.03
1.0
5.5
1.08
99.9
CoTTA
CVPR
86.57
4.7
16.2
3.24
53.5
Round
ViDA
ICER
93.70
35.3
9.3
14.03
42.3
DPCorc
JCME/25
1.03
2.1
8.4
1.73
46.8
Ours
0.20
1.8
6.1
1.60
34.5
38
38
37
Error raze (5))
Eirot Eale
35
35
34
20
40
60
80
160
2001
300
400
500
Domain Prompt Pool
Size
Class Prompt N.
Figure 11: Ablation Study on ImaggeNer-o-lmmagNeetCC online CTTA task.
Meanwhile, it outperforms CoTTA and ViDA in both efficiency and performance. Regarding DPCore, maintains comparable early-round efficiency while achieving a 5.1% performance gain, and in later rounds (the 10th round), its KFI module ceases new prompt generation when no new domains emerge and reduces computational load, by contrast, DPCore suffers from escalating parameters, inference latency, memory usage, and error rates due unconstrained prompt accumulation. These findings collectively confirm that our method successfully harmonizes performance enhancement and efficiency, validating its suitability for real-world continuous test-time adaptation scenarios.
Further Analysis towards the Hyper-parameters No and No- We further explored how Ne and Na balance pool expansion control and retention of subtle, future-valuable knowledge by spanning broader parameter range. The results in Figure TT show that excessively small restricts prompt diversity, losing critical subtle knowledge for future adaptation, while overly large N causes uncontrolled pool expansion, leading to poorly optimized redundant prompts due to limited samples. Our method selects the optimal N₂ and Na. balancing these trade-offs to avoid both indefinite expansion and valuable knowledge loss.
More Analysis towards the Hyper-parameteern 7 and We conducted extensive experiments on the influences of key hyper-parameteer (y and a) on knowledge fission and fusion, with results summarized in Table 8 The results show that while the optimal To and na value varies slightly between ImageNet-C, CIFARIO, and CIFARI 100, the performance remains stable within reasonable range (0.5%) across all datasets, and that varying and ? within typical range leads performance fluctuations of less than 0.2% on different datasets. Furthermore, excessively small or large Ye/Ya degrades performance consistently: a small Ye/Naa introduces irrelevant knowledge, while large Ya/Ma limits knowledge reuse. This consistent cross-dataset behavior highlights Ye/Ta" 's robustness: though not universally optimal, its core tuning logic, balancing knowledge relevance and reuse, generalizes well, simplifying deployment via systematic adjustment rather than arbitrary search.
Consistency of the Proposed Method Under Different Random Seeds. We evaluated perfor- mance of our proposed method for ImageNet-to- Imageeee-eee--CCCC with 10 different random seeds. As demonstrated T our approach exhibits consistent performance across different initializations, highlighting its stability in handling varied starting conditions.

---

Table 8: Effect of and y across different datasets. Average error rate for CTTA tasks. Numbers with represent the original default parameters tuned on the disjoint validation set from ImageNet-C.
(a) Effect of a, across different datasets.
(b) Effect of ou across different datasets.
0.05
0.1
0.15
0.2
0.3
0.5
0
0.05
0.1*
0.15
0.2
0.3
0.5
IN-C
34.9
34.8
34.8
34.8
34.8
34,9
35.0
IN-C
34.9
34.8
34.8
34.8
34.9
34,9
34.9
C10-C
12.6
12.5
12.4
12.4
12.5
12.6
12.6
C10-C
12.5
12.4
12.4
12.5
12.5
12.6
12.6
C100-C
22.7
22.7
22.5
22.5
22.6
22.5
22.6
C100-C
22.7
22.6
22.5
22.5
22.5
22.6
22.6
(c) Effect of "ye across different datasets.
lc-4
le-3
5e-3*
7c-3
le-2
5e-2
lc-1
IN-C
36.0
35.2
34.8
34.8
35.1
35.2
35.8
C10-C
12.7
12.6
12.4
12.4
12.3
12.2
12.2
C100-C
23.0
22.7
22.5
22.5
22.4
22.1
22.3
(d) Effect of 7 across different datasets.
(c) Effect nh. across different datasets.
10
15
20
25*
30
35
40
1
1.5
2*
2.5
3.5
4
IN-C
36.2
35.0
34.8
34.8
35.0
35.7
35.1
IN-C
35.0
34.8
34.8
35.0
35.1
35.4
35.8
CIO-C
12.7
12.5
12.4
12.4
12.4
12.5
12.5
C10-C
12.6
12.5
12.4
12.4
12.5
12.5
12.5
C100-C
22.9
22.8
22.6
22.5
22.5
22.7
22.8
C100-C
22.8
22.7
22.5
22.5
22.6
22.7
22.7
Table 9: Classification error rate(%) for ImageeNet-t---mageeeeee online CTTA task with 10 different random seeds (RI (RI-R10).
Method
R1
R2
R3
R4
R5
R6
R7
R8
R9
R10
Mean
Std
Ours
35.4
35.5
34.6
34.9
34.6
34.8
34.7
35.2
34.8
34.8
34.8
0.3
C Limitations.
Although our method shows a great improvement in CTTA, there are some limitations that may affect broader application. Firstly, The method relies on accessing source domain statistics during the adaptation phase. This dependency poses challenges in scenarios where source data is proprietary, privacy-restricted,, or unavailable due to compliance barriers (e.g,, medical/financial datasets). Secondly, experimental validation is currently restricted to synthetic, manually designed corruptions, which may not fully mimic the complexity real-world distribution shifts. While synthetic corruptions provide controlled evaluation, the method's 's robustness unseen, naturalistic corruptions remains unvalidated. Furthermore, although our method is computationally efficient compared to most of current methods, we still introduce additional computational overhead during the adaptation phase, posing challenges for resource-constraine edge devices or real-time applications with strict latency budgets.
D Broader Impact.
Our method enhances the generalization of the pre-trained model in continual test-time adaptation, promoting its application in the real world.
E
Asset License and Consent
The ViTs we used is loaded from timm which is released under Apache-2.0 license. Some compo- nents of code and pretrained model is utilized from the official repository of CoTTA [47). ViDA (27]. DPCore [6] and RobustBench [9], which is released under MIT license. The datasets used, CIFAR- 10-C, CIFAR-I00- and ImageNet-C [5 23,, are publicly available online, released under Apache-
2.0 license.
25


Reviews:
[{"review_id": "PMYF6DyclR", "reviewer": "Reviewer_er6C", "summary": "The paper proposes a method for continual test-time adaptation (CTTA) that fine-tunes model behavior during the test phase while accounting for potential domain shifts in the input stream. The core idea involves dynamically managing domain knowledge through a Knowledge Fission (KFI) process, which enables the model to adapt to new domains without forgetting previously seen ones. To prevent unbounded growth of the domain knowledge pool, the authors introduce a complementary Domain Fusion (DFU) mechanism that merges similar domain knowledge representations to maintain a manageable size. Adaptation is achieved using learnable domain and class prompt tokens, which store new knowledge while keeping the backbone of the pretrained ViT classifier frozen throughout the test-time adaptation process.", "questions": "I have two main concerns, both of which, if properly addressed, would significantly strengthen the paper and could change my evaluation from a borderline accept to an accept:\n\nEvaluation on Larger, Real-World CTTA Benchmarks:\nThe current evaluation is primarily based on ImageNet-C and two small-scale toy datasets (CIFAR10/100-C). To better demonstrate the method’s scalability and real-world applicability, I strongly recommend including experiments on large-scale CTTA benchmarks with natural domain shifts, such as Cityscapes-to-ACDC. This would also allow for a more meaningful comparison with DPCore, which reports results on this benchmark.\n\nSensitivity and Robustness of Hyperparameters (γ and α):\nThe method introduces several critical hyperparameters (γ_d, γ_c, γ_h, α_d, α_c) that govern domain knowledge fission and fusion. While some sensitivity analysis is presented for prompt pool size, it would be important to assess how robust these γ and α parameters are across different datasets. A sensitivity study or discussion on their generalizability would greatly improve confidence in the method’s stability and ease of deployment.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\n- The paper is clearly written, with well-articulated motivation and methodology, making it easy to follow.\n\n- The proposed Knowledge Fission (KFI) and Domain Fusion (DFU) mechanisms significantly improve adaptation performance on ImageNet-to-ImageNet-C. This improvement is particularly notable in the repeating domains setting, where CTTA methods must preserve knowledge of previously seen domains.\n\n- The method achieves state-of-the-art performance while maintaining a compact set of learnable parameters, avoiding unnecessary model bloat.\n\nThe paper includes ablation studies demonstrating the effectiveness of KFI and DFU, along with sensitivity analysis for the prompt pool size, providing valuable insights into the model's behavior.\n\nWeaknesses\n- The method is primarily evaluated on ImageNet-to-ImageNet-C, supplemented by CIFAR10 and CIFAR100, which are relatively small-scale and arguably toy datasets. More compelling evaluation on large-scale CTTA benchmarks with natural domain shifts—such as Cityscapes-to-ACDC, as reported in DPCore (a key competing method)—would strengthen the empirical evidence.\n\n- The method introduces a large number of hyperparameters (e.g., γ_d, γ_c, γ_h, α_d, α_c, τ_d, τ_c, a, N_d, N_c). While sensitivity analysis was conducted for N_d and N_c, the influence of the remaining hyperparameters—especially γ and α parameters—on fission/fusion and knowledge retention is not discussed. This is particularly relevant because the method shows large gains on ImageNet-to-ImageNet-C with parameters tuned on a disjoint validation set from ImageNet-C. However, when transferred to CIFAR10 and CIFAR100 without further tuning, the gains over DPCore are minimal or marginal.\n\n- While the KFI and DFU components are novel, the overall CTTA approach is clearly inspired by DPCore, particularly in the use of dynamic prompt-based adaptation. A more thorough comparison or differentiation from DPCore would be helpful to better highlight the novelty.", "was_revised": true, "final_justification": "The author response addressed most of my concerns in my initial review by providing additional experiments on the Cityscapes-to-ACDC dataset as well as more extensive hyperparameter analysis.  Thus, I change my final rating from a weak accept to an accept.", "rebuttal": "Thank you sincerely for your insightful feedback and constructive suggestions, which have been invaluable in refining our work. We greatly appreciate your recognition of the method’s strengths in CTTA performance, parameter efficiency, and the clarity of our presentation. Addressing your key concerns, we have supplemented critical experiments and analyses to strengthen the paper, as detailed below:\n\n### **1. Evaluation on Larger, Real-World CTTA Benchmarks.**\n\nFollowing the common practice in the CTTA community, we have conducted fair comparisons with existing methods on ImageNet-to-ImageNet-C, CIFAR10, and CIFAR100. In addition, we further validated the effectiveness of our method on datasets with natural domain shifts (Cityscapes-to-ACDC). The experimental results are shown in Table A. The results demonstrate that **our method outperforms the existing SOTA method (DPCore) by 1.1%**. This is because our knowledge fusion and fission can adaptively adjust according to the shift degree of different domains and select the most appropriate prompt for optimization. The result on Cityscapes-to-ACDC dataset indicates that our knowledge fission and knowledge fusion still maintain high generalization on datasets with natural domain shifts.\n\nTable A: Average mIoU score(%) for Cityscapes-to-ACDC, with SegFormer-B5 model pre-trained on the Cityscapes dataset as backbone.\n|Methods|Round1|Round2|Round3|Mean|\n|-|-|-|-|-|\n|Source|56.7|56.7|56.7|56.7|\n|Tent|56.7|55.9|55.0|55.7|\n|CoTTA|58.6|58.6|58.6|58.6|\n|ViDA|61.1|62.2|62.3|61.9|\n|DPCore|62.1|62.6|62.8|62.5|\n|Ours|**63.1**|**63.6**|**64.0**|**63.6**|\n\n### **2. Analyze Sensitivity and Robustness of Hyperparameters (γ and α).**\n\nWe conducted further analyses on the influences of key hyperparameters (γ and α) on knowledge fission/fusion, with results summarized in Table B-E. Numbers with * represent the original default parameter tuned on the disjoint validation set from ImageNet-C.\n\nFor γ: As shown in cross-dataset experiments  (Table B,C), while the optimal γ value varies slightly between ImageNet-C, CIFAR10, and CIFAR100, **the performance remains stable within a reasonable range (±0.5%) across all datasets.** Excessively small or large γ degrades performance consistently: **a small γ introduces irrelevant knowledge, while a large γ limits knowledge reuse.** This consistent cross-dataset behavior highlights γ’s robustness: **though not universally optimal, its core tuning logic, balancing knowledge relevance and reuse, generalizes well, simplifying deployment via systematic adjustment rather than arbitrary search.**\n\nFor α: **Across all tested datasets, α shows minimal impact on performance.** Our experiments (Table D,E) demonstrate that varying α within a typical range (e.g., 0–0.5) leads to performance fluctuations of **less than 0.2%** on ImageNet-C, CIFAR10, and CIFAR100. This insensitivity stems from the fact that our knowledge fission mechanism selects similar prompts for updates, promoting the reuse and evolution of similar knowledge while balancing the consistency of knowledge before and after fission. Thus, **hyperparameter α has minimal impact on consistent knowledge, further verifying the robustness and generalization of our method.**\n\nRegarding the performance on CIFAR10 and CIFAR100: We have organized additional experiments to clarify this, as shown in Table F (and Appendix Tables 5/6). Without further parameter tuning, our method already achieves improvements of 2.6% and 3% over DPCore on CIFAR10 and CIFAR100, respectively. These are not marginal gains.\n\nFurthermore, with simple parameter tuning (specifically adjusting γ_c to a larger value and N_c to a lower value) on CIFAR10 and CIFAR100, the average performance on the two datasets is further improved, resulting in an average gain of 3.3% over DPCore (denoted as \"Ours+\"). This adjustment is reasonable because CIFAR10/100 have smaller data scales compared to ImageNet-C; a stricter γ_c, through more rigorous knowledge matching, better models the discriminative knowledge between samples of different classes. This further demonstrates the generalization of our method across datasets of varying sizes.\n\nTable B: Effect of γ_d across different datasets. Average error rate(%) for CTTA tasks.\n||10|15|20|25*|30|35|40|\n|-|-|-|-|-|-|-|-|\n|ImageNet-C|36.2|35.0|**34.8**|**34.8**|35.0|35.7|35.1|\n|CIFAR10-C|12.7|12.5|**12.4**|**12.4**|**12.4**|12.5|12.5|\n|CIFAR100-C|22.9|22.8|22.6|**22.5**|**22.5**|22.7|22.8|\n\nTable C: Effect of γ_c across different datasets. Average error rate(%) for CTTA tasks.\n||1e-4|1e-3|5e-3*|7e-3|1e-2|5e-2|1e-1|\n|-|-|-|-|-|-|-|-|\n|ImageNet-C|36.0|35.2|**34.8**|**34.8**|35.1|35.2|35.8|\n|CIFAR10-C|12.7|12.6|12.4|12.4|12.3|**12.2**|**12.2**|\n|CIFAR100-C|23.0|22.7|22.5|22.5|22.4|**22.1**|22.3|\n\nTable D: Effect of α_d across different datasets. Average error rate(%) for CTTA tasks.\n||0|0.05|0.1*|0.15|0.2|0.3|0.5|\n|-|-|-|-|-|-|-|-|\n|ImageNet-C|34.9|**34.8**|**34.8**|**34.8**|34.9|34.9|34.9|\n|CIFAR10-C|12.5|**12.4**|**12.4**|12.5|12.5|12.6|12.6|\n|CIFAR100-C|22.7|22.6|**22.5**|**22.5**|**22.5**|22.6|22.6|\n\nTable E: Effect of α_c across different datasets. Average error rate(%) for CTTA tasks.\n||0|0.05|0.1*|0.15|0.2|0.3|0.5|\n|-|-|-|-|-|-|-|-|\n|ImageNet-C|34.9|**34.8**|**34.8**|**34.8**|**34.8**|34.9|35.0|\n|CIFAR10-C|12.6|12.5|**12.4**|**12.4**|12.5|12.6|12.6|\n|CIFAR100-C|22.7|22.7|**22.5**|**22.5**|22.6|**22.5**|22.6|\n\nTable F: Average error rate(%) for CTTA tasks across different datasets.\n|Methods|CIFAR10-C|CIFAR100-C|Average|\n|-|-|-|-|\n|Source|28.2|35.4|31.8|\n|DPCore|15.4|25.1|20.3|\n|Ours|12.4|22.5|17.5|\n|Ours+|**11.9**|**21.8**|**16.9**|\n\n\n### **3. A more thorough comparison or differentiation from DPCore.**\n\nOur method differs from DPCore in three key aspects, with experimental evidence supporting the effectiveness of each innovation:\n\n1. **Knowledge Fusion Mechanism:** Unlike DPCore, which allows unrestricted prompt growth (leading to escalating model size and potential redundancy), we introduce a knowledge fusion mechanism that merges similar prompts during adaptation. This not only controls parameter explosion but also promotes the integration of analogous knowledge, enhancing the generality of learned representations. As shown in our ablation study (Table 3 in our paper), this mechanism alone yields a **performance improvement of 2.1%**, validating its role in balancing efficiency and knowledge utility.\n   \n2. **Class-Specific Prompts:** DPCore relies on a single type of prompts. In contrast, we introduce class prompts to explicitly capture discriminative features shared by the same/similar classes across domains. This design enables the model to leverage cross-domain commonalities of classes for better classification. Our ablation results (Table 3) demonstrate that adding class prompts contributes **an additional 4.7% improvement**, highlighting its effectiveness in exploiting inter-domain class-level consistency.\n   \n3. **Dynamic Prompt Selection Strategy:** DPCore aggregates all existing prompts via similarity-weighted fusion for each test batch, which inevitably introduces noise from irrelevant prompts. Instead, our method selects only a small subset of the most relevant prompts for fusion, significantly reducing noise interference. To verify this, we conducted a controlled experiment where we modified our method to adopt DPCore’s full prompt weighting strategy (denoted as \"Ours*\"), shown in Table G. The observed performance dropped **1.8%** in \"Ours*\" compared to our original design, which confirms that our selective fusion strategy is critical for avoiding noise and maintaining robustness.\n\nThese innovations collectively distinguish our approach from DPCore: by **controlling parameter growth, enhancing class-level knowledge utilization, and reducing noise when using prompts**, our method achieves superior performance as validated by both ablation studies and comparative experiments.\n\nTable G: Performance Impact of different Domain Prompt Selection Strategies: Ours vs. DPCore-style (denoted as Ours*)\n|Methods|Err Mean|\n|-|-|\n|Source|55.8|\n|DPCore|39.9|\n|Ours|**34.8**|\n|Ours*|36.6|"}, {"review_id": "9z1Q3lHcx7", "reviewer": "Reviewer_m3Qy", "summary": "The paper presents a Class-aware Domain Knowledge Fusion and Fission (KFF) framework for Continual Test-Time Adaptation (CTTA), addressing critical challenges of catastrophic forgetting and inefficient new knowledge assimilation. The key idea is to adaptively expand and merge class-aware domain knowledge from old and new domains based on test-time data, allowing discriminative historical knowledge to accumulate dynamically. The authors evaluate KFF on datasets like ImageNet-C, demonstrating its effectiveness against other methods in both non-repeating and repeating domain scenarios.", "questions": "1.\tIn the class prompt pool, not all potential labels are saved, as similar classes are clustered and updated with weighted averaging. Does this have an impact? In other words, if the prompt pool were structured to save prompts for potentially all categories based on pseudo-labels (with the pool size fixed to the total number of labels), would the performance decrease?\n\n2.\tAll experiments are based on Vision Transformer. Can the KFF method be transferred to other models such as ResNet or MobileNet?", "limitations": "The authors adequately addressed the limitations in the paper.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**:\n\n1.\tThe paper addresses challenges of catastrophic forgetting and insufficient new knowledge assimilation in CTTA.\n2.\tIt proposes a novel class-aware knowledge fusion and fission framework for adaptive knowledge management.\n3.\tThe method demonstrates state-of-the-art performance with high parameter efficiency in various continual test-time adaptation scenarios.\n\n**Weaknesses**:\n\n1.\tMore comparative experiments are needed. Methods applicable to CTTA scenarios, such as EATA[1] and BECoTTA[2], should be included in the comparative experiments.\n2.\tA comparison with knowledge fusion frameworks similar to KFF is necessary. A recent work, CoLA[3], proposes a plug-and-play TTA framework that performs knowledge fusion at the weight level. It could potentially achieve better improvements in CTTA scenarios when combined with existing baseline methods.\n3.\tLack of deeper analysis on knowledge fusion. While fusion can prevent the prompt pool from expanding indefinitely, it might also lead to the loss of subtle knowledge potentially valuable for future adaptation. The current ablation studies on N_d and N_c are too limited in scope, requiring more extensive analysis. \n4.\tAblation experiments need to be supplemented. KFF only updates prompts whose entropy values are below a certain threshold. Please provide an ablation study for this threshold.\n5.\tThe “Domain Knowledge Fission” section appears to share similarities with DPCore[A]. Please explain in more detail the differences between this part and DPCore.\n6.\tThe theoretical analysis (Section A) assumes well-separated clusters with intra-cluster distances smaller than inter-cluster distances. However, domain distributions in real-world scenarios often overlap. It would be helpful to analyze KFF’s performance under such overlapping scenarios to better validate its practicality.\n7.\tExperiments are primarily conducted on synthetic corrupted datasets (e.g., ImageNet-C, CIFAR100-C), with limited validation on real-world complex distribution shifts (e.g., varying illumination, weather, or device-captured natural images). Supplementing such experiments would strengthen the demonstration of practical utility.\n\n[A] DPCore: Dynamic Prompt Coreset for Continual Test-Time Adaptation. ICML 2025.\n\n[1] Efficient Test-Time Model Adaptation without Forgetting. ICML 2022.\n\n[2] BECoTTA: Input-dependent Online Blending of Experts for Continual Test-time Adaptation. ICML 2024.\n\n[3] Cross-Device Collaborative Test-Time Adaptation. NIPS 2024.", "was_revised": true, "final_justification": "The authors provide more clarification to demonstrate the robustness of hyper-parameters, and commit to including more discussions with related works. I maintain positive toward the paper.", "rebuttal": "Thank you for your detailed evaluation of our KFF framework and recognition of its strengths. The constructive points you raised are highly valuable. To address these, we conducted supplementary experiments and analyses detailed below:\n\n### **1. More comparative experiments.**\n\nWe further supplemented experiments comparing our method with EATA[34] and BECoTTA[24] on the ImageNet-to-ImageNet-C benchmark, with results presented in Table A.\n\nIt can be seen that our method achieves an average error rate reduction of **more than 10%** compared to both EATA and BECoTTA. This performance gain can be attributed to the limitations of the competing methods: EATA and BECoTTA do not account for the dynamic evolution of domain knowledge, which reduces the efficiency of knowledge utilization. In contrast, our knowledge fission and fusion mechanism not only significantly controls model size but also promotes the integration of similar knowledge, thereby enhancing the generality of learned representations. Additionally, our class-specific prompts and selective fusion strategy better leverage shared features of the same/similar classes across domains, further boosting classification accuracy. These results provide additional evidence for the effectiveness of our approach in CTTA scenarios.\n\nTable A: Average error rate(%) for ImageNet-to-ImageNet-C CTTA task across different methods.\n|Methods|Publish|Err Mean|\n|-|-|-|\n|Source|-|55.8|\n|EATA|ICML'22|45.4|\n|BECoTTA|ICML'24|47.7|\n|Ours|-|**34.8**|\n\n### **2. Comparison with knowledge fusion frameworks similar to KFF.**\n\nWe have conducted two supplementary experiments by integrating the knowledge fusion module proposed in CoLA[a]: (1) integrating CoLA’s knowledge fusion module into our baseline DPCore; (2) replacing our KFU module with CoLA’s fusion mechanism. The results were presented in Table B, which show that incorporating CoLA into the baseline yields only a 0.8% improvement, while replacing our KFU with CoLA leads to a 3.0% gain—both significantly lower than our method’s 5.1% enhancement. **Compared to these two setups, our approach achieves additional gains of 4.3% and 2.1%, respectively.**\n\nThe gap arises because CoLA aggregates weights unoptimizedly based solely on domain similarity, ignoring class-level relationships. In contrast, our KFF uses global similarity to merge both domain- and class-specific knowledge, better leveraging cross-domain shared features of same/similar classes—validating its superiority.\n\nTable B: Average error rate(%) for ImageNet-to-ImageNet-C CTTA task across different fusion strategies.\n|Methods|Err Mean|\n|-|-|\n|Source|55.8|\n|DPCore|39.9|\n|DPCore+CoLA|39.1|\n|Ours|**34.8**|\n|Ours+CoLA|36.9|\n\n[a] Cross-Device Collaborative Test-Time Adaptation. NeurIPS 2024.\n\n### **3. Deeper analysis on knowledge fusion.**\n\nWe have extended analyses on how N_d and N_c balance pool expansion control and retention of subtle, future-valuable knowledge. Experiments, spanning a broader parameter range under the \"CTTA with Repeating Domains\" setting (R1 and R10 for the first and 10th round), are in Tables C and D.\n\nResults show that **excessively small N restrict prompt diversity, losing subtle knowledge critical for future adaptation, while overly large N cause uncontrolled pool expansion, leading to redundant prompts poorly optimized by limited samples.**\n\nOur method selects optimal N_d/N_c, balancing these trade-offs to avoid both indefinite expansion and valuable knowledge loss, performing best in both R1 and R10.\n\nNotably, in R1, large N_d/N_c yield stable performance because the Knowledge Fission Module (KFI) generates too few prompts to trigger fusion, letting the framework dynamically avoid unnecessary early fusion and preserve initial subtle knowledge.\n\nIn summary, our extended analyses confirm that properly tuned N_d and N_c effectively prevent both unchecked pool expansion and loss of valuable subtle knowledge, validating the robustness of our knowledge fusion design.\n\nTable C: Effect of N_c. Average error rate(%) for ImageNet-to-ImageNet-C CTTA task.\n|N_c|50|100*|150|200|500|1000|2000|\n|-|-|-|-|-|-|-|-|\n|R1|34.9|**34.8**|**34.8**|**34.8**|35.2|35.2|35.2|\n|R10|35.1|**34.5**|34.7|34.8|35.3|35.5|36.6|\n\nTable D: Effect of N_d. Average error rate(%) for ImageNet-to-ImageNet-C CTTA task.\n|N_d|10|20*|40|50|60|100|200|\n|-|-|-|-|-|-|-|-|\n|R1|35.1|**34.8**|34.9|36.1|36.1|36.1|36.1|\n|R10|35.1|**34.5**|34.8|36.3|36.8|36.8|37.0|\n\n### **4. Ablation experiments for γ_h.**\n\nWe further conducted an ablation study on the entropy threshold (γ_h), with results in Table E. The analysis reveals that γ_h impacts performance through two key mechanisms: **While too low γ_h reduces valid samples for updates, hindering adaptation, too high γ_h includes high-entropy (low-confidence) samples as noise, hurting performance**. This finding is consistent with EATA [34], which also demonstrated that adaptation on test samples with very high entropy may hurt performance.\n\nOur method selects an optimal γ_h that balances these trade-offs, ensuring sufficient valid samples for updates while avoiding noise from high-entropy instances, thereby achieving robust performance. This ablation study confirms the effectiveness of our threshold selection strategy.\n\nTable E: Effect of γ_h, we choose 2 in our paper. Average error rate(%) for ImageNet-to-ImageNet-C CTTA task.\n|γ_h|1|1.5|2*|2.5|3|3.5|4|\n|-|-|-|-|-|-|-|-|\n|Ours|35.0|**34.8**|**34.8**|35.0|35.1|35.4|35.8|\n\n### **5. Explain the differences between Domain Knowledge Fission and DPCore.**\n\nIn terms of Domain Knowledge Fission, **the core distinction from DPCore lies in our Domain Prompt Selection Strategy**:\n\nDPCore aggregates all existing domain prompts via similarity-weighted fusion during adaptation, which inevitably introduces noise from irrelevant domain prompts. In contrast, our Domain Knowledge Fission selectively picks only the most relevant domain prompts to guide the generation of new domain prompts, significantly reducing noise interference. A controlled experiment (Table F) shows that adopting DPCore’s full domain prompt weighting strategy (\"Ours*\") leads to a 1.8% performance drop, validating the effectiveness of our selective domain prompt selection in fission.\n\nBeyond fission, our method differs from DPCore in two other key aspects: (1) We introduce class prompts (absent in DPCore) to explicitly capture cross-domain class-level commonalities, boosting performance by 4.7% (ablation in Table 3). (2) Our prompt fusion merges similar prompts to avoid redundancy, whereas DPCore’s prompt pool grows unbounded, alleviating noise accumulation and parameter explosion.\n\nTable F: Performance Impact of different Domain Prompt Selection Strategies: Ours vs. DPCore-style (denoted as Ours*).\n|Methods|Err Mean|\n|-|-|\n|Source|55.8|\n|DPCore|39.9|\n|Ours|**34.8**|\n|Ours*|36.6|\n\n### **6. Analyze KFF’s performance under overlapping scenarios.**\n\nWe supplemented experiments on overlapping domain distributions, following CCC[38] to construct scenarios with controlled feature overlap between domains.\n\nResults (Table G) show our method outperforms previous SOTA DPCore by **3.7%** on average under such conditions. This shows our knowledge fission/fusion mechanisms can still accurately distinguish and integrate knowledge types despite overlaps, maintaining strong generalization, validating KFF’s practicality for real-world scenarios.\n\nTable G: Average error rate(%) on ImageNet-C, CCC-Medium and CCC-Hard.\n|Methods|ImageNet-C|CCC-Medium|CCC-Hard|\n|-|-|-|-|\n|Source|55.8|58.0|78.0|\n|RDumb|45.8|49.8|76.2|\n|CoTTA|49.9|52.5|79.5|\n|DPCore|39.9|43.2|74.1|\n|Ours|**34.8**|**37.2**|**72.7**|\n\n### **7. Experiments on real-world complex distribution shifts.**\n\nWe validated the effectiveness of our method on datasets with natural domain shifts (Cityscapes-to-ACDC). The experimental results are shown in Table H, which demonstrate that **our method outperforms the existing SOTA method (DPCore) by 1.1%**. This is because our knowledge fusion and fission can adaptively adjust according to the shift degree of different domains and select the most appropriate prompt for optimization, which indicates that our knowledge fission and knowledge fusion still maintain high generalization on datasets with real-world complex distribution shifts.\n\nTable H: Average mIoU score(%) for Cityscapes-to-ACDC, with SegFormer-B5 model pre-trained on the Cityscapes dataset as backbone.\n|Methods|round1|round2|round3|Mean|\n|-|-|-|-|-|\n|Source|56.7|56.7|56.7|56.7|\n|Tent|56.7|55.9|55.0|55.7|\n|CoTTA|58.6|58.6|58.6|58.6|\n|ViDA|61.1|62.2|62.3|61.9|\n|DPCore|62.1|62.6|62.8|62.5|\n|Ours|**63.1**|**63.6**|**64.0**|**63.6**|\n\n### **Q1. Would the performance decrease if the prompt pool were based on pseudo-labels?**\n\n**Yes, saving all class prompts via pseudo-labels reduces performance: Appendix Table 4 shows a 1% accuracy drop**, as pseudo-labels fail to capture shared traits among similar classes, resulting in suboptimal knowledge representation. By contrast, our design leverages inter-class commonalities, achieving the highest accuracy among compared approaches.\n\n### **Q2. Can the KFF method be transferred to other models such as ResNet or MobileNet?**\n\nOur KFF method, like many other prompt-based approaches (e.g. DePT[12], DPCore[52]), is initially designed around Vision Transformers (ViTs)[10, b] due to their token-based structure, which naturally supports prompt integration. **Conventional CNNs (e.g., ResNet, MobileNet) lack such token mechanisms, making direct adaptation of existing prompt frameworks challenging.**\n\nTo align with common practices in CTTA and ensure fair comparisons (most SOTAs[25, 26, 52] use ViTs), we adopt ViTs as the backbone. However, we plan to explore CNN-compatible prompt designs in future work to verify KFF’s transferability across architectures.\n\n[b] SegFormer: Simple and Efficient Design for Semantic  Segmentation with Transformers. NeurIPS 2021."}, {"review_id": "U3X7zSA356", "reviewer": "Reviewer_TUaD", "summary": "While prior CTTA methods mainly focus on mitigating catastrophic forgetting, they tend to suffer from insufficient learning of new domain knowledge and interference from outdated or harmful historical information. To address these limitations in CTTA, the authors propose KFF, a class-aware method that dynamically expands and merges domain knowledge at test time. The proposed method achieves strong performance with a low number of parameters, as demonstrated by extensive experiments on the ImageNet-C benchmark.", "questions": "- Misc) Could you explain more about Fig1 (c)? Which accuracy was decreased? I already got the authors point, but still have some vague point. \n- For others, please refer to weakness.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "1. Strengths\n\n- This paper is overall well-organized, well-motivated, and easy to follow. The authors highlight an important point: adaptation to new domains can sometimes be more crucial than preserving knowledge from past domains—an aspect that current CTTA works often overlook. Their proposed knowledge fusion/fission approach based on prompt learning is novel and intuitive.\n- The experimental results are thorough and informative. Additionally, the proposed model is parameter-efficient compared to other CTTA baselines (similar to Tent), while achieving the lowest error rate. This is impressive.\n\n2. Weaknesses\n\n- While the model is indeed parameter-efficient, parameter number is not the only metric for efficiency. Could the authors also provide a comparison of inference time and memory usage with other baselines? For example, Tent is known for being fast and memory-efficient (from my experience), as it only modifies the batch normalization layers. In particular, comparisons with other prompt-based CTTA models would offer valuable insight.\n\n- In Table 2, KFF performs well in maintaining initial performance (R1) compared to other baselines. However, the repeated performance levels appear to be merely preserved without noticeable improvement. Is this due to a strong reliance on source domain statistics? As I understand it, KFF focuses on acquiring new domain knowledge more actively than other methods, making the knowledge fission mechanism particularly relevant. A deeper analysis from the authors on this point would be appreciated.", "was_revised": true, "final_justification": "Thanks for the author's rebuttal and effort. My concern regarding efficiency and repetitive scenarios has been resolved. I will maintain my current score, considering other reviewers' comments. I recommend that authors add more comparison with EATA and BECoTTA in their final version of the paper.", "rebuttal": "Thank you sincerely for your thoughtful assessment of our KFF method and your recognition of its strengths. Your constructive feedback, particularly regarding efficiency metrics beyond parameters and the analysis of repeated domain performance, is incredibly helpful. Addressing these, \nwe’ve made additional experiments and clarifications to strengthen the paper, as detailed below:\n\n### **1. More comparision on efficiency.**\n\nWe have supplemented comparisons of learnable parameter count, GPU memory usage, and computational efficiency with baselines, evaluated on a single NVIDIA 4090 GPU. Results are provided in two tables: table A for the first round and table B for the 10th round under the repeating domains' setting.\n\n1. **Comparison with Tent.**\nTent, which only modifies batch normalization layers, exhibits superior raw efficiency (faster computation, lower memory). However, our method delivers a 15.2% performance improvement with little additional cost, making this trade-off worthwhile.\n\n2. **Comparison with other baseline models.**\nAgainst CoTTA and ViDA, our method outperforms them in both efficiency and performance.\nAgainst DPCore, our method achieves a 5.1% performance gain while maintaining comparable efficiency: computation cost, memory usage, and parameter count remain on par in early rounds.\n\n1. **Advantage of long-term adaptation.**\nA critical efficiency edge emerges in later rounds (R10, shown in Table B): our KFI module stops generating new prompts when no new domains appear, reducing computational load over time. In contrast, DPCore, lacking fusion and prompt selection, sees escalating parameters, slower inference, and higher memory usage due to unconstrained prompt accumulation (accompanied by rising error rates). This validates our design’s efficiency and stability in continual test-time adaptation.\n\nOverall, our method balances performance gains with efficiency, outperforming both prompt-based and non-prompt-based baselines in performance while maintaining comparable efficiency, and offering far greater performance improvements than ultra-lightweight methods like Tent with minimal additional cost.\n\nTable A: Computational analysis on ImageNet-to-ImageNet-C.\n|Methods|Learnable Params|Relative Time|Memory Usage|Avg TFLOPs/batch|Err Mean|\n|-|-|-|-|-|-|\n|Tent|0.03M|1.0|5.5GB|1.08|51.0|\n|CoTTA|86.57M|4.7|16.2GB|3.24|49.9|\n|ViDA|93.70M|5.3|9.3GB|14.03|43.4|\n|DPCore|0.08M|1.6|5.7GB|1.67|39.9|\n|Ours|0.09M|1.9|6.0GB|1.68|34.8|\n\nTable B: Computational analysis on ImageNet-to-ImageNet-C with repeating domains, round 10.\n|Methods|Learnable Params|Relative Time|Memory Usage|Avg TFLOPs/batch|Err Mean|\n|-|-|-|-|-|-|\n|Tent|0.03M|1.0|5.5GB|1.08|99.9|\n|DPCore|1.03M|2.1|8.4GB|1.73|46.8|\n|Ours|0.20M|1.8|6.1GB|1.60|34.5|\n\n### **2. A deeper analysis on why the repeated performance levels appear to be merely preserved without noticeable improvement.**\n\n**The stable performance across repeated rounds does not stem from the strong reliance on source statistics.** **In our setup, R1 samples already provide sufficient information for the model to capture robust patterns, establishing a strong baseline, and our methods could maintain that performance.** This is a deliberate strength, as many baselines struggle to even maintain initial performance due to the inevitable mixing of conflict domain knowledge (e.g., Tent drops by 48.9% from R1 to R10, while DPCore drops by 6.9% in Table 2 in our paper).\n\nOur knowledge fission/fusion mechanisms focus on retaining and refining knowledge rather than forcing unnecessary \"improvement\" when domains stabilize. When we reduced per-domain samples to 1/10, as shown in Table C, clear improvements emerged across rounds, showcasing KFF’s ability to actively acquire new knowledge when needed. This validates its balance of stability (with sufficient data) and adaptability (with limited data).\n\nTable C: Classification error rate(%) for ImageNet-to-ImageNet-C online CTTA task in 10 repeated rounds with limited samples(R1-R10).\n|Methods|R1|R2|R3|R4|R5|R6|R7|R8|R9|R10|Avg|\n|-|-|-|-|-|-|-|-|-|-|-|-|\n|Source|50.5|50.5|50.5|50.5|50.5|50.5|50.5|50.5|50.5|50.5(+0.0%)|50.5|\n|CoTTA|49.8|49.9|50.1|50.2|50.3|50.6|50.6|50.8|50.9|50.9(-1.1%)|50.4|\n|ViDA|46.6|43.7|42.5|42.3|42.8|42.6|42.4|42.3|42.2|42.1(+4.5%)|43.0|\n|DPCore|49.9|46.6|45.7|45.8|45.6|45.3|45.7|45.4|45.4|45.3(+4.5%)|46.1|\n|Ours|43.4|40.2|39.1|38.3|37.6|37.1|36.7|36.5|36.5|**35.8(+6.9%)**|**38.1**|\n\n### **3. Explain more about Fig1 (c).**\n\nFig. 1(c) illustrates the performance of existing methods during a domain change: fog → bright → fog. Specifically, **when the model encounters the fog domain again (after adapting to the bright domain), its classification accuracy drops sharply by 14.6% compared to the first fog domain**, even when tested on identical fog samples.\n\nThis decline indicates that knowledge learned from the bright domain conflicts with prior knowledge from the fog domain, impairing the model’s ability to reuse previously acquired fog-specific patterns. In contrast, our method mitigates such conflicts through knowledge fission strategy, preserving accuracy across repeated domains, which further validates its effectiveness."}, {"review_id": "J10nbyVY9h", "reviewer": "Reviewer_8gyU", "summary": "This paper proposes a visual prompt tuning-based method for Continual Test-Time Adaptation (CTTA). Specifically, the method introduces a class prompt pool and a domain prompt pool. A fusion-and-fission mechanism is designed to update these pools by expanding prompts for new classes and domains, and refining existing prompts for closely related classes and domains. Experiments are conducted on commonly used benchmark datasets in the CTTA literature to evaluate the proposed method.", "questions": "1. Could the authors elaborate on the conceptual and technical contributions of the proposed method in comparison to prior visual prompt tuning-based CTTA methods, such as DPCore?\n2. As mentioned in Weakness 2, I recommend conducting a more comprehensive efficiency analysis beyond the limited parameter count comparison.\n3. How is the data stream constructed? Is it sampled in an independent and identically distributed manner, similar to regular training? Have the evaluations considered temporally correlated data streams, as in baselines like RoTTA [47] and SAR [35]?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n1. The method overview figure is well-illustrated and effectively conveys the core idea.\n2. The evaluations on benchmark datasets demonstrate that the proposed method outperforms recent baselines, indicating its effectiveness.\n\n\n**Weaknesses**\n\n1. The method appears technically incremental compared to previous visual prompt tuning-based CTTA approaches, such as DPCore [52]. It mainly extends DPCore by introducing a class prompt pool in addition to the domain prompt pool, and replaces the Online K-Means clustering in DPCore with MST-based clustering for prompt grouping. These seem like modest extensions, making it difficult to identify significant technical contributions.\n2. Since the proposed method introduces additional operations during online adaptation and appears more complex than the baselines, it is important to clarify the performance-efficiency trade-off through a comprehensive computational analysis. However, the paper only reports learnable parameter counts, which is especially insufficient for prompt-tuning-based methods. I recommend including additional metrics such as memory usage, runtime, and computation cost (e.g., FLOPs).", "was_revised": true, "final_justification": "This work designs a more comprehensive prompt pool method for continual test-time adaptation and achieves better performance compared with the previous prompt pool-based method (DPCore [52]), albeit with increased computational burden. Given the comprehensive evaluation and the accuracy improvements demonstrated, I increase my rating to 4.\n\nThe reason I do not lean toward a higher score is that the “prompt pool” has been a classic technique in continual learning for several years, and this paper is not the first to apply it to continual test-time adaptation. While the technical improvement over DPCore is effective, it is not particularly impressive at the conceptual level.", "rebuttal": "Thank you for your detailed review and recognition of our work’s strengths. Your valuable feedback on technical positioning relative to DPCore, comprehensive efficiency analysis, and clarifications on data stream construction (including model performance under different modes) is deeply appreciated. We’ve made additional experiments and analyses to address your concerns as well as strenghten the paper, detailed below:\n\n### **1. Elaborate on the conceptual and technical contributions of the proposed method in comparison to DPCore.**\n\nWe respectfully clarify that our contributions extend beyond incremental modifications to DPCore, with key distinctions addressing critical limitations in prior prompt-based CTTA methods:\n\nFirst, a correction: **DPCore does not employ Online K-Means clustering for prompts**, and its prompt pool grows unbounded without grouping mechanisms, leading to redundancy and noise accumulation. In contrast, our MST-based clustering is not a \"replacement\" but a novel design to actively merge similar prompts, which directly mitigates the parameter explosion and noise issues in DPCore.\n\nSecond, the introduction of a class prompt pool is not merely an \"additional\" component. **It explicitly models class-level shared knowledge across domains, a dimension entirely absent in DPCore’s domain-only prompts.** This enables our method to leverage cross-domain class commonalities, a capability validated by our ablation study in Table 3, showing **significant performance gains (4.7%)** from this design.\n\n**Together with our knowledge fusion mechanism and class prompt introduction, these innovations form a cohesive framework that outperforms DPCore by 5.1%, which is a substantial improvement.** These designs address challenges in prompt-based CTTA (scalability and knowledge utilization), establishing clear technical contributions.\n\n### **2. A more comprehensive efficiency analysis beyond the limited parameter count comparison.**\n\nWe fully agree that a comprehensive computational analysis is critical to clarify the performance-efficiency trade-off, especially for prompt-tuning-based methods. To address this, we have supplemented detailed comparisons across multiple metrics, including learnable parameters, memory usage, runtime, and FLOPs, evaluated on a single NVIDIA 4090 GPU . Results are presented in two tables: one for ImageNet-to-ImageNet-C (first-round adaptation) and another for the 10th round under the repeating domains setting.\n\nOur findings demonstrate that despite introducing additional operations during online adaptation, **our method achieves a favorable balance between performance and efficiency:**\n\n- Compared to baselines like CoTTA and ViDA, our method leads in both efficiency and performance.\n- Against DPCore, we achieve a 5.1% performance gain with comparable efficiency metrics.\n- Compared to Tent, which only modifies batch normalization layers, has superior raw efficiency, our method achieves a 15.2% higher performance gain with little additional cost, a worthwhile trade-off.\n\n**A key efficiency advantage emerges in long-term adaptation:** In later rounds (e.g., R10), our KFI module stops generating new prompts when no new domains appear, reducing computational load over time. In contrast, DPCore—lacking fusion and prompt selection—suffers from escalating parameters, FLOPs, and runtime due to unconstrained prompt accumulation, alongside rising error rates. This dynamic efficiency gain further validates our design’s practicality for continuous adaptation scenarios.\n\nThese results confirm that our method’s additional operations are justified by its performance gains and long-term efficiency stability, making it suitable for real-world deployment.\n\nTable A: Computational analysis on ImageNet-to-ImageNet-C.\n|Methods|Learnable Params|Relative Time|Memory Usage|Avg TFLOPs/batch|Err Mean|\n|-|-|-|-|-|-|\n|Tent|0.03M|1.0|5.5GB|1.08|51.0|\n|CoTTA|86.57M|4.7|16.2GB|3.24|49.9|\n|ViDA|93.70M|5.3|9.3GB|14.03|43.4|\n|DPCore|0.08M|1.6|5.7GB|1.67|39.9|\n|Ours|0.09M|1.9|6.0GB|1.68|34.8|\n\nTable B: Computational analysis on ImageNet-to-ImageNet-C with repeating domains, round 10.\n|Methods|Learnable Params|Relative Time|Memory Usage|Avg TFLOPs/batch|Err Mean|\n|-|-|-|-|-|-|\n|Tent|0.03M|1.0|5.5GB|1.08|99.9|\n|DPCore|1.03M|2.1|8.4GB|1.73|46.8|\n|Ours|0.20M|1.8|6.1GB|1.60|34.5|\n\n### **3. Clarifications on data stream construction and evaluations considered temporally correlated data streams.**\n\n**Following common practices in CTTA, our data stream is constructed via independent and identically distributed (i.i.d.) sampling, consistent with regular training protocols.** This aligns with most baselines to ensure fair comparisons.\n\nAdditionally, we further supplemented experiments on temporally correlated data streams, **following the \"practical TTA\" setting in RoTTA[47]**. Results in Table C show that our method achieves a further **4.7% improvement** in this scenario.\n\nThis performance gain stems from the ability of our knowledge fission and fusion mechanisms to model temporal dependencies: fission captures time-varying knowledge, while fusion preserves stable temporal patterns across sequential data. These results validate the generalization of our method to real-world temporally correlated streams.\n\nTable C: Average error rate(%) over practical TTA settings on ImageNet-to-ImageNet-C.\n|Methods|Published|Err Mean|\n|-|-|-|\n|Source|-|55.8|\n|RoTTA|CVPR'23|49.5|\n|SAR|ICLR'23|44.0|\n|DPCore|ICML'25|43.9|\n|Ours|-|**39.2**|"}]

Author rebuttal:

Meta-review: The paper proposes a continual test-time adaptation method that separates new domain knowledge from old knowledge to reduce interference across the domains, and later merges them to control complexity and retain useful information. Reviewers in general agreed that the paper is clearly written, the tackled problem is important, and that the proposed method shows strong performance across standard corrupted datasets as well as more realistic domain-shift scenarios, while remaining efficient. While there were some concerns raised by the reviewers, the rebuttal sufficiently addressed them with additional experiments, sensitivity analyses, and comparisons to additional baselines, clarifying robustness and efficiency trade-offs, which led to all reviewers leaning toward acceptance. Yet, there were remaining concerns regarding too many hyperparameters, limited novelty and efficiency improvement over existing works. Overall, the work makes steady progress on a timely problem, with thorough experiments and solid improvements, making it a borderline accept case.
Decision: Accept (poster)