DynaPrompt: Dynamic Test-Time Prompt Tuning
Authors: Zehao Xiao
Shilin Yan
Jack Hong
Jiayin Cai
Xiaolong Jiang
Yao Hu
Jiayi Shen
Cheems Wang
Cees G. M. Snoek
Venue: ICLR 2025 Poster (2025)
Keywords: Test-time prompt tuning; test-time adaptation; vision-language model; CLIP
Primary area: transfer learning, meta learning, and lifelong learning

Abstract: Test-time prompt tuning enhances zero-shot generalization of vision-language models but tends to ignore the relatedness among test samples during inference. Online test-time prompt tuning provides a simple way to leverage the information in previous test samples, albeit with the risk of prompt collapse due to error accumulation. To enhance test-time prompt tuning, we propose DynaPrompt, short for dynamic test-time prompt tuning, exploiting relevant data distribution information while reducing error accumulation. Built on an online prompt buffer, DynaPrompt adaptively selects and optimizes the relevant prompts for each test sample during tuning. Specifically, we introduce a dynamic prompt selection strategy based on two metrics: prediction entropy and probability difference. For unseen test data information, we develop dynamic prompt appending, which allows the buffer to append new prompts and delete the inactive ones. By doing so, the prompts are optimized to exploit beneficial information on specific test data, while alleviating error accumulation. Experiments on fourteen datasets demonstrate the effectiveness of dynamic test-time prompt tuning.

Published as a conference paper at ICLR 2025
DYNAPROMPT: DYNAMIC TEST-TIME PROMPT TUNING
Zehao Xiao', Shilin Yan', Jack Hong². " Jiayin Caf², Xiaolong Jiang', Yao Hu²,
Jiayi Shen'; Qi Wang', Cees G. M. Snoek'
'AIM Lab, University of Amsterdam
Xiaohongshu Inc. Depaarmennt of Automation, Tsinghua University
ABSTRACT
Test-time prompt tuning enhances zero-shot generalization of vision-language models but tends ignore the relatedness among test samples during inference. Online test-time prompt tuning provides a simple way to leverage the information in previous test samples, albeit with the risk of prompt collapse due to error accumulation. To enhance test-time prompt tuning, we propose DynaPrompt, short for dynamic test-time prompt tuning, exploiting relevant data distribution information while reducing error accumulation. Built on an online prompt buffer, DynaPrompt adaptively selects and optimizes the relevant prompts for each test sample during tuning. Specifically, we introduce dynamic prompt selection strategy based on two metrics: prediction entropy and probability difference. For unseen test data information, we develop dynamic prompt appending. which allows the buffer to append new prompts and delete the inactive ones. By doing so, the prompts are optimized exploit beneficial information on specific test data, while alleviating error accumulation. Experiments on fourteen datasets demonstrate the effectiveness of dynamic test-time prompt tuning."
INTRODUCTION
Despite achieving remarkable successes, foundation models such as Contrastive Language-Image Pretraining (CLIP) (Radford et al., 2021) still suffer from distribution shifts when adapting to downstream tasks (Zhou al., 2022a;b; Xiao et al., 2024). To improve test-time adaptation of the model in the presence of distribution shifts, recent works introduce learnable prompts at test time. The methods freeze the CLIP model parameters while only tuning the learnable prompts for test data. As shown in Figure 1a, test-time prompt tuning (TPT) (Shu et al., 2022) adapts the prompt to each test sample individually, which is widely followed by recent works (Ma et al., 2023; Samadh et al., 2023; Yoon et al., 2024). However, tuning in such way ignores the relatedness among test samples, which offers rich information on the test data distribution. To incorporate the information from relevant test samples, one straightforward method is to follow previous test-time adaptation methods (Wang et al., 2021; Goyal et al., 2022) and tune the test prompts online (Figure 1b). This encourages the prompts exploit previous test information for better model adaptation. However, as detailed later on in this paper, we establish that online test-time tuning leads to severe prompt collapse due error accumulation.
To have test-time prompt tuning benefit from relevant online information while reducing error accu- mulation, we constitute the concept of dynamic test-time prompt tuning, abbreviated as DynaPrompt. Specifically, DynaPrompt adaptively selects and optimizes the relevant online prompts for each sample while freezing the rest, yielding effective adaptation for the entire test set. As illustrated in Figure Ic, a prompt buffer is involved, which enables set of online prompts to be flexibly selected, updated, and appended for each test sample. To ensure the appropriate prompt selection without collapse in DynaPrompt, we devise comprehensive selection strategy with two metries: prediction entropy and probability difference, which measure the model uncertainty of the predictions and the model sensitivity to the input changes. Then construct the prompt screening threshold from these two metrics to achieve adaptive selection for different test samples. Such a screening rule selects the prompts with lower prediction entropy and larger probability differences and prefers those with
"Correspodingg author. Codess available at https: / /tte::////tttta htt.//tb..ccc////// wtw.m///zzlllc 1244 Promp..

---

Published as a conference paper at ICLR 2025
Vo
Vo
Cisa
Zioal E(si)
Zipo)
E(x))
2(pa)
(a) Test-time Prompt Tuning (Shu et al. 2022)
V
V2
V3
VA
V8
VO Pppersl
Vo
Vo
Vo
Apponst
VO
Paperal
(b) Online Test-time Prompt Tuning
(c) Dynamic Test-time Prompt Tuning (This paper)
Figure I: Illustrations of different test-time prompt tuning methods. Circles with specific colors denote learned prompts for individual test samples. For single test sample, (a) test-time prompt tuning learns prompts from a shared initialization vo, ignoring the relatedness among test samples. (b) online test-time prompt tuning incorporates previous test sample information by using the previous- sample optimized prompt as the starting point, which leads to eтror accumulation. (c) we propose dynamic dest-time prompt tuning (top) to adaptively exploit relevant information from previous test samples and alleviate error accumulation, which is achieved by autonomously selecting, updating, appending, and deleting online prompts prompt buffer (bottom).
higher certainty in the sample predictions and more sensitivity to structural changes of the input. As result, these selected prompts tend to be more relevant to the test sample while preventing collapse.
To incorporate new prompts in the prompt buffer for unexplored test data information, we include the operation of dynamic prompt appending in DynaPrompt. In the case of relevant prompts available in the prompt buffer, DynaPrompt autonomously appends new online prompts and deletes the inactive ones. Through adaptively selecting, updating, appending, and deleting prompts, the online prompts are tailored and optimized for test samples with relevant data information. this way, the predictions the test samples are enhanced by leveraging relevant online information, while the eгror accumulation is reduced by flexible prompt updates in the prompt buffer.
Empirically, we conduct experiments fourteen benchmarks, covering typical evaluation scenarios such as domain generalization and cross-dataset. The results show the effectiveness of the proposed method. Moreover, our method can be seamlessly incorporated into most prompt-tuning methods (Zhou et al., 2022b; Khattak et al., 2023) to enhance their performance further.
2 PRELIMINARY
We first provide a brief preliminary on the CLIP model, prompt learning, and test-time prompt tuning, containing background and commonly used techniques for test-time prompt tuning.
CLIP model (Radford et al., 2021). This pretrained model consists of an image encoder Fo, (-) and text encoder Faz(( (-), where O, and Or denote pre-trained parameters of the corresponding encoders. The image and text encoders take input image and text prompts as inputs, respectively. Given downstream classification task with a set of class names, CLIP performs zero-shot classification on each input image Specifically, CLIP gets the image feature fx Fe, (x) and text features (i.c., zero-shot classifier) as [fe, [f.
C is the number of class names and t. is manual- crafted text prompt corresponding to class C, e.g., "a photo of [class c]." The probability of exp(cos((f..f..)1
belonging to class c is p0O = c(x) = E exp(cux((f..f__/55 where cos(..) :) denotes cosine similarity and a learned temperature. Thus, CLIP directly obtains the prediction arg max p(g c(x).
Prompt learning. To further enhance the adaptation of CLIP downstream tasks, recent methods, e.g. (Zhou et al., 2022a;b; Khattak et al., 2023), introduce learnable prompts v = [0,][0>] -.. [u,]. while freezing the parameters of CLIP's encoders. Zhou et al. (2022b) introduce learnable prompts
2

---

Published as a conference paper at ICLR 2025
TPT
Acc:54.77%
60
Online TPT
Acc:6.96%
40
Oracle
AcC:59.38%
20
This paper
Acc:56.17%
0
1000
2000
3000
4000
5000
6000
7000
Online test samples
Figure 2: Prompt collapse in online test-time prompt tuning, We measure online test-time accuracy for different methods on ImageNet-A, where the accuracy for each block calculated on 200 samples. Test-time Prompt Tuning (TPT) achieves stable accuracy by independently tuning the prompts for each test sample. Online TPT has severe eтror accumulation problems with competitive performance the beginning while dropping significantly during online learning. Oracle is more stable by online tuning prompts only for correct predictions and achieves much better performance by incorporating the relevant information. Our method aims to exploit relevant information from previous online samples automatically while reducing error accumulation.
change the "a photo of a", making the prompts as t. [v]]class c]. For prompt tuning, these methods rely a small-scale training set of the downstream task containing input images and their corresponding class names, i.e., (x, Vg where V/: E 21. C). the ground-truth, During training, the model optimizes the learnable prompts v by minimizing the cross-entropy loss, i.e., min - log p(ğ = -yg |x, v).
Test-time prompt tuning. Due to the existence of distribution shifts time, conventional prompt tuning methods usually suffer from overfitting, thus degrading the generalization ability of CLIP (Xiao et al., 2024). To address this issue, test-time prompt tuning (Shu et al., 2022: Samadh et al., 2023; Ma et al., 2023) are developed to independently tune each test sample's 's learnable prompts at time. Here, every sample tunes the prompts from a shared initial state Vo. which can be manually crafted (Shu et al., 2022) or pre-trained (Zhou et al., 2022b; Khattak et al., 2023). Since the labels are not available test time, the optimization prompts is guided by the prediction entropy of the selected sample augmentations:
min Cene (V; = min Σ p(ū c| log c|X,, v),
(I)
where X,, denotes selected augmentations with lower prediction entropy for a test sample x,, (Shu et al., 2022; Samadh et al., 2023) and p(ù = c|X,.. v) is an averaged prediction probability across selected augmentations.
It is worth noting that most test-time tuning methods independently adjust the prompt for each test sample, ignoring the exploitation of relevant information in previous samples. To explore the benefits of the relevant information, we dive into test-time prompt tuning in an online manner.
3 PROMPT COLLAPSE IN ONLINE PROMPT TUNING
In several real-world applications, there are a large number of related test samples that arrive sequentially. In such cases, earlier observed samples can provide beneficial information about the test distribution and reserve the potential to improve the prediction for subsequent samples. Inspired by this insight, we propose to extend test-time prompt tuning (Shu al., 2022) to online scenarios, formulating online test-time prompt tuning (Online TPT).
Online TPT retains most of the setups TPT (Shu et al., 2022), where prompts are obtained through one-step optimization using entropy minimization for each test sample. As illustrated in Figure 1, the primary distinction lies in the initialization the prompt for each test sample. While TPT (Shu et al., 2022) resets the prompt the initial state Vo for each sample, Online TPT uses the optimized prompt from the previous sample as the starting point for the current sample.
As shown in Figure 2, Online TPT performs competitively with TPT initially, but the performance declines rapidly, nearly approaching 0% at the end. This work refers to this phenomenon as prompt collapse, where the prompt tends to accumulate excessive noise and prevent the model from making
3

---

Published as a conference paper at ICLR 2025
Stage m
Stage P
piat 8.)
Frimpyy
Enrreyy
Xexe
entrcyy Tramaaaa
Sm.
S.
Sn
Probubiily by
Probubelly
Optimiittion and prediction with Dynamis prompt appanding
Vm
Dylame prompt selection
Optimizaionn and predittonVvsss
Vn
Dynamio prompt selection
Va-1
Figure 3: selection, we select the relevant online prompts from the prompt buffer V for each test sample using the intersection of the prompt subsets obtained by entropy and probability difference metrics. The selected prompts are optimized by entropy minimization before making predictions. (Right) If no prompt is selected, our dynamic prompt appending strategy assigns new prompt initialized by vo for the test sample and appends it to the prompt buffer. We always append the new optimized prompt on top of the buffer, moving the inactive ones the bottom, which we can remove directly when appending new prompts to the full buffer.
The process of our dynamic test-time prompt tuning. (Left) In dynamic prompt
accurate predictions. We attribute this to the entropy-based objectives, which might drive the opti- mization in the wrong directions without ground truth labels (Lee et al., 2024). The errors introduced during these optimization steps accumulate throughout online learning. Such error accumulation causes the prompts progressively degenerate, resulting in gradually degraded performance in Figure 2. Notably, the existence of error accumulation will ultimately cause collapsed prompts that generate incorrect predictions with lower entropies.
To further understand the relevant online information's 's influence on prompt collapse, we implement an Oracle method for online test-time prompt tuning. The method follows most setups in Online TPT. By introducing the ground truth, Oracle only updates the prompts with correct predictions and skips the incorrect ones, thereby circumventing error accumulation from incorrect predictions. Still, entropy minimization works as the objective function for prompt tuning, the same as TPT and Online TPT. In Figure 2, Oracle considerably outperforms online TPT, confirming that error accumulation of entropy minimization with incorrect predictions hurts online test-time prompt tuning. Meanwhile, Oracle also outperforms TPT, which implies that the relevant information in online test samples benefits prompt tuning on the test distribution. All of the above observations motivate to develop DynaPrompt in this work.
DYNAMIC PROMPT TUNING
As previously mentioned, this work develops DynaPrompt enrich the family of test-time prompt tuning. Our motivation is to exploit beneficial information from prompt histories and alleviate the error accumulation online prompt tuning. DynaPrompt adaptively selects relevant online prompts for each test sample to optimize and includes an online update prompt buffer Vn, for each specific test sample Xn. The buffer contains a set of online learnable prompts V. { to store the distribution information in past samples, where M,, denotes the number of prompts the buffer at test step n. Each online prompt is initialized with a hand hand-crafted prompt (Shu et al., 2022) or a pretrained prompt (Zhou et al., 2022b) Vo and adaptively optimized with the online samples. During prompt tuning, each test sample first selects subset of the online prompts in the buffer and updates the buffer by optimizing the selected prompts. As shown in Figure 3, DynaPrompt consists of dynamic prompt selection (Section 4. and prompt appending (Section 4.2) strategies, as well as optimization and prediction with the dynamic prompts.
4.1 DYNAMIC PROMPT SELECTION
DynaPrompt introduces the dynamic prompt selection strategy to select appropriate prompts for each test sample from the online prompt buffer. Such strategy returns subset of the selected prompts S, = {v; E V, f(v.)} for sample x,, where f(v,) denotes selection conditions with specific metrics. We introduce the prediction entropy and probability difference the selection metrics.
4

---

Published as a conference paper at ICLR 2025
Entropy-based selection. We employ the prediction entropy one of our prompt selection metrics. Widely used in classification tasks, entropy quantifies the uncertainty of predictions, assessing how confident the prompt is on the test data. Prompts with lower entropies reflect more confident predictions on the test sample (Niu et al., 2022; 2023), indicating the prompt has more prior inolledge and relevant information on the sample. Given test sample x, and the online prompt in the corresponding prompt buffer V; E Vn. we can calculate the entropy as:
C
(xn vi) = Σ PÚ-= c( logp(D = c(x,, Vi),
where p(ù = c(X,, v.) denotes the averaged prediction across the selected augmentations similar to Eq. (1). In operation, we use the entropy of the initial prompt Vo the threshold and select the online prompts with lower entropy, where the selected prompts are more confident on the test sample (Niu et al., 2022). Formally, we have the entropy-selected online prompts subset En:
En = (v. E Vn (en v,) & Dent (en vo)}-
Therefore, e, contains the online prompts that produce more confident predictions than the initial prompt, indicating they incorporate more relevant information and are better suited for Xn.
Probability difference selection. However, the entropy is not always reliable in test-time tuning, especially when encountering distribution shifts (Lee et al., 2024). When continually selected and optimized for low entropy, the online prompts can be tuned overconfidently and produce incorrect predictions with very low entropy, causing prompt collapse discussed in Section 3. To avoid selecting overconfident prompts, we further introduce a probability difference metric for dynamic prompt selection. The probability difference D Pyro(Xa, (Xn: Vz) quantifies the prediction probability differences between the original test sample x,, and its augmentations X,, assessing the sensitivity of the prompts to the changes in the structure information of the input sample.
Given test sample x,, and online prompt v; Va, calculate the probability difference
D Ppro (Xn, v.) = pl = c" x vi) pÙù = c² X",,, V:).
where c" = arg max = c xn2,, vi) denotes the pseudo-label of the test sample x,, predicted by e
prompt v,. Prompts with higher Pro are more sensitive to the changes of the sample X,, which are less likely to be overconfident. By contrast, lower Drro indicates similar predictions regardless of input modifications, increasing the risk of overconfidence and prompt collapse, especially with low prediction entropy.
To cireumvent overconfident prompts during dynamic selection, we propose to select the online prompts with higher Drro values. Similar to the entropy metric, we use the probability difference of the initial prompt as the threshold for prompt selection. Formally, the subset of online prompts with high probability difference on the test sample x,, are formulated as:
Rr (v: E V. D (Xno vi) 2 D Ppro(Xn; Vo)
Dynamic selected prompts. Combining Eq. (3) and Eq. (5) together, we obtain the subset of selected prompt Sn, where the selected prompt meets the requirements in both above selection processes:
S, En R...
By taking the intersection of the two subsets in Eq. (6), the selected prompts simultaneously satisfy both lower entropy in Eq. (3) and larger probability differences in Eq. (5), which produce more confident predictions and are more sensitive selected prompts are more relevant to the test samples and have low risks of collapse. Moreover, since we utilize the entropy and probability differences of the initial prompt as thresholds, our method enables adaptively prompt selection for each test sample. By autonomously selecting relevant prompts for each test sample, the online prompts are enriched with more specific data distribution information, enhancing both the predictive performance of the current test sample and the optimization of the selected prompts. Since the irrelevant and collapse prompts are frozen, potential conflicting optimization directions for these prompts avoided, thereby reducing error accumulation.
the changes of the test sample. Therefore, the
Note that the predictions and entropy in Eq. (2) and (4) are inherently calculated for test-time prompt tuning in Eq. (1). Thus, our method introduces very few extra operations for prompt selection.
5

---

Published as a conference paper at ICLR 2025
Optimization and prediction. For each test sample x.. after dynamically selecting the online prompts Vn, we first optimize the selected prompts by minimizing the entropy as:
C
Cent(Sn: (Sn;),)) = Σ p(ù = c cxx,,, log) llg = Xx,,, Sn), S, + Sn - a Lent Sa), (7)
where p0 = c Sn) denotes the average prediction probabilities across the selected prompts and sample augmentations. With the updated prompts S.., we perform prediction for the test sample arg max p S.).
4.2 DYNAMIC PROMPT APPENDING
During dynamic prompt selection, there can be no appropriate prompt in the prompt set Vn for specific test samples, leading to an empty S,.. That means the online prompts in V, are either irrelevant for the current sample or collapsed. In this case, utilizing existing online prompts for the sample can lead to conflict optimization or severe error accumulation. To fix this issue, we introduce dynamic prompt appending as complementary strategy. Specifically, our method appends initial prompt Vo into the prompt set S,, when it is empty. As a result, the selected prompt set S,, is reformulated as:
S, = (vol, En RRR otherwise.
if & R., Ø;
(8)
The prompt in S, is optimized the same as Eq. (7) and then utilized in the inference of the sample.
However, the size of the prompt buffer Vn, cannot infinite when appending new learnable prompts due to memory constraints and computational costs. To avoid infinitely increasing numbers of prompts in the prompt buffer, we set the maximum number M of prompts in Vn as a hyperparameter, i.., Mn < M and introduce prompt deletion mechanism: when new prompt is appended into the prompt buffer and the current number of prompts is M, the method will remove the most inactive prompt Vincttvv from the buffer. By optimizing S, in Eq. (8) to Š,, through entropy minimization in Eq. (7), we formulate the update of the prompt buffer with dynamic prompt appending as:
Vn + S, {Vinactiv), if E,, R., Ø and Mn M;
Vu+1=
Vn + Sn Sn,
otherwise.
It is worth noting that the "4" and operations are the append and delete operations for the prompt buffer. As shown in Figure 3, we always put the optimized prompts in S,, the start of the prompt buffer. Therefore, we achieve the deletion mechanism by entirely removing the online prompt at the end of the buffer, which has not been activated for the maximum allowed time. By appending new online prompts and ejecting the inactive ones, our dynamic prompt tuning effectively incorporates information from new data distributions and reduces error accumulation. We provide algorithm of our method in Appendix A.
5 RELATED WORK
Prompt learning. To adapt vision-language models such as CLIP (Radford et al., 2021) and ALIGN (Jia et al., 2021) downstream tasks, prompt learning methods are introduced (Lester al., 2021; Li & Liang, 2021; Zhou et al., 2022b). Zhou et al. (2022b) propose learnable prompts in the input embedding space of the language model in CLIP. ProGrad (Zhu et al., 2023) aligns the gradients of the learnable prompts with the original prompt. In addition to the language input space, Bahng et al. (2022) introduces prompt learning into the vision branch of the CL.P model. Khattak et al. (2023) further proposes joint prompts for both vision and language encoders. To improve the generalization ability of the learned prompts, Zhou et al. (2022a) introduce imaging conditions into the language prompts. KgCoOp (Yao et al., 2023) reduces the forgetting of the general knowledge in the CLIP model by reducing the discrepancy between the learnable and handcrafted prompts. Derakhshani et al. (2023) propose Bayesian prompt learning to incorporate uncertainty in the learnable prompts. CoPrompt (Roy& Etemad, 2024) enforces the prediction consistency of the trainable and pre-trained models prevent overfitting on the downstream task. Any-shift prompting (Xiao al., 2024)

---

Published as a conference paper at ICLR 2025
generates the test-specific prompt for visual and text encoders in a single feedforward pass, without fine-tuning at test time, Our method also aims to address distribution shifts in downstream tasks adaptation of the CLIP model. Differently, we tune the prompt at test time to adapt the prompts to unseen distributions.
Test-time adaptation. Test-time adaptation (Liang et al., 2023; Xiao & Snoek, 2024) aims adapt pretrained models to test data distributions at test time by unsupervised optimization objectives. The idea of optimizing models at test time was first proposed by Sun et al. (2020), who introduced auxiliary self-supervised objective functions for test time training, followed by several recent works (Liu et al., 2021: Hardt & Sun, 2024). Alternatively, Wang et al. (2021) proposed model adaptation by entropy minimization at test time, which is widely used and investigated in recent test-time adaptation algorithms (Zhang et al., 2022; Goyal et al., 2022; Niu et al., 2022; 2023). There are two main settings for test-time adaptation methods, online adaptation (Wang et al., 2021; Iwasawa et al., 2021; Lee et al., 2023; Zhang et al., 2023; 2024b) and batch- wise adaptation (Schneider et al., 2020; Gao et al., 2022; Lim al., 2023). Online adaptation incrementally improves the adaptation by gradually incorporating test information from online samples, while taking the risk of error accumulation (Wang al., 2021; Niu et al., 2022; Lee et al., 2023). Batch- adaptation adapts the pretrained model to each test batch individually, avoiding error accumulation of online learning while ignoring the context information (Xiao al., 2022; Gao et al., 2022). Our method introduces online optimization into test-time prompt tuning while introducing a dynamic tuning method to reduce error accumulation.
Test-time prompt tuning. To address distribution shifts during the downstream task adaptation of the CLIP model, recent works propose test-time prompt tuning. TPT (Shu et al., 2022) tunes the prompts in the text embedding space of the CLIP model test-time, using an entropy minimization objective on randomly augmented samples. Following TPT, DiffTPT (Feng et al., 2023) generates more data for test-time tuning by pretrained diffusion models. Samadh et al. (2023) test-time tunes the prompts pretrained by MaPLe (Khattak et al., 2023) by aligning the test sample statistics to the offline source statistics. C-TPT (Yoon et al., 2024) considers the calibration error for tuning the prompts. RI.CF (Zhao et al., 2024) adopts an extra CLIP model as the reward model to provide feedback during test-time prompt tuning. Following TPT, most previous methods independently tune the prompt for each test sample. AdaPrompt (Zhang et al., 2024a) performs prompt tuning each batch of 64 test samples with a buffer to store confident, class- balanced samples for improved tuning and prediction. Our method aims to utilize the beneficial information from online samples to enhance the test-time prompt tuning for each individual sample while reducing error accumulation.
EXPERIMENTS
Fifteen datasets. Following previous methods (Shu et al., 2022; Samadh et al., 2023), we conduct experiments across two settings that suffer from distribution shifts to demonstrate the effectiveness of our method: domain generalization and cross-dataset shifts. For the domain generalization setting, we evaluate the method on ImageNet (Deng et al., 2009) and its four variant datasets: ImageNet- V2 (Recht et al., 2019), (SageNett(S)ketch (Wang et al., 2019), ImageNet-A (Hendrycks et al., 2021b), and Net- (Hendrycks al., 2021a). For the cross-dataset setting, we evaluate our method on 10 image classification datasets covering various tasks: Caltech101 (Fei-Fei et al., 2004), OxfordPets (Parkhi et al., 2012), StanfordCars (Krause al., 2013), Flowers (Nilsback Zisserman, 2008), Food101 (Bossard et al., 2014), FGVCAircraft (Maji et al., 2013), SUN397 (Xiao et al., 2010), DTD (Cimpoi et al., 2014), Au (Helber al., 2019), and UCF101 (Soomro et al., 2012).
Implementation details. Based on the CLIP model with ViT-Base-16 16 (Dosovitskiy et al., 2020), we initialize our dynamic prompts with the manually crafted "a photo of a" and optimize the prompts online in the text input embedding space. The prompt set optimized by one test sample is utilized for the next sample. Following TPT (Shu et al., 2022), we generate 63 augmentations by random resize crops for each individual test image to construct batch of 64 images including the original image. During the dynamic tuning, we calculate the entropy and augmentation probability differences over these 63 augmented images as the dynamic prompt selection metrics. The thresholds are obtained in the same way based on the initial prompt. We set the maximum number of the prompt set M as 10. We append new prompts the dynamic prompt set when appropriate prompt is selected for the test sample. Once the number of prompts in the prompt set V exceeds M, we remove the prompt that has been inactive for the longest time. For optimization, we select the top 10% confident samples
7

---

Published as a conference paper at ICLR 2025
Table 1: Compartsons on the domain generalization setting with both prompt learning and test- time prompt tuning methods. The prompt learning methods train their prompts on ImageNet. The proposed method outperforms both kinds of methods in terms of accuracy. Combining our method with pretrained prompts further improves the performance. Reproduced results are indicated in italiss.
Method
ImageNet
ImageNet-V2 V2
ImageNet-S
ImageNet-A
ImageNer-R
OoD Mean
CLIP (Radford et al,, 2021)
66.73
60.86
46.09
73.98
57.20
Prompt learning methods without test-Lime tuning
CoOp (2hou et al., 20226)
71.51
64.20
47,99
49.71
75.21
59,28
CoCoOp (Zhou sl. 2022a)
71.02
64.07
48.75
50.63
76.18
59.90
KoCoOp (Yáo al., 2023)
71.20
64.10
48.97
50.69
76.70
60.11
MaPLe (Khatrak et al., 2023)
70.72
64.07
15
50.90
76.98
60.28
CoPrompt (Roy & Exemad, 2024)
70.80
64.25
50.50
77.51
60.42
Any-shift Prompt (Xiao et al., 2024)
64.53
49. 99800
51,52
77,56
60.85
Test-time promip tuning methods
TPT (Shu al., 2022)
68.98
63.45
47.94
54.77
77.06
60.81
AdaPrompt (Zhang et al., 2024a)
39.32
47.72
47.71
73.98
57.18
DHITPT (Feng et al., 2023)
70.30
65.10
46.80
55.68
75.00
60.65
C-IPT (Yoon et al., 2024)
69.30
63,40
48.50
$2.90
78.00
60.70
This paper
69.61
64,67
48.22
56.17
78.17
61.81
CoOp+ + TPT (Shu et al., 2022)
73.61
66.83
49.29
$7.95
77.27
62.84
CoOp This paper
74.08
67.25
50.28
60.55
79.15
64.31
MaPLe + TPT (Shu et al., 2022)
71.87
64.87
48.16
58.08
78.12
62.31
PromptAlign (Samadh et al., 2023)
65.29
50.23
MaPLe + This paper
72.71
66.34
50.25
60.72
79.57
64.22
among the batch and calculate the entropy of the averaged logits of the selected predictions following Shu et al. (2022). We utilize learning rate of 0.005 for domain generalization and 0.003 for the cross-dataset settings with the AdamW optimizer.
When combing our method with other prompt tuning methods (Zhou et al., 2022b; Khattak et al., 2023), we initialize the prompts as the pretrained prompts of these methods trained on ImageNet. When integrated with MaPLe (Khattak et al., 2023), our dynamic test-time prompt tuning is applied on both the textual and visual branches, therefore evolved into a multimodal setting. Our method runs NVIDIA A100 GPU.
6.1 COMPARISIONS
Comparisons on domain generalization setting. We compare our method on the domain general- ization setting with both prompt learning (Zhou et al., 2022a;b; Khattak et al., 2023; Roy & Etemad, 2024) and test-time prompt tuning methods (Shu et al., 2022: Samadh et al., 2023). The prompt learning methods train their prompts by supervised cross-entropy loss ImageNet. As shown in Table 1, our method achieves better overall performance compared with the prompt learning methods. Moreover, since our DynaPrompt is orthogonal to most of these prompt learning methods, applying our method together with prompt learning methods like CoOp (Zhou et al., 2022b) and MaPLe (Khattak al., 2023)) further improves the performance.
DynaPrompt also surpasses recent test-time prompt tuning methods. Our method outperforms TPT (Shu al., 2022) on all datasets with both hand-crafted and learned prompts (Zhou al., 2022b; Khattak et al., 2023), achieving at least 1% improvements. The method also achieves better overall performance compared with other recent methods DifTTPT (Feng al., 2023), AdaPrompt (Zhang et al., 2024a), and C-TPT (Yoon et al., 2024). Compared with PromptAlign (Samadh et al., 2023). which is constructed on MaPLe (Khattak et al., 2023) and utilizes extra source data during test-time tuning, our method is also superior.
Comparisons on cross-dataset setting. the cross-dataset setting, we also compare our method with both prompt learning and test-time prompt tuning methods. As shown in Table 2, our method outperforms the common prompt learning methods for of the 10 datasets and achieves the best overall performance. Compared with the test-time prompt tuning methods, the proposed method again outperforms TPT with both hand-crafted and learned prompts (Khattak et al., 2023). Our method also achieves higher accuracy compared with the recent test-time prompt tuning methods Samadh et al. (2023). We observe that the improvements on these datasets for our method, and even most test-time prompt tuning methods, are not obvious in the domain generalization setting. The
8

---

Published as a conference paper at ICLR 2025
Table Comparisons on the cross-dataset setting with both prompt learning and test-time prompt tuning methods. The prompt learning methods train their prompts on ImageNet. The proposed method again outperforms both kinds of methods on datasets and achieves the best overall performance. Reproduced results are indicated in italics.
Caltech
Flowers
Food101
Airenuft
SUN397
ET
UCF101
Pots
Cars
DTD
Mean
Method
CLIP
93.35
88,25
65.48
67.44
83,65
23.67
62.59
44.27
42.01
65.13
63.58
Prompt
learning methods without test-time tuning
CoOp
93.70
89.14
64.51
68.71
85.30
18.47
64.15
41.92
46.39
66.55
63.88
CoCoOp
94,43
90.14
65.32
71.88
86.06
22.94
67.36
45.73
45.37
68.21
65.74
MaPLe
93.53
90.49
65.57
72.23
86.20
24.74
67.01
46.49
48.06
68.69
66.30
CoPrompt
94.50
90.73
65.67
72.30
86.43
24.00
67.57
47.07
51.90
69.73
67.00
Test-time
prompt tuning
methods
TPT
94.16
87.79
66.87
68. 98
84.67
24.78
65.50
47.75
42,44
68.04
65.10
DifITPT
92.49
88.22
67.01
70.10
87.23
25.60
65.74
47.00
41.04
68.22
65.47
C-TPT
93.60
88.20
65.80
69.80
83,70
24.00
64.80
43.20
65.70
64.80
AdaPrompt This paper
94.07
89.64
63.29
72.97
84.72
21.21
65.37
44.75
47.20
67.22
65.04
94.32
88.28
67.65
69.95
85.42
24.33
66.32
47.96
42.28
68.72
65.52
CoOp TPT
93.15
89.48
66.77
68.48
86.48
20.51
66.06
43.32
37.73
68.91
64.09
CoDp This paper
94.40
90.04
67.35
69.38
86.45
21.35
66.17
46.98
38.55
69.54
65.02
MaPLe TPT
93.59
90.72
66.50
72.37
86.64
24.70
67.54
45.87
47.80
69.19
66.49
MaPLe PromptAlign
94.01
90.76
68.50
72.39
86.65
24.80
67.54
47.24
47.86 86
69.47
66.92
MaPLe This paper
95.17
90.95
68.26
73.28
86.60
24.36
68.18
48.75
47.53
69.85
67.29
Table 3: Ablations on our dynamic prompt selection strategy. Either without the entropy-based
selection in Eq. (3) or the probability difference selection in Eq. (5) leads to obvious performance degradation.
Method
ImageNet-V2 ImageNet-S
ImageNet-A
ImageNet-R
Mean
w/o entropy w/o probability difference
61.45
46.86
50.25
76.19
58.69
61.93
46.95
50.95
77.37
59.23
With both selection metries
64.67
48.22
56.17
78.17
61.81
Table 4: Ablations on dynamic prompt appending strategy. Without the appending strategy, the performance drops considerably due error accumulation and prompt collapse.
Method
ImageNet-V2
ImageNet-S
ImageNet-A
ImageNet-R
Mean
w/o dynamic appending w/ dynamic appending
35.66
20.38
30.74
43.72
32.63
64.67
48.22
56.17
78.17 17
61.81
main reason can be that the fine-gratd fine-grained tasks (e.g., Aircraft and Food101) and specific tasks (e.g.. EuroSAT with satellite images) are more detailed and challenging for prompt tuning at test-time.
6.2 ABLATION STUDIES
Effectiveness of dynamic prompt selection and appending. To investigate the roles of our dynamic selection and appending strategies dynamic prompt learning. we conduct experiments on domain generalization datasets. As shown in Table 3, removing either the entropy the probability difference metric during dynamic prompt selection results in obvious performance degradation on all datasets. Without the entropy, the selected prompts can be either confident or uncertain to the sample, which is not suitable for the sample, leading to performance degradation. Without the probability difference, the proposed method may select collapsed prompts during test-time tuning, resulting in the wrong direction of the prediction and optimization.
As shown in Table 4, the performance degrades considerably when we remove the dynamic prompt appending strategy from our method. Without the appending and removing strategy, the method can only select the online prompts in the buffer, even if there is no appropriate one. In this case, the risks of optimization in conflict directions are highly amplified. The error during the optimization is then accumulated the sequential online samples, leading to prompt collapse.

---

Published as a conference paper at ICLR 2025
Analyses on error accumulation. Here we provide more analysis on our DynaPrompt. As shown in Table 5, TPT achieves good overall per- formance as it avoids error accumulation by in- dependent prompt tuning. Online TPT performs competitively at the start but declines rapidly as shown in Figure 2. While Oracle also has a per- formance degradation initially, the performance stabilizes since it avoids error accumulation by tuning prompts only with correct predictions. By incorporating the beneficial information from online samples, Oracle surpasses TPT and Online TPT. Our DynaPrompt reduces error accumulation and achieves stable performance by dynamically selecting, appending, and deleting the online prompts. Therefore, the method achieves good overall performance and beats TPT by incorporating relevant information in online data. Moreover, our method performs better during online learning, proving its capability to capture beneficial information.
Table 5: Analyses on error accumulation. Dy- naPrompt reduces error accumulation and benefits from selected online prompts, outperforming TPT and Online
Method
TPT
Online TPT
Oracle
This paper
Accuracy 54.77
6.96
59.38
56.17
Influence of the prompt buffer size. We also ablte the influence of the prompt buffer size M on our method in Figure 4a. The experiments are
5.6
conducted on ImageNet-A. As the prompt buffer size increases, the proposed method shows an upward slope. The improvement is faster when the buffer size is smaller than 10. Figure 4b shows the time costs of the proposed method, which continually increases with larger buffer sizes. Compared with TPT, our method requires more time costs, which can be a limitation of the approach. Note that most of the additional time cost stems from optimizing multiple prompts rather than the selection and appending strategy. For instance, with the buffer size 10, the total processing time per test sample is approximately 0.39 seconds, of which the selection and appending steps account for only 0.004 seconds. For good trade-off between performance and time costs, we set the maximum size of the prompt buffer to 10.
16
CLP
T2T
40
10
This paper
c.0
This peper
25
Prompt buffersto
Prompt butfer
(a) Top-1 Accuracy dif- (h) Time costs with differ- ferent prompt buffer sizes. ent prompt buffer sizes.
Figure 4: Influence of the prompt buffer size on performance (a) and time costs (b). Larger buffer sizes lead to better performance with higher costs.
Sensitivity to test time sample order. As our DynaPrompt optimizes the prompt online, the performance of the method can be influenced by the order of the test samples. To investi- gate this influence, we conduct experiments on ImageNet-A and ImageNet-R for six rounds, which have different sample orders. We shuffle the sample order with different random seeds at test time, which leads to different sample orders. The results are provided in Figure 5a and 5b, respectively. Order 0 denotes the default order the same as the experiments of TPT (Shu et al., 2022). We observe that there are fluctuations in the performance of both datasets. The performance on ImageNet-R is more stable with larger number of test samples (30,000) than ImageNet-A (7,500). Nevertheless, independent of the test order, the proposed method surpasses TPT consistently.
24
(a) Different sample orders (b) Different sample orders for ImageNet-A.
for ImageNet-R.
Figure 5: Sensitivity to test-time sample order on ImageNet-A (a) and ImageNet-R (b). More test samples lead to more stable performance.
CONCLUSION
In this paper, we propose DynaPrompt, new test-time prompt tuning approach, which exploits beneficial information from the online test samples while alleviating error accumulation. Our method introduces dynamic prompt buffer, which adaptively selects and optimizes prompts for each test sample. The selected prompts incorporate relevant information from previous test samples, thereby benefiting the prediction for the current sample. By optimizing the selected prompts while freezing the rest, the method further enhances the learned prompts to incorporate relevant information from test data. DynaPrompt also enables the buffer to autonomously append new learnable prompts and delete the inactive ones, improving adaptability to new test data and reducing the risk of error accumulation. Experiments on fourteen benchmarks validate the effectiveness of our proposal.
10

---

Published as a conference paper at ICLR 2025
REPRODUCIBILITY
We include all necessary details to facilitate the reproducibility of our work. The experimental setup, including benchmarks, model configurations. hyperparameters, and evaluation protocols, is thoroughly explained in the experiments section. We also give an algorithm the Appendix to provide the detailed process of our method. We will make our code publicly available.
REFERENCES
Hyojin Bahng, Ali Jahanian, Swami Sankaranarayanan, and Phillip Isola. Exploring visual prompts for adapting large-scale models. arXiv preprint arXiv:2203.1727: 2022.
Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. Food-101 mond-igg discriminative compo- nents with random forests. European Conference on Computer Vision, pp. 446-461. Springer, 2014.
Mircea Cimpoi, Subhransu Maji, lasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. De- scribing textures in the wild. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 3606-3613, 2014.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large-scale hierarchical image database. In IEEE Conference Computer Vision and Pattern Recognition, pp. 248-255, 2009.
Mohammad Mahdi Derakhshani, Enrique Sanchez, Adrian Bulat, Victor G Turrisi Costa, Cees GM Snoek, Georgios Tzimiropoulos, and Brais Martinez. Bayesian prompt learning for image-language model generalization. In IEEE International Conference on Computer Vision, pp. 15237-15246, 2023.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representutions, 2020.
Li Fei-Fei, Rob Fergus, and Pietro Perona. Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. In IEEE Conference Computer Vision and Pattern Recognition Workshop, pp. 178- 178. IEEE, 2004.
Chun-Mei Feng, Kai Yu, Yong Liu, Salman Khan, and Wangmeng Zuo. Diverse data augmentation with diffusions for effective test-time prompt tuning. In IEEE International Conference on Computer Vision, pp. 2704-2714, 2023.
Jin Gao, Jialing Zhang, Xihui Liu, Trevor Darrell, Evan Shelhamer, and Dequan Wang. Back to the source: Diffusion-driven test-time adaptation. arXiv preprint arXiv:2207 00342, 2022.
Sachin Goyal, Mingjie Sun, Aditi Raghunathan, and J Zico Kolter. Test time adaptation via conjugate pscudo-labels. In Advances Neural Information Processing Systems, volume 35, pp. 6204-6218, 2022.
Moritz Hardt and Yu Sun. Test-time training on nearest neighbors for large language models. In International Conference on Learning Representations, 2024.
Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(7):2217-2266 2019.
Dan Hendrycks, Norman Mu, Ekin D Cubuk, Barret Zoph, Justin Gilmer, and Balaji Lakshmi- narayanan. Augmix: simple data processing method to improve robustness and uncertainty. In International Conference on Learning Representations, 2020.
Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, et al. The many faces of robustness: A critical analysis of out-of- nutrof-distribution generalization. In IEEE International Conference on Computer Vision, pp. 8349, 2021a.
11

---

Published as a conference paper at ICLR 2025
Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial examples. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 15262-1527)) 2021b.
Yusuke Iwasawa et al. Test-time classifier adjustment module for model-agnostic domain generaliza- tion. In Advances in Neural Information Processing Systems, volume 34, 2021.
Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. In International Conference on Machine Learning, pp. 4904-4916. PMIR, 2021.
Muhammad Uzair Khattak, Hanoona Rasheed, Muhammad Maaz, Salman Khan, and Fahad Shahbaz Khan. Maple: Multi-modal prompt learning. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 19113-1122, 2023.
Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine-grained categorization. In IEEE International Conference on Computer Vision Workshops, pp. 554-561, 2013.
Jonghyun Lee, Dahuin Jung, Sachyung Lee, Junsung Park, Juhyeon Shin, Uiwon Hwang, and Sungroh Yoon. Entropy is not enough for test-time adaptation: From the perspective of disentangled factors. In International Conference on Learning Representations, 2024.
Yoonho Lee, Annie Chen, Fahim Tajwar, Ananya Kumar, Huaxiu Yao, Percy Liang, and Chelsea Finn. Surgical fine-tuning improves adaptation to distribution shifts. In International Conference on Learning Representations, 2023.
Brian Lester, Rami AI-Rfou, and Noah Constant. The power of scale for parameter-efffcient prompt tuning. arXiv preprint arXiv:2104. 04.08691, 2021.
Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint arXiv:2101.00/1 2021.
Jian Liang, Ran He, and Tieniu Tan. A comprehensive survey on test-time adaptation under distribu- tion shifts. arXiv preprint vrXiv:2303.15366 2023.
Hyesu Lim, Byeonggeun Kim, Jaegul Choo, and Sungha Choi. Tin: A domain-shift aware batch normalization in test-time adaptation. In International Conference on Learning Representations, 2023.
Yuejiang Liu, Parth Kothari, Bastien van Delft, Baptiste Bellot-Gurlet, Taylor Mordan, and Alexandre Alahi. Tit++: When does self-supervised test-time training fail or thrive? In Advances in Neural Information Processing Systems, volume 34, 2021.
Xiaosong Ma, Jie Zhang, Song Guo, and Wenchao Xu. Swapprompt: Test-time prompt adaptation for vision-language models. In Advances in Neural Information Processing Systems, 2023.
Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi. Fine-grained visual classification of aircraft, arXiv preprint arXiv: 106.5513....11 2013.
Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In 2008 Sixth Indian conference on computer vision, graphics & image processing, pp. 722-729. IEEE, 2008
Shuaicheng Niu, Jiaxiang WU, Yifan Zhang, Yaofo Chen, Shijian Zheng, Peilin Zhao, and Mingkui Tan. Efficient test-time model adaptation without forgetting. In International Conference Machine Learning, pp. 18888166905 PMLR, 2022.
Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Zhiquan Wen, Yaofo Chen, Peilin Zhao, and Mingkui Tan. Towards stable test-time adaptation in dynamic wild world. In International Conference on Learning Representations, 2023.
12

---

Published as a conference paper at ICLR 2025
Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar. Cats and dogs. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 1498-3503 IEEE, 2012.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, pp. 8748-8763. PMLR, 2021.
Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do imagenet classifiers generalize to imagenet? In International Conference on Machine Learning, pp. 5389-5400, PMLR, 2019.
Shuvendu Roy and Ali Etemad. Consistency-guided prompt learning for vision-languan models, In International Conference on Learning Representations, 2024.
Jameel Hassan Abdul Samadh, Hanan Gani, Noor Hazim Hussein, Muhammad Uzair Khattak, Muzammal Naseer, Fahad Khan, and Salman Khan. Align your prompts: Test-time prompting with distribution alignment for zero-shot generalization. In Advances in Neural Information Processing Systems, 2023.
Steffen Schneider, Evgenia Rusak, Luisa Eck, Oliver Bringmann, Wieland Brendel, and Matthias Bethge. Improving robustness against common corruptions by covariate shift adaptation. In Advances in Neural Information Processing Systems, volume 33, pp. 11539-11551, 2020.
Manli Shu, Weili Nie, De-An Huang, Zhiding Yu, Tom Goldstein, Anima Anandkumar, and Chaowei Xiao. Test-time prompt tuning for zero-shot generalization in vision-language models. In Advances in Neural Information Processing Systems, volume 35, pp. 14274-14289, 2022.
Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. Ucf101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint 1212. 040., 2012.
Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei Efros, and Moritz Hardt. Test-time training with supervsin for generalization under distribution shifts. In International Conference on Machine Learning, pp. 0009922222222 PMLR, 2020.
Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test- time adaptation by entropy minimization. In International Conference on Learning Representations, 2021.
Haohan Wang. songwei Ge, Zachary Lipton, and Eric P Xing. Learning robust global representations by penalizing local predictive power. In Advances in Neural Information Processing Systems, volume 32, 2019.
Jianxiong Xiao, James Hays, Krista A Ehinger, Aude Oliva, and Antonio Torralba. Sun database: Large-scale scene recognition from abbey zoo. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 3485-3492, IEEF, 2010.
Zehao Xiao and Cees GM Snoek. Beyond model adaptation at test time: A survey, arXiv preprint arXiv:2411.03687, 2024.
Zchao Xiao, Xiantong Zhen, Ling Shao, and Cees G M Snock. Learning generalize across domains on single test samples. In International Conference on Learning Representations, 2022.
Zehao Xiao, Jiayi Shen, Mohammad Mahdi Derakhshani, Shengcai Liao, and Cees G. M. Snoek. Any-shift prompting for generalization over distributions. IEEE Conference on Computer Vision and Pattern Recognition, 2024.
Hantao Yao, Rui Zhang, and Changsheng Xu. Visual-language prompt tuning with knowledge-guice context optimization. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 6757-6767, 2023.
Hee Suk Yoon, Eunseop Yoon, Joshua Tian Jin Tee, Mark Hasegawa-Johnson, Yingzhen Li, and Chang Yoo. C-tpt: Calibrated test-time prompt tuning for dision-language models via text feature dispersion. In International Conference on Learning Representations, 2024.
13

---

Published as a conference paper at ICLR 2025
Ding-Chin Chu Zhang, Zhi Zhou, and Yu-Feng Li. Robust test-time adaptation for zero-shot prompt tuming. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 16714-16722, 2024a.
Marvin Zhang, Sergey Levine, and Chelsea Finn. Memo: Test time robustness via adaptation and augmentation. In Advances in Neural Information Processing Systems, volume 35, pp. 38629- 38642, 2022.
Yifan Zhang, Xue Wang. Kexin Jin, Kun Yuan, Zhang Zhang, Liang Wang, Rong Jin, and Tieniu Tan. Adanpe: Exploring non-parametric classifier for test-time adaptation. In International Conference on Machine Learning, pp. 41647-41677. PMLR, 2023.
Yunbei Zhang, Akshay Mehra, and Jihun Hamm. Dynamic domains, dynamic solutions: Dpcore for continual test-time adaptation. arXiv preprint
2024b.
Shuai Zhao. Xiaohan Wang, Linehao Zhu, and Yi Yang. Test-time adaptation with clip reward for zero-shot generalization in lision-lagguage models. In International Conference on Learning Representations, 2024.
Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Conditional prompt learning for vision-language models. In IEEE Conference Computer Vision and Pattern Recognition, pp. 16816-16825, 2022a.
Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision- language models. International Journal of Computer Vision, 13009))2337-2348, 2022b.
Beier Zhu, Yulei Niu, Yucheng Han, Yue Wu, and Hanwang Zhang. Prompt-aligned gradient for prompt tuning. In IEEE International Conference on Computer Vision, pp. 16669-1566,, 2023.
14

---

Published as a conference paper at ICLR 2025
A ALGORITHM
Algorithm Dynamic Test-Time Prompt Tuning (DynaPrompt)
1: Input: Test samples {x,, W prompt buffer V. with length Ma initialized as Vo 22;
maximum buffer size M; hand-crafted or pretrained initial prompt Vo
2: for 0: do
3: Randomly augment X,, to X
4: Obtain predictions {p(y)X,, v
Mr and p(y)X,, vo).
// Dynamic prompt selection.
5:
Calculate (Xn, Va) and Dent (Xa, vo) by Eq. (2), then select prompts subset E, by Eq. (3). Calculate pyro (Xns Vi) and Paro (Xns Vo) by Eq. (4), then select prompts subset R, by Eq. (5).
6:
7:
Select relevant prompts S, by & RR..
// Dynamic prompt appending. if S, Ø then
8:
9:
S, [vo].
10:
end if'
// Optimizing selected prompts.
11:
Tune the prompts in S, by entropy minimization in Eq. (7): Sn S, LVeen (Xn;
// Update the prompt buffer Vn to Vn+1
12:
if M, = M and & R,, Ø then
13:
Append the updated prompt S, to the top of Vn. Remove the prompt Vinoctive at the bottom of Vn
14:
15:
else
16:
Append the optimized prompts in S, to the top of Vn-
17:
Remove the selected prompts in S, from Vn-
18:
end if
19:
end for
B DETAILED IMPLEMENTATIONS
Details of data augmentation. To generate the augmented data X,, for each sample, we follow the same data augmentation strategy used in TPT (Shu al., 2022). That is, we use AugMix (Hendrycks et al., 2020) to augment the original test image into 63 different augmentation samples, leading to 64 samples in total for each test image. Each test image first augmented by resize and random crop, then fed into the AugMix strategy with several augmentation methods including auto contrast, equalization, posterization, rotation, solarization, shearing, and translating.
C EXTRA EXPERIMENTS
Effect of initial prompts. The initial prompts can affect the final performance in prompt learning (Zhou et al., 2022a). To investigate the effect of the initial prompts of the proposed method, we conducted experiments
ImageNet-A using various initial text prompts. As shown in Table the initial prompts affect the performance of CLIP (Radford et al., 2021), TPT (Shu et al., 2022), as well as our method. The reason can be related the initial predictions of the original CLIP model. Nonetheless, our method consistently outperforms TPT, showing robustness despite variations in initialization.
Ablations on prompt length for online test-time prompt tuning, To investigate the prompt length effect, we experiment with longer prompts for both TPT and Online TPT. We set the prompt length to 40, which is 10 times longer than the 4-item original "a photo of a". We consider two types of long prompts: (A) 10 times copy of "a photo of a" (B) Leet us solve an image classification task: a
photo of distinct object. animal, plant, scene, captured in diverse environments and representing meaningful categories. Carefully analyze its features; the exact category of the photo a" generated by GPT-40.
15

---

Published as a conference paper at ICLR 2025
Table 6: Effect of different initial prompts. The initial prompts affect the performance of CLIP (Radford et al., 2021), TPT (Shu et al., 2022), as well our method. Our method consistently outperforms TPT, showing robustness despite variations initialization.
Initial prompt
CLIP
TPT
This paper
"a photo of a"
47.87
54.77
56.17
"an image of a"
48.31
54.84
56.19
"high-quality of a" " "Identify feature of Visual features of" the
44.48
51.47
52.57
46.08
50.18
51.84
46.21
51.48
53.13
"natural photo of a"
44.82
52.56
54.08
"classify the photo of"
46.60
52.51
53.27
Mean
46.35
52.54
53.89
Table 7: Ablations on prompt length for online test-time prompt tuning, Well-designed long prompt improves the performance of the original CLIP, but may lead more difficult optimization for test-time prompt tuning, resulting in worse TPT performance. The long prompts may even lead to worse prompt collapse due to the difficult optimization and error accumulation. By contrast, our method achieves better performance while reducing error accumulation by our dynamic prompt selection and appending strategies.
Method
Initial prompt
Trainable prompt length
Accuracy
"a photo of a"
47.87
CLIP
long prompt (A)
40
46.99
long prompt (B)
40
48.21
"a photo of a"
4
54.77
TPT
long prompt (A)
40
52.97
long prompt (B)
40
52.23
"a photo of d"
6.96
Online TPT
long prompt (A)
40
2.06
long prompt (B)
40
4.24
This paper
"a photo of a"
4 10
56.17
As shown in Table 7, longer trainable prompts do not solve the problem of prompt collapse for online learning, even worsening the problem. Since the online testing fails due to error accumulation and prompt collapse, simply improving the length of the prompts does not help. Specifically designed long prompts (B) perform better the optimization-free CLIP model. However, may lead to more difficult optimization for test-time tuning, resulting in worse TPT performance. By contrast, based on the prompt selection and appending strategy, our method achieves better performance while reducing error accumulation and prompt collapse.
Experiments on different backbones. To evaluate the proposed method on different backbones, we conduct experiments for ImageNet-bart datasets on ResNet-50 and ViT-B/32. The experiments are provided in Table 8. Our method again outperforms CLIP and TPT.
Online prompts tuning with identical initialization. To provide more insights into the prompt collapse of online prompt tuning, we conduct experiments on ImageNet-A with 10 initialized online prompts. The prompts are initialized by the embedding of "a photo of a" with random noise to be identical. We use our selection strategy to select and optimize the prompts online. When no prompt selected, use the initial prompt "a photo of a" for prediction.
As shown in Table 9, the variant method outperforms online TPT and CLIP, which demonstrates that it reduces collapse since it provides diverse optimization directions for different test samples through dynamic selection. However, the method underperforms TPT and our DynaPrompt. However, underperforms TPT and our method. The reason be that the negative influence of previous test
16

---

Published as a conference paper at ICLR 2025
Table 8: Experiments on different backbones. Our method again outperforms CLIP (Radford et al., 2021) and TPT (Shu et al., 2022), despite the backbone of the CLIP model.
Method
ImageNet
ImageNet-V2
ImageNer-S
ImageNer-A
ImageNer-R
mean
OoD
ResNet-50
CLIP
58, 16
51.41
33.37
21.83
56,15
44.18
40.69
TPT
60.74
54.7
35.09
26.67
59.11
47.26
43.89
This paper
61.56
55.12
35.64
27.84
60.63
48.16
44.81
ViT-B/32
62.05
54.79
40.82
29.57
65.99
50.64
47.79
TPT
63.64
57.22
41.66
34.63
69.42
53.31
50.73
This paper
64.72
58.10
42.04
36.05
70.46
54.27
51.66
Table Identical initialization online prompts with dynamic selection. We introduce 10 online prompts with identical initialization and dynamically select and optimize subsets of the prompts at test time. The method outperforms CLIP and Online demonstrating the variant method reduces prompt collapse during online tuning. However, the performance is still worse than TPT and Our DynaPrompt, indicating error accumulation still exists.
Method
ImageNet-A
CLIP
47.87
Online TPT
6.96
Identical initialized online prompts with dynamic selection
48.14
TPT
54.77
This paper
56.17
samples during online updating is still entirely solved by the limited number of predefined online prompts, which leads to error accumulation and suboptimal predictions.
17


Reviews:
[{"review_id": "Wn9vgurSCo", "reviewer": "Reviewer_tKkt", "summary": "This paper introduces a dynamic test-time prompt tuning approach that enhances zero-shot generalization in vision-language models. It leverages the beneficial information from previous online samples to adaptively select and optimize prompts, reducing error accumulation and improving model performance across various datasets.", "questions": "1. I am curious about how the order for each round in the 'Sensitivity to test time sample order' section was set.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well-organized and easy to follow. \n2. The proposed DynaPrompt effectively mitigates error accumulation, a prevalent challenge in online test-time tuning，leading to more stable performance across sequential test samples while exploiting beneficial information from prior online test samples.\n3. Despite the increased time costs associated with larger prompt buffer sizes, the experimental outcomes confirm the  effectiveness of the proposed method.", "weaknesses": "1. In prompt learning, the initial prompts might affect the final performance. I wonder whether a similar situation can occur with the proposed method. The authors are encouraged to conduct related experiments.\n2. Could the proposed method be extended to incorporate visual prompts, thereby evolving into a multimodal approach? Additionally, when integrated with MaPLe, is the method only applied to the textual branch?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qbuZAfKUV4", "reviewer": "Reviewer_FeBK", "summary": "This paper addresses fundamental issues in test-time prompt tuning, specifically focusing on the selection, updating, appending, and deletion of prompts. The authors introduce an innovative dynamic test-time prompt tuning approach, which incorporates two novel prompt evaluation metrics alongside a prompt buffer modification strategy. Extensive experimental results underscore the effectiveness of the proposed method.", "questions": "Please refer to the `Weaknesses` section.", "limitations": "", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper is well-structured and easy to follow, with the three technical components clearly and accessibly presented.\n2. The proposed method is well-reasoned, employing dynamic prompt selection and updating mechanisms that are both effective and distinct from prior studies, which primarily focus on data manipulation.\n3. The experiments are thorough, and the results convincingly demonstrate the effectiveness of the proposed method.", "weaknesses": "1. The range of comparison methods could be expanded, as the paper overlooks one relevant comparison method [1].\n2. Both the prediction entropy metric and probability difference metric provide insights into model prediction confidence, though from different perspectives. It is unclear why entropy is specifically used to measure relevance while difference is used to maintain diversity. Why does the direct combination of these two types of prompts yield effective results? Would a two-step prompt selection process, satisfying both conditions simultaneously, be more advantageous?\n3. Details regarding the data augmentation set  $X_n$  are insufficiently discussed in this paper.\n\n[1] Dingchu Zhang, Zhi Zhou, Yufeng Li: Robust Test-Time Adaptation for Zero-Shot Prompt Tuning. AAAI 2024: 16714-16722", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4HazmkmBHv", "reviewer": "Reviewer_qkjX", "summary": "This paper proposes a new test-time prompt tuning method called DynaPrompt, which leverages a dynamic prompt buffer to extract beneficial information from online test samples while reducing error accumulation. The method adaptively selects and optimizes prompts for each test sample, enabling the selected prompts to integrate relevant information from previous test samples, thus improving the prediction performance of the current sample. Experimental results demonstrate that this method performs effectively across multiple benchmark datasets.", "questions": "1. It is recommended that the authors add ablation experiments to further demonstrate the validity of the methodology by performing online tests from scratch using the same prompt lengths as the final model and performing performance comparisons.\n2. The author's paper mentions deleting the prompt, does the author mean that the entire prompt is deleted in its entirety, or does he mean that only the parameters of the prompt are set to zero?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper propose a dynamic prompting method that utilizes useful information from online test samples while mitigating the problem of error accumulation.\n2. DynaPrompt improves the adaptive capability of the model by introducing a dynamic prompt selection strategy that adaptively selects and optimizes relevant prompts for each test sample based on two metrics: predictive entropy and probability difference.\n3. During the dynamic prompt selection process, if no suitable prompts can be found, DynaPrompt employs a dynamic prompt appending strategy to append new initial prompts to the set of prompts and remove the least active prompts, thus effectively incorporating information from the new data distribution.", "weaknesses": "1.  The computational complexity increases, and the authors' approach requires dynamic updating and selection of cues at each stage, whether it introduces more computational time.\n2.  The authors' approach demonstrates the advantages of dynamic prompting, however did the authors consider whether comparable performance could also be achieved if online testing was performed from scratch using a prompt length comparable to that of the final model.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "X4xGmFcfm9", "reviewer": "Reviewer_kMEy", "summary": "This paper introduces DynaPrompt, a test-time prompt-tuning (TPT) approach that exploit information from previous test samples while avoiding error accumulation. While naively adapting TPT to the online setting lead to collapse, DynaPrompt leverages a dynamic buffer of prompts and optimize only the most relevant prompts for each test sample. Furthermore, DynaPrompt introduces only one additional hyper-parameter, the buffer size, thanks to an adaptative thresholding strategy.  DynaPrompt demonstrate consistent improvement over TPT and its variants (CoOp+TPT, MaPLe+TPT) making it a simple and effective alternative.", "questions": "- Unless I missed it, it seems that there is no results on the Imagenet dataset. Could you provide results on the Imagenet dataset ?  \n- Could you provide evaluation results on other CLIP vision backbones such as ViT-B/32 or Resnet-50 ?\n- Why did you not included results for CoOp + DynaPrompt in Table 2 ?\n- If I understand your method correctly, the buffer initially contains only one prompt initialized as ‘a photo of a’. For a given test sample, if your selection criteria (entropy and probability difference) are not met, a new prompt is initialized with ‘a photo of a’; otherwise, selected past prompts are used for optimization. I’m curious whether you considered initializing a set of prompts from the beginning and using your selection criteria to optimize subsets of this buffer. Initially, all prompts would be identical, but a small degree of randomness at the start could help address this issue. Do you think collapse would occur in that scenario? An experimental result would be helpful, but your thoughts or intuition on this would be sufficient.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- **Motivation**: The authors made a good job motivating the paper, illustrating the potential benefit of moving to the on-line scenario while showing the non-triviality of extending TPT to this setup.\n- **Contribution**: The technical contribution of the paper is simple and effectively solve a clearly identified issue.\n- **Clarity**: The paper is easy to follow and arguments are clearly articulated.\n- **Experiments**: The experiments are convincing and show consistent improvements over the baselines.", "weaknesses": "- **Missing experiments** : Some experiments are missing like evaluating on different backbones and more importantly evaluation on the Imagenet dataset. I will consider raising my score if results on the Imagenet dataset are added.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)