PSP: Prompt-Guided Self-Training Sampling Policy for Active Prompt Learning
Authors: Sen Tao
Kaiduo Feng
Jiawei Liu
Peng Zeng
Yongchao Xu
Yufei Zheng
Zheng-Jun Zha
Venue: ICLR 2026 Poster (2026)
Keywords: Active Prompt Learning
Reinforcement Learning
CLIP
Primary area: other topics in machine learning (i.e., none of the above)
TL;DR: We propose a Prompt-Guided Self-Training Sampling Policy (PSP) for Active Prompt Learning, which integrates Soft Actor-Critic with a customized real-pseudo hybrid reward and vectorized critics.

Abstract: Active Prompt Learning (APL) using vision-language models (\textit{e.g.}, CLIP) has attracted considerable attention for mitigating the dependence on fully labeled dataset in downstream task adaptation. However, existing methods fail to explicitly leverage prompt to guide sample selection, resulting in the selected samples being ineffective in facilitating the prompt template's downstream task adaptation, while also overlooking valuable complementary information in the unselected samples. To fill this gap, we propose a novel Prompt-Guided Self-Training Sampling Policy (PSP) for APL, which integrates Soft Actor-Critic with a customized real-pseudo hybrid reward and vectorized critics to incorporate prompts in guiding sample selection toward those that facilitate the optimization of prompt template, by jointly considering both selected and unselected samples. Specifically, PSP comprises two prominent components: Vectorized Soft Actor-Critic Sampling Policy (VSSP) and Uncertainty Augmented Self-Training (UST) mechanism.  VSSP customizes a real-pseudo hybrid reward based on learned prompts and image features, which is fed into vectorized critics to estimate Q-value for each sample and compute gradients that optimize the actor, allowing it to refine its sampling policy in an End-to-End manner to identify the most informative samples for prompt learning. Moreover, UST leverages the CLIP from the previous round to generate reliable pseudo-labeled data based on uncertainty and confidence of average predictions, thereby deepening the understanding of the overall data. Extensive experiments conducted on diverse real-world datasets validate the effectiveness of our PSP.

Published as a conference paper at ICLR 2026
PSP: PROMPT-GUIDED SELF- SAMPLING POLICY FOR ACTIVE PROMPT LEARNING
T. Sen Tao 12,,344 Kaiduo Feng 3.44 Jiawei Ping Zeng' Yongchao Xu²,
Feng!.3.4
Yufei Zheng?, Zheng-Jun Zha
State Key Laboratory of Robotics, Shenyang Institute of Automation, CAS, Shenyang, China 2 University Science and Technology of China, Hefei, China
3 KKey Laboratory of Networked Control Systems, Shenyang Institute of Automation, China University of Chinese Academy of Sciences, Beijing, China
tacsen23emaile uww..uc.cccoo.. (jwliu6, zhazj) Buste. edu, en, zpžsia.cm
ABSTRACT
Active Prompt Learning (APL) using vision-language models (e.g., CLIP) has attracted considerable attention for mitigating the dependence on fully labeled dataset in downstream task adaptation. However, existing methods fail to explicitly leverage prompt to guide sample selection, resulting in the selected samples being ineffective in facilitating the prompt template's downstream task adaptation, while also overlooking valuable complementary information in the unselected samples. To fill this gap, we propose a novel Prompt-Guided Self-Training Sampling Policy (PSP) for APL, which integrates Soft Actor-Critic with customized real- -eal-pseudo hybrid reward and vectorized critics incorporate prompts in guiding sample selection toward those that facilitate the optimization of prompt template, by jointly considering both selected and unselected samples. Specifically, PSP comprises two prominent components: Vectorized Soft Actor-Critic Sampling Policy (VSSP) and Uncertainty Augmented Training (UST) mechanism. VSSP customizes a real-pseudo hybrid reward based learned prompts and image features, which is fed into vectorized critics to estimate Q-value for each sample and compute gradients that optimize the actor, allowing it to refine its sampling policy in End-to-End manner to identify the most informative samples for prompt learning. Moreover, UST leverages the CLIP from the previous round to generate reliable pseudo-labeled data based on uncertainty and confidence of average predictions, thereby deepening the understanding of the overall data. Extensive experiments conducted on diverse real-world datasets validate the effectiveness of our PSP.
INTRODUCTION
Recent research in pre-trained Vision-Language Models (VLMs) has demonstrated impressive performance across various tasks, largely through prompt learning that fine-tunes a small set of parameters within learnable prompt fully labeled dataset. For instance, Contrastive Language- Image Pre-training (CLIP) (Radford et al., 2021) is representative model that consists of image and text encoders using contrastive loss function, trained on 0.4 billion text-image pairs, and is renowned for its robust transferability. Building on CLIP (Radford et al.) (021). Zhou et al. proposed CoOp Zhou al.) 20226), a notable approach that freezes both the image and text encoders, enabling learnable context vectors to serve as templates. However, the resource consumption required for annotation remains substantial.
In response, some researchers have turned to active learning, which selects the most informative samples within a limited annotation budget to maximize performance (Xie et al., (023). The core challenge of active learning lies in formulating effective criterion for sample selection. Conventional active learning methods are divided into three categories based on the sampling
"Corresponding authors.

---

Published as a conference paper at ICLR 2026
Learnable Prompt
LLamed Pronpe
in
Nundow CLIP
Vectonizal
Criiks
I Learnable Prompt-
Loboled
Feaner
Labeled Dana-
CLI
Model Inference
in Round I
Pronupt Louriing
Gradient
BADGE
Query Ser
BS
Coreect
Query Sei
MS
Unloboled
Oracle
Entropy
Unlobeled Data
Uracle
FSSP
Action
Data
(4) PCB
(b) PSP
Figure I: Illustration of (a) PCB (Bang et al., 2024), (b) Our PSP. PSP introduces the Vectorized
Soft Actor- Critic Sampling Policy (VSSP) to replace the conventional sampling algorithm in PCB while filtering reliable pseudo-labeled data through UST.
algorithm: (1) Uncertainty-basee sampling (Lewis & Catlett 1994 Gal et al. 2017 Kirsch et al.
2019: Holub et al., 2008) selects the most uncertain samples, such as Entropy (Holub et al. 2008). (2) Diversity-based sampling prioritizes ensuring that the queried samples represent the entire data
distribution, as in Clustering (Hu et al. 2021) and Coreset (Sener & Savarese 2018).
(3) Hybrid
sampling aims to query informative samples by jointly considering uncertainty and diversity, such as
BADGE (Ash et al. 2020). ALFA-Mix (Parvaneh et al. 2022) and GCNAL (Caramalau et al. 2021). (4) RL-based sampling formulates sample selection as a policy optimization problem, leveraging Reinforcement Learning (RL) to iteratively refine this policy and select samples that maximize the cumulative reward, such as DRAL (Liu et al. 2019) and AOL (Woodward & Finn 2017).
These approaches generally rely on smaller foundational models, such as ResNet (He et al. 2016). which lack the commonsense knowledge and specialized domain expertise that larger models possess. Therefore, Bang et al. (Bang et al. 2024) introduced the pre-trained CLIP into active learning and proposed Active Prompt Learning (APL), along with the Pseudo-Clas Class Balance (PCB) framework. As shown in Figure T a), PCB mechanically applies conventional sampling methods (ie., Entropy, Coreset, and BADGE) to select candidates, which are then sent to a Balance Sampler (BS) to create query by preferentially selecting candidates whose pseudo-labels correspond the most under- represented classes in labeled data. However, three PCB variants isolate sample selection and prompt learning in the APL task, lacking an explicit connection between the two, which conceptually inap- propriate. Additionally, three PCB variants also overlook complementary information in unselected samples, limiting further improvements in model performance. Therefore, in the context of APL, we urgently need a method that bridges these two stages by explicitly leveraging prompt to guide sample selection and fully exploiting the complementary information in unselected samples.
To address these issues, we propose novel Prompt-Guice Guided Self-irainng Sampling Policy (PSP) for APL, which combines Soft Actor-Critic (SAC) (Haarnoja et al.) 2018) with a tailored real-pseudo hybrid reward and vectorized critics to integrate prompts in directing sample selection toward those that advance the optimization of prompt template, by jointly considering both selected and unselected samples, shown in Figure [[b). Specifically, PSP establishes self-training teacher-student framework composed of two key components: Vectorized Soft Actor-Critic Sampling Policy (VSSP) and Uncertainty Augmented Self-Training (UST) mechanism. VSSP first designs actor map the gradient embeddings of samples from unlabeled data pool into action, where each element represents the probability of selecting given unlabeled sample. Next, VSSP utilizes Multinomial Sampling (MS) to construct the query set in round I, acquires real-labeled data through Oracle annotation, and combines it with pseudo-labeles data from UST for the student CLIP's prompt learning. After prompt learning, the learned prompts and image features are integrated into the computation of the real-pseudo hybrid reward, which is then passed to vectorized critics to estimate the Q-value for each sample and derive the actor's gradients, enabling it to refine its sampling policy in an End-to-End fashion and effectively identify the most informative samples for further enhancement. To harness the complementary information in unselected samples, UST employs the teacher CLIP model from round 1 to generate reliable pseudo-labels by evaluating the uncertainty and confidence of the average predictions. Extensive experiments on multiple datasets prove the effectiveness of our PSP.
The main contribution of this work can be summarized as follows: (i) We propose a novel Prompt- Guided Self-Training Sampling Policy for active prompt learning, combining SAC with customized real-pseudo hybrid reward and vectorized critics to guide sample selection towards those that promote the optimization of prompt template. (ii) We construct the Vectorized Soft Actor-Critic Sampling
2

---

Published as a conference paper at ICLR 2026
Policy, which tailors real-pseudo hybrid reward based learned prompts and image features, feeding it into vectorized critics to compute gradients of the actor, allowing to refine its sampling policy and identify the most informative samples for prompt learning. (iit) We develop Uncertainty Augmented Self- Training mechanism, which generates reliable pseudo-labeled data based on the uncertainty and confidence of the average predictions to reveal data structures not reflected in the real-labeled data.
2
RELATED WORK
Active Learning identifies criteria for selecting the most informative samples under a limited labeling
budget. Based on the criteria, active learning methods be categorized into three main approaches:
Uncertainty- -based sampling (Gal et al. [017 Wang et al.] (201)). Diversity- based sampling (Hacohen
et al. et al. 2021) D. and Ra-eb sampling (Ash et al. 2020 Kirsch et al. 2019). For Uncertainty-based
2022 Shui al. 2020), Hybrid sampling (Ash et al. 2020; Parvaneh al. 2022. Caramalau
sampling. Entropy (Holub al., 2008) selects the samples with the highest entropy for annotation on object recognition. For Diversity-
sampling, Coreset (Sener & Savarese 2018)) provides an
approximate upper bound on the loss for feature space coverage-based active learning algorithms. For Hybrid Sampling, ALFA-Mix (Parvaneh et al. (2022) utilizes unlabeled data to support active learning by interpolating between the representations of labeled and unlabeled instances and identifying features the model fails to recognize through inconsistencies in predicted labels. For RL-based sampling, AOL (Woodward & Finn (017)) combines meta-leerning and reinforcement learning for one-shot classification tasks. DRAL (Liu et al. 2019) designs agent in acquiring pairwise annotated data. Notably, PAL (Fang et al. 2017) builds a deep Q-network as an adaptive policy for sample selection. Therefore, believe that RL-based methods have the potential to incorporate prompts for guiding sample selection. However, AOL (Woodward & Finn 2017) and PAL (Fang et al. 2017) model the decision of whether annotate streaming unlabeled sample
binary classification
problem, while MedSelect (Vrabac et al. (022) and DARL (Liu et al. 2019) rely on pairwise data, making them unsuitable for direct application in Active Prompt Learning (APL). Therefore, we introduce Soft Actor-Critic (SAC) (Haamoja et al.) 2018). a representative reinforcement learning algorithm known for its robustness to hyperparameters and strong performance in continuous action spaces. By designing a customized real-pseudo hybrid reward and vectorized critics, SAC can be seamlessly integrated into API
Vision-Language Models have recently demonstrated remarkable advancements in downstream tasks, leveraging their robust transfer learning capabilities. A prominent example is CLIP (Radford)
ett al. 2021), which has been extensively adopted across wide range of downstream applications
(Yu et al.| 2023. Zhou et al. 2023 Ning et al., 2023 Liang et al.
2023 Jia et al. 2022). Inspired by
prompt optimization in natural language processing (Jiang et al.
2020
Khattak et al. 2023), , CoOp
(Zhou et al. 20226) is a representative approach that transforms context words into learnable context
vectors via text encoder. Building this, CoCoOp (Zhou et al.
2022a) further refines the learnable
prompt by adapting it to individual image instances.
Active Prompt Learning (APL) resolves the dilemma between the need for additional labeled data to enhance prompt learning and the high cost of data annotation. It annotates valuable samples for prompt learning within a fixed budget, improving performance on downstream tasks. Furthermore, Bang el al. (Bang et al. 2024) proposed the Pseudo-Class Balance (PCB) framework for APL, which employs selection algorithms such as Entropy (Holub et al.) (2008). Coreset (Sener & Savarese 2018). and BADGE (Ash et al. 2020) to identify candidates. These candidates are then passed by a balance sampler to select candidates whose pseudo-label, correspond to the most underrepresented classes in the labeled data. Notably, Entropy is prone to noisy data and outliers, while Coreset prioritizes diversity but also includes less informative samples, and BADGE relies on fixed rules to balance diversity and uncertainty, limiting its adaptability across tasks. More importantly, these works treat sample selection and prompt learning as two decoupled, discrete stages, with a lack of explicit connection between them. which makes the APL task fragmented. In contrast, our method bridges these two stages by refining the sample policy through a customized reward derived from the prompt learning process, thereby explicitly leveraging the prompt to guide sample selection.
3

---

Published as a conference paper at ICLR 2026
Clrl Dtog. Flaner
Text Features
Text Encuder
Uncertainty Augmented Self- Trainingg Mechauism
Prompt in Rooldd 6-1 Augmentations of Unlabeled Data
Logiss
Pooudo-Labeled
Teacher
Image Features
BPLS
Dog
Image Encuder
log Pm
Prompi
Lourning
Image Encoder
rt
Qultt, ae?
Laboled
Parudo aaooled
diee Stel
Verteetedd Qualitr
Q-Critic
Student
Cc
Unlabeled Datu
M
M Q
KIT
Replay Buffer
Text Encuder
Prompr Round
Vactoriord state volue
V-Critic
Prompti
ESSP Updating
Query
VSSP
MS
G (8r. B.)))
Satisfy ny
81
x7
xf
Round 1
Muintnomial Sampling
Indexer
log Pm
Pu
G2y Sampling
Actor
St
The overall structure of our PSP. The CLIP collaborative learning framework for PSP consists of two core components; the Vectorized Soft Actor-Critic Sampling Policy (VSSP) and the Uncertainty Augmented Self- Self-Training (UST) mechanism.
Figure 2:
METHODOLOGY
3.1 OVERVIEW
PSP comprises two crucial components: the Vectorized Soft Actor-Critic Sampling Policy (VSSP) and the Uncertainty Augmented Self-Training (UST) mechanism, as described in Figure 2 VSSP employs the Actor network map the gradient embeddings of n² unlabeled samples into action a; = {ai, .2, Onz) ), where each element indicates the probability of unlabeled sample being selected. Next, VSSP utilizes Multinomial Sampling (MS) to derive sampling results G = {M), .2.............................. 9ay }. which are combined with a, and then fed into the Indexer index the query set round t. MS additionally outputs the MS indicator log Pm to reflect the quality of the sampling scheme. Ultimately, Oracle utilized to annotate the query set, thereby yielding the real-labeled data. Simultaneously, UST leverages the teacher CL.IP F'(Fi and Fi. to obtain the pseudo-labels for the remaining unlabeled data. To mitigate the disruptive effects of noise in pseudo-labels, UST introduces the Balanced Pscudo-Labeled Selective (BPLS) module, which filters out balanced and reliable pseudo- labeled data by jointly evaluating the uncertainty and confidence of the average predictions across L augmentations. Real-labeled data is combined with psbele-labeled data for the student CLIP F'FF; and F&) in prompt learning, optimized with cross-entropy loss Lee- In prompt learning, the text prompt P. for class is constructed by appending the class token [cls.] the end of sequence, as illustrated below:
Pe [c]] [c]
where [c/m indicates the learnable context vector of the prompt Pe. with dimensions matching those of the word embeddings, while M represents the prompt size. We design experiments regarding learnable prompt analysis in Appendix A.2 Following PCB, we incorporate spasi-c descriptions generated by GPT-3 (Brown et al.) 2020) for augmentation to further enhance prompt learning, as detailed in Appendix A.4] After prompt learning, the real-pseudo hybrid reward r. is computed by the product of log P. P.. and the sum of a real-labeled reward r. and pseudo-labeled reward Ty. Since the unlabeled data pool excludes the query set, the next state $141 is calculated by feeding n unlabeled samples into the student CLIP after prompt learning. At the end of each round, experience tuple (s,, a,, Ft. 1;+1) is stored in replay buffer, which is subsequently used to update the networks in VSSP.
4

---

Published as a conference paper at ICLR 2026
3.2 VECTORIZED SOFT ACTOR-CRITIC SAMPLING POLICY
VSSP introduces prompt to guide sampling policy in selecting the most informative samples for prompt learning, thereby bridging sample selection and prompt learning- two stages that previous works have treated in isolation. Next, we provide a detailed elaboration of our VSSP.
We define APL by the tuple- < S.A. Ps. >, where the state space S and the action space A are continuous, the state transition probability of the next state 8141 given the current state 8t and action a,, and the reward r S A + R. We also adopt a vectorized state value function (st), vectorized Q-function Qo(se. az), and a sampling policy & (a, st), with network parameters v) 8, and ø. inspired by SAC (Haarnoja et al. 2018)) The Q-Critic and V-Critic networks are modeled as fully connected networks, while the sampling policy built as a Gaussian distribution, with its mean and covariance predicted by the Actor network.
S x S x A - [O, [0o+ denotes the probability density
State. Given the unlabeled data pool Du. VSSP defines the state E R XM, a matrix representing the gradient embeddings of the nž unlabeled samples. Here, M, = K x Di denotes the dimension of the gradient embeddings, K indicates the total number of classes and Dí, is the
dimension of the teacher image features fi' (x)). Gradient embedding incorporates prompt information to enrich the state representation, offering richer gradient insights during sampling policy updating compared a single image feature. The analysis of state modeling is executed in Section 4.3 Formally, the i-th value of 8, is expressed below:
sV [1- ff)), ifc= 9
(2)
- c (V.), i # d
Here, (F) f denotes the score that an unlabeled sample an belongs to class c, for = 1,2..., K. Bs represents predicted category of unlabeled sample x...
Action. We define the action as a vector a, E R" where each element represents the probability of the unlabeled sample selected by the actor. The sampling policy generates the action a, E R based on the current state 8y. After obtaining the action vector, VSSP adopts the Multinomial Sampling (MS) to obtain the query set, which introduces randomness and helps to distribute the selected samples more evenly. The sampling results G = {91. (91.92..... .2........v..................... 9xp } in MS follows a Multinomial Distribution, where 9j denotes the number of times xx is selected, and satisfying n, = Σ Di- Therefore, VSSP defines the log probability of the sampling scheme as MS indicator to evaluate the quality of the sampling scheme.
nad
(gg(pm(g)) log( g1! g2!
Si log a;
Inp!
A larger value of log(pm (g)) indicates that the current scheme aligns well with the distribution of ae and vice versa. The sampling results and action are fed into the Indexer retrieve the selected samples, replacing any duplicates with the sample that has the higher probability. Hence, VSSP obtains the query set {=; 1= which is then presented the Oracle, resulting in the labeled set {a) of _ The real-labeled data are indicated as D. = [xU v |e Mcanwhile, the unlabeled data pool will exclude the labeled set, i.e., Du \ (=t)
Shape-Variable Variable State Transition. The real- labeled data Dj is incorporated with the pseudo-labeled data Dp = {xf D. "p for the student's prompt learning with a cross-entropy loss. After prompt learning, the next state 8141 is obtained by feeding the remaining ni+1 unlabeled samples into the student CLIIP, where n
n n. Consequently, the state transition exhibits a variable shape, leading to alignment issues in Equation 8 when optimizing the Q-Critic network, distinguishing it from SAC (Haarnoja et al. 2018
Reward. To explicitly utilize prompt for guiding sample selection, we first propose the real-pseudo hybrid reward, inspired by (Wang et al. 2020), as outlined below.
r(s a.) log(p..(g)) (r. + (Tp,)
(4)
X r/ max cos (FF)
- cas( FF (x+)))
(5)
where ri denotes the reward for a single sample, reflecting the quality of individual prediction, k E {8,p} indicates the data type, where (#f, vf) denotes the real-labeled sample with label yf, and
5

---

Published as a conference paper at ICLR 2026
(af, v.") denotes the pseudo-labeled sample with pseudo label yf" D,' where D." is the prediction of the teacher CLIP model, and coefficient /8 represents the contribution of pseudo-labeled rewards. Notably, F, indicates the mean of vector Ts- Considering that MS indicator log(pm. (g)) is negative while r. and 'p are positive, maximizing the real-pseudo hybrid reward results in a reduction of rss Brp. which improves the model's classification ability. log(Pm (g)) is closely related to the construction of the query set for labeling and conveys information for the subsequent training of the actor and critic, reflecting the quality of the sampling scheme.
Training. As shown in Algorithm|] T each round stores an experience in the replay buffer. Given the limited number of experiences, VSSP trains the actor and critics by sampling a single experience per gradient step once the number of stored experiences exceeds the buffer threshold Tb, with the analysis of Tu are provided in Figure 5 from Appendix A.2 In API., the selection of each sample is made independently. However, using a scalar state value and Q-value assigns shared global information value to all samples, preventing the differentiation of individual sample contributions for sampling policy updating. To address this, VSSP employs a vectorized V-Critic and Q-Critic to estimate the state value and Q-value for each sample, enabling finer-grained control over each sample's 's role in optimizing the sampling policy. The V-Critic network updates its parameters by minimizing the following squared residual error.
Jv (v,
Voo(s) U. 1B
(6)
where B denotes replay buffer that stores history experiences, U. Eaiwso (Q,(s, a,) log Th (a(a]ss))] indicates the target value for training the V-Critic net- work. Qo(Sr, ax) E "i"" represents the Q-value predicted by the Q-Critic network for the current state 8 E IR"" MM and action a,, and (s:) E R. indicates the state value predicted by the V-Critic network. The unbiased estimates of the gradient of Equation computed as below:
Vv) (V.(s.) Qo a.) log logns(as|s:)) V Vo.))
(7)
Here, the action a is drawn according to the current policy TT(-)st). instead of the replay buffer B. log n (a)(())) broadcasted to match the shape of (s)) Similarly, the Q-Critic network is optimized to minimize the modified soft Bellman residual:
( = (a)
QQi(s, at) Q' (8. 1
(8)
where Qú: Q' Q'= W[(Q Q) indicates the aligned target Q-value and Q-value after alignment through Soft Dynamic Time Warping (Soft-DTW) (Cuturi & Blondel 2017). as detailed in Appendix A.5 We adopt Soft-DTW for alignment because it preserves the relative order and structural relationships between elements, which is crucial given the strong correlation between target Q-value and Q-value. In addition, the differentiability of Soft-DTW enables gradients to propagate through the alignment process, ensuring that does not interfere with the update procedures of either the Actor or the Critic.
The target Q-value Q E Th="" is defined below:
Q(s.. r(st, a.) + - [Vg(+++i]
(9)
where 7 denotes the discount factor, (8++1) E "R represents the state value predicted by the target V-Critic network with parameters σ.. The sealar reward r(st, ax) E R is broadcasted to match the shape of (8++1).
The state is vaapelvariable, causing Qo and Q to have mismatched dimensions, making direct subtraction infeasible in the soft Bellman residual of SAC. Soft-DTWW addresses this issue by optimizing smooth, differentiable relaxation of the optimal matching cost, ensuring that Q/ is differentiable function and preventing information loss result from cropping or padding. Consequently, the stochastic gradient of Equation 8 is illustrated below:
(10) The parameter update uses a target V-Critic network Vg.. whose update a weighted average, with hyperparameter controlling the degree of mixing between the V-Critic network parameters v and the target V-Critic network parameters vb.
(0) (Q' a.)
('(s., (o a.)

---

Published as a conference paper at ICLR 2026
Algorithm 1 VSSP
Algorithm 2 UST
Initialize parameter vectors v/, 4/
Initial: Dp
Ø.
for each unlabeled sample do
2: for round in range (It) do
2:
{ 11=1
x;
3:
(T())
3:
z Fa
udl
4: S(+1 Ps (= at)
5:
t PEBB(((S ((s,, d,, Tt, S(+1)}
4:
Dr argmax 2, 20 = argmax Σ Zi
6:
for each gradient step do
5:
g con fidence(z
7:
v - v vv( 0 - dQ (0) o A V 1 - Tt + (1 7)t6
6:
9 std (oonfidence(((( (ddence(2)){+
7:
if of Z To and 9.¹ S Ta then
8:
8:
where
# Σ Tu
9:
B ΣE= 1 ghd and denotes batch size.
9:
D t Dp (ax N. y2 0
11: end for
10: end if
12:
end for
end for
To ensure that VSSP effectively explores while maintaining stable and efficient convergence during training, we incorporate the reparamcterization trick, which introduces stochasticity by reparameter izing the policy through the Actor network follows:
a = fal (E:; 8,) f (s.) + (8)
(11)
where is an input noise vector sampled from a standard normal distribution, (s, and fg (s:) indicates the mean and covariance predicted by the Actor network, respectively. The goal of the Actor network is to learn the policy that maximizes the following objective (Ziebart 2010):
nn Σ Th
(+( Ba - log no (fole;; 8.)
(12)
Here, T is defined as Gaussian distribution based on the Actor network outputs (s,) and (B)). The gradient of Equation 12 can be approximated as follows:
(.)
(oso.a) Fo nh
log (a'ls.) "
Vololeris))
(13)
3.3 UNCERTAINTY AUGMENTED SELF-TRAINING
The UST mechanism is designed to uncover the latent, task-specific knowledge embedded in unlabeled data that prior works have overlooked, thereby enriching the model's understanding of the overall data distribution. Notably, we compare UST with other semi-supervised (Chakraborty et al. 2024 and unsupervised (Huang et al. 2022) prompt learning methods further assess effectiveness,
as detailed in Table 4 from Appendix A.2 As outlined in Algorithm 2 UST begins by employing
the same data augmentation to sample x. from remaining unlabeled data D. [ax ) to generate
augmentations of each unlabeled sample (ze JI=1 After freezing the learnable prompt from round t construct the teacher CLIP model, UST feeds L augmentations of the unlabeled data into the teacher CLIP model F to compute logits for each augmented versions of the unlabeled samples {z])[_ To achieve stable predictions from the teacher CLIP model, UST calculates the average of the logits z4vg to obtain the average prediction ú!!", which serves the pseudo-label for the unlabeled sample x.
1
Considering that noise in pseudo-labels can interfere optimization process, UST designs a Balanced Pseudo-Label Selective (BPL.)) module to filter out samples with more reliable pseudo-labels by jointly evaluating prediction uncertainty and confidence. In BPLS, of represents the confidence score computed from the average logits 2," anw across augmentations of unlabeled data, while g." quantifies uncertainty using the standard deviation of confidence scores from different augmented versions. BPLS selects samples where of exceeds the prediction confidence threshold т. and g is below the prediction uncertainty threshold T.. Samples meeting these criteria incorporated into the reliable
7

---

Published as a conference paper at ICLR 2026
pseudo-labeled data Dp. After filtering all samples, UST identifies the missing categories in the filtered pseudo-labeled and selects high- nigh-confidence samples corresponding to those categories, filling the gaps until the number of samples matches the minimum class count in Dz. Finally, pseudo-labeled data are combined with real-labeled data for the student CLIP's prompt learning, enabling a more comprehensive understanding of the intrinsic structure and relationships within the data.
4 EXPERIMENTS
4.1 EXPERIMENTAL SETTING
Datasets & Metrics. We adopt seven commonly used datasets to evaluate our PSP. These datasets encompass diverse categories and are sufficient to demonstrate that PSP can address various real-
world scenarios, including Stanford Cars (Krause et al., 22013) AT (Helber et al. 2019),
FGVC-Airerati (Maji et al. 2013), Caltech101 (Fei-Fei et al. 22044, DTD (Cimpoi et al. 2014). Flowers 101 NNilsback& & Zisserman 2008) and Oxford Pets (Parkhi et al., 2012). We use the "Final
Acc" metric to represent the accuracy of the last round, while the "Average Acc" metric provides a comprehensive evaluation of PSP by computing the average accuracy of the final round across all datasets. All experiments were conducted three times, and the results are reported as average values.
Implementation Details. We adopt PCB (Bang et al. 2024) as the baseline model, following the setup of eight rounds (i.e., R = 8). In each round, we select the query set whose size corresponds to the number of classes in the datasets, i.e., ni = K. For VSSP, coefficient [3, hyperparameter UST, we apply the same data augmentation to the
and 7 are set to 0.7, I and 0.9, respectively. remaining unlabeled data five times, i.e. L = 5, which involves random resized crop to 224x224 with scale=(0.08, 1.0), random grayscale conversion with probability 0.2, color jittering, random horizontal flip, and normalization. All the experiments are conducted using the PyTorch platform and executed on NVIDIA RTX 3090 GPUs. More implementation details can be found in Appendix A.
Table 1: Final accuracy on these commonly used downstream tasks using the VIT-B/32 image encoder. The performances with the pre-trained zero-shot CLIP model are reported from (Rakesh & Jain 2021). The performance with the entire labeled dataset during prompt learning is marked as Fuuly Labeled Data", serves the upper bound for comparison.
Method
DTD
Oxford Pets
FarSSAT
Flowers102
Caltech101
Stanford Cars
Aireraft
Average Ace (f)
CLIP ieero Shot)
44.5
67.0
69.4
66.7
87.9
59,4
21.2
59.64
Random
58.77=1
78 00.00-0744 74
77.62=1.12
92.92-0611
89.55=1.00
65 05-06:-::00
30.0900331
GCNAL Commmba
$2.09:0.55
12:13:30:3
93.1910.23
92.44:0.00
65.3410.30.
29 48
72.
ALBA vannno
61
83.1320.13
82.39
96.7620.17
95.37=0.11
71.0480.67
27.83::0225
74.01
Earrpy
Holob et al. 200K
18:1131
:6:81:1130
04.80:0.75
91.67:0.09
68:00.999
25. 05.80:0078
+ AB
60801110
18.35
79,97222.70 70
96.06.0063
92.87-0.20
65 (3.9910..0
26.69
71.33
+ AS
81
79 88-88-144 43
19.88-0.445
05.67=1.10 19
93 93.28-0055 55
68. $4+11.09
26.04=11
71.75
+
PCB (Bang et al. 2024
59.73.1 96
50. 64
24
50.02 2.88
96,16 10.045
10,50
67. 18.10.3
26, 06.88
71.93
PCB (XE)
60072:100 69
30.8=00000 00
81.72:00.55
96. 33:03:00066
01.14
66 42:422 86
27.09:0.13
72.23
PCB (AS)
59.50++1 99
90-94-111 00
10.75--1:15
96,94 16.94-0.19
93.48-0.22 26
68 93+0 86
27. 58-00-33
72. 2..90
Coresct Sener Savarese 2018
50.39.10.54
76.70 0..52
68.
$8.65100.6
88.78_00.4
61.7510.80
24.3210.45
53
$1,89=1
78.08=
67.02-22.80
29.66
XM.99
60.65:01.5 53
25
52.76:1122
39.73:10.9
90.63-0.55
15:00.77
26. 11:016
(Bang et al.
55.7711.33
47.50
91.3010.900
09,95
63.6310.27
25. 25.38.30.64
+ PCD (X0
57.09 60
78. 1.44
79.28
91,7010.22
90.29
62.0810.335
26. 1911111.0
69.31
* PCB (AS)
76.38:00773
79, 79,50=991
70.28:1144
92.33:20444
91. 70:70=:000 48
65 75.75:51155
26:02:0.47
70.17
RADGE Ath 2020
08-88-11 38
50.03-1110 19
70.79-0.04 04
06.35-H0.30 39
92. 02.54-0001
68 08
31.25-0:45
72.43
+ AB
59. 59.9711071
$1.9910.55
$0,57.1.40
96.24.10.29
92.93.00022
67. 67.1010447
31.04.0.322
12.83
ASS
61. 61.52+=1 25
82.33=0.72 72
81 81.66=00.4
96.44=0010 16
08.79=0.25 25
70 70.56+00331
31.79-0.74
74.01
PCB (Bang al. 2024
60. 00.28::100
$0.22::1 69
$1.98:0.51
96. 12:10
92.21 0.92
68 56:10...66
31. 31.35:0.21
72.95
PCB
61.9211100
81,45:00.8 8M
80.7023367
96.35-10.27
92.52=0.32
67.70: 848
31. 80.0000
PCB
62.33+1100
81.50-1.11
96.71-4020
93.85-0.31
70---077
32.27-0.66
PSP
65.6610.38
86.57
85.43.10.06
Fully Laheled Data
74.7
89.3
94.5
97.9
94.4
80.8
82.14
4.2 OVERALL RESULTS
We evaluate our PSP against three PCB variants that implement different description augmentation: one that omits augmentation, one that uses the average score, and another that employs the average embedding, denoted as PCB, PCB (AS), and PCB (AE), respectively. The details and experimental results of description augmentation are provided in Appendix A.4 Moreover, we compare our PSP with the pre-trained zero-shot CLIP, ALFA-Mix, GCNAL and the Random approach that randomly selects a query set in each round. The results on downstream tasks with ViT-B/32 image encoder are summarized in Table For smaller datasets, PSP achieves improvements of 3.33%, 3.41% % and 4.15%
8

---

Published as a conference paper at ICLR 2026
Table Final accuracy with the ViT-B/32 CLIP image encoder on DTD, The baseline model is combined with UST, and VSSP.
Method
DTD
Oxford Pets
EuroSAT
Aircruft
Average
wlo
wlo UST
63.77
85.55
83.97
32.40
66.42
PSP
65.66
86.57
85.43
36,42
68.52
Full Labeled Data
74.7
89.3
94.5
43.4
75.5
over PCB (AS) DTD, Oxford Pets and Aircraft, respectively, For larger datasets, PSP demonstrates gains of 3.14% and 3.93% over PCB (AS) on Stanford Cars and EuroSAT, respectively. Furthermore, PSP achieves improvements of 2.51% the "Average Ace" metric across these datasets. These results provide convincing evidence that our PSP effectively enhances performance.
SACEMSI
Saf-dTWW
SACTTOpK
PSPOESI
PST(lopk)
(a) SAC
(b) Alignment
(c) Multinomial Distribution
Figure 3: Influence of different designs of the Vectorized Soft Actor-Critic Sampling Policy, (a) Different query strategies (i.e., MS and TopK) in VSSP. TopK means selecting the samples with the highest probabilities the action. (b) Various alignment algorithms in VSSP. (c) Different usages of MS indicator within VSSP.
Table 3: Ablation study on DTD, evaluating the impact of hyperparameter 3.
3
Final Accuracy
8
Final Accuracy
0.1
64.95
0.7
65.66
64.66
4.3 ABLATION STUDY
Effectiveness of each component in PSP. We present the influence of each component in PSP in Table reporting the final accuracy on DTD, Oxford Pets, EuroSAT, and Aircraft. wo VSSP indicates the removal of the sampling policy, which is equivalent to PCB combined with UST, while w/o UST denotes the absence of pseudo-labeled data, corresponding to PCB integrated with VSSP. We can notice that the average performance across four datasets reduced by 1.26 % and 2.10 % respectively when sampling policy and pscudo-labeled data are removed. These results suggest that both VSSP and UST are crucial for effectively guiding the student CLIP" prompt learning. Consequently. we conclude that guiding the sampling policy with prompts effectively enhances the optimization of the prompt template by considering both selected and unselected samples.
Effectiveness of each parts in VSSP. To study the effectiveness of different designs for VSSP, we conduct ablation studies on DTD and report the accuracy for each round. VSSP built upon SAC with vectorized critics and real-pseudo hybrid reward. First, we remove the vectorized crities from VSSP, denoted as SAC (MS), and observe substantial performance drop compared with PSP (MS), detailed in Figure 3p. These results indicate that the vectorized critics play an indispensable role in achieving the performance gains. Moreover, we replace MS with TopK as the query strategy within VSSP, denoted PSP (TopK). As shown in Figure 3,, PSP (MS) consistently outperforms PSP (TopK), validating that MS is more compatible with our PSP.
Second, analyze the impact of different alignment methods (i.e., Soft-DTW, PAD, and VAE) on DTD, as shown in Figure 33.. We adopt Soft-DTW as the default alignment algorithm, which improves the final accuracy by 1.83% % and 0.83% compared to PAD and VAE, respectively.

---

Published as a conference paper at ICLR 2026
Third, remove the real-pseudo hybrid reward from VSSP (i.e., removing the MS indicator in Equation 4 denoted as "wlo") and observe a 1.83% performance decrease, as shown Figure Bo This finding demonstrates that real-pseudo hybrid reward has significant impact performance. Notably, the results show that using a positive coefficient improves the final accuracy by 1.06 % compared to the negative coefficient, This supports analysis that since MS indicator is inherently negative, maximizing the reward leads to a smaller absolute sum of real and pseudo rewards, which correlates with better classification performance.
Average accuracy in each round. To more thoroughly analyze the performance of PSP in each round, we report the average accuracy across seven commonly used datasets, referred to as the learning curve. As shown in Figure 4 PSP consistently outperforms all three PCB variants after the initial round, with the performance disparity progressively increasing. has been validated that PSP has almost achieved an enhancement in overall performance in each round compared to the three PCB variants.
80
Accuracy (%
80
BADGE
02
Entropy
50
PCB(Coreset)
PCB(Enttroy)
PCB(BADGE)
PSP
Figure 4: Learning curve. Average accuracy across downstream tasks with the ViT-B/32 image encoder for each round.
Hyperparameter sensitivity. To determine the optimal coefficient 33, we compare PSP with different values of the hyperparameter in Table 3 The results show that PSP exhibits robustness with respect to the hyperparameter 39. Ultimately, we select / = 0.7 as the default setting for superior Algorithm[] the target value smoothing coefficient 7 is used stabilize the training of the V-Critic network. We conduct experiments comparing different values of the and conclude that PSP insensitive to r, showing minimal performance variation. Additionally, PSP is robust to the discount factor More ablation study can be found in Appendix
performance.
A.2 A.2
CONCLUSION
In this work, propose novel Prompt-Guided Self-Training Sampling Policy (PSP) for APL, which integrates SAC with a tailored real-pseudo hybrid reward and vectorized critics to leverage prompt in steering sample selection toward those that drive the optimization of prompt template, by jointly considering both selected and unselected samples. PSP constructs a self-training framework composed of VSSP and UST. VSSP utilizes learned prompts and image features to compute real-pseudo hybrid reward, which is fed into vectorized critics to estimate the each sample's Q-value and compute gradients for actor's optimization. This process enables the actor guided by the prompt to refine its sampling policy in an End-to-End manner and identify the most crucial samples for the student CLIP's s prompt learning, distinguishing it from PCB. UST extracts valuable complementary information from unselected samples by utilizing the teacher CLIP to generate reliable pseudo labeled data based on uncertainty and confidence. Extensive experiments prove that PSP can identify the most crucial samples for prompt learning maximize performance within constrained budget.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENT
This work was supported by National Natural Science Foundation China [U24A20277, 62225207, 62476260, 62436008, 92267205, 62503466, 92467301, 92367301J, Natural Science Foundation of Liaoning Province (2024-MSBA-83, 2025-MS-000 MM-085], the Fundamental Research Funds for the Central Universities under Grant WK2 10020000, the National Program for Funded Postdoctoral Researchers [GZB20230805], Fundamental Research Project of S1A,[2024ICI 2024JC3K03), the State Key Laboratory of Robotics and Intelligent Systems of China [2025-Z12].
REFERENCES
Jordan T. Ash, Chicheng Zhang, Akshay Krishnamurthy, John Langford, and Alekh Agarwal. Deep batch active learning by diverse, uncertain gradient lower bounds. In 8th International Conference on Learning Representations, 2020.
Jihwan Bang. Sumyeong Ahn, and Jac-Gil Lee. Active prompt learning in vision language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 27004 27014, 2024.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In Advances in Neural Information Processing Systems, volume 33, pp. 1877-1901, 2020.
Razvan Caramalau, Binod Bhattarai, and Tae-Kyun Kim. Sequential graph convolutional network for active learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9583-9592, 2021.
Omprakash Chakraborty, Aadarsh Sahoo, Rameswar Panda, and Abir Das. Xpl: A cross-model framework for semi-supervised prompt learning in vision-language models. Transactions on Machine Learning Research, 2024.
Mircea Cimpoi. Subhransu Maji, lasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. De- scribing textures in the wild. In Proceedings of the IEEÉ Conference on Computer Vision and Pattern Recognition, pp. 3606-3613, 2014.
Marco Cuturi and Mathieu Blondel. Soft-dtw: differentiable loss function for time-series. In International conference on machine learning, pp.
PMLR, 2017.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: large-scale hier- archical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248-255. IEEE, 2009.
Meng Fang, Yuan Li, and Trevor Cohn. Learning how to active learn: A deep reinforcement learning approach. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 505-605. Association for Computational Linguistics, 2017.
Li Fei-Fei, Rob Fergus, and Pietro Perona. Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. In 2004 Conference on Computer Vision and Pattern Recognition Workshop, pp. 178-178. IEFE, 2004.
Yarin Gal, Riashat Islam, and Zoubin Ghahramani. Deep bayesian active learning with image data. In International Conference on Machine Learning, pp. 1 1183-1192. PMLR, 2017.
Philippe Henri Gosselin and Matthieu Cord. Active learning methods for interactive image retrieval. IEEE Transactions on Image Processing, 1707):1200-121, 2008.
Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International Conference Machine Learning, pp. 1861-1870. PMLR, 2018.

---

Published as a conference paper at ICLR 2026
Guy Hacohen, Avihu Dekel, and Daphna Weinshall. Active learning budget: Opposite strategies suit high and low budgets. In International Conference on Machine Learning, volume 162, pp. $175-8195. PMLR, 2022.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pultern Recognition, pp. 770-778, 2016.
Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(7): 2226, 2019.
Alex Holub, Pietro Perona, and Michael C Burl. Entropy-based active learning for object recogni- tion. In 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 1-8. IEEE, 2008
Bingyu Hu, Zheng-Jun Zha, Jiawei Liu, Xierong Zhu, and Hongtao Xie. Cluster and scatter: A multi-grained active sumi-supervised learning framework for scalable person re-identification. In Proceedings of the 29th ACM International Conference on Multimedia, pp. 2605-2614, 2021.
Tony Huang, Jack Chu, and Fangyun Wei. Unsupervised prompt learning for vision- Lsioon-anguuge models. arXiv preprint ar
2022.
Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim. Visual prompt tuning. In European Conference on Computer Vision, pp. 709-727. Springer, 2022.
Zhengbao Jiang, Frank Xu, Jun Araki, and Graham Neubig. How can know what language models know? Transactions of the Association for Computational Linguistics, 8:423-438, 2020.
Ashish Kapoor, Kristen Grauman, Raquel Urtasun, and Trevor Darrell. Gaussian processes for object categorization. International Journal of Computer Vision, 88:169-188, 2010.
Muhammad Uzair Khattak, Hanoona Rasheed, Muhammad Maaz, Salman Khan, and Fahad Shahbaz Khan. Maple: Multi-modal prompt learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 19112219122, 2023.
Andreas Kirsch, Joost Van Amersfoort, and Yarin Gal. Batchbald: Efficient and diverse batch acquisition for deep bayesian active learning. Advances in Neural Information Processing Systems, 32, 2019.
Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine- grained categorization. In Proceedings of the IEEE International Conference Computer Vision Workshops, pp. 554-561, 2013.
David D Lewis and Jason Catlett. Heterogeneoos uncertainty sampling for supervised learning. In Machine Learning Proceedings 1994, pp. 148-156. Elsevier, 1994.
Feng Liang, Bichen Wu, Xiaoliang Dai, Kumpeng Li, Yinan Zhao, Hang Zhang, Peizhao Zhang, Peter Vajda, and Diana Marculescu. Open- vocauloccbulaary semantic segmentation with mask-adapted clip. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7061-7070, 2023.
Zimo Liu, Jingya Wang, Shaogang Gong, Huchuan Lu, and Dacheng Tao. Deep reinforcement active learning for human-in-the-lose person re-identification, In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 6122-6131, 2019.
Dwarikanath Mahapatra, Behzad Bozorgtabar, Jean-Philippe Thiran, and Mauricio Reyes. Efficient active learning for image classification and segmentation using a sample selection and conditional generative adversarial network. In International Conference on Medical Image Computing and Computer-Assise Intervention, pp. 580-588. Springer, 2018
Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi. Fine-grained 2013.
visual classification of aircraft. arXiv preprint
12

---

Published as a conference paper at ICLR 2026
Christoph Mayer and Radu Timofte. Adversarial sampling for active learning. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 3071-3079, 2020.
Alexander Narr, Rudolph Triebel, and Daniel Cremers. Stream-based active learning for efficient and adaptive classification of 3d objects. In 2016 IEEE International Conference on Robotics and Automation (ICRA), pp. 227-233. IEEE, 2016.
Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing, pp- 722-729. IEFE, 2008
Shan Ning. Longtian Qiu, Yongfei L Liu, and Xuming He. Hoiclip: Efficient knowledge transfer for hoi detection with vision-language models. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 23507-23517, 2023.
Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar. Cats and dogs. In 2012 IEEE Conference Computer Vision and Pattern Recognition, pp. 1498-3505. IEFF, 2012.
Amin Parvaneh, Ehsan Abbasnejad, Damien Teney, Gholamreza Reza Haffari, Anton Van Den Hengel, and Javen Qinfeng Shi. Active learning by feature mixing. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 12237-12246, 2022.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. International Conference on Machine Learning, pp. 7748-87658 PMLR, 2021.
Vineeth Rakesh and Swayambhoo Jain. Efficacy of bayesian neural networks in active learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
2601 2609, 2021.
Ozan Sener and Silvio Savarese. Active learning for convolutional neural networks: A core-set approach. In 6th International Conference on Learning Representations, 2018
Changjian Shui, Fan Zhou, Christian Gagné, and Boyu Wang. Deep active learning: Unified and principled method for query and training. In International Conference on Artificial Intelligence and Statistics, pp. 1318. PMLR, 2020.
Sudheendra Vijayanarasimhan and Kristen Grauman. Cost-sensitive active visual category learning. International Journal of Computer Vision, 91:24-44, 2011.
Damir Vrabac, Akshay Smit, Yujie He, Andrew Y Ng. Andrew Beam, and Pranav Rajpurkar. Medselect: Selective labeling for medical image classification using meta-learning. In International Conference on Medical Imaging with Deep Learning, pp. 1301-1310. PMLR, 2022.
Gaoang Wang, Nenq Hwang, Craig Rose, and Farron Wallace. Uncertainty-based active learning via sparse modeling for image classification. IEEE Transactions on Image Processing, 28(1): 316-329, 2019.
Jingwen Wang, Yuguang Yan, Yubing Zhang, Guiping Cao, Ming Yang, and Michael Ng. Deep reinforcement active learning for medical image classification. In Medical Image Computing and Computer Assisted Intervention, pp. 33 33-42, Cham, 2020. Springer.
Mark Woodward and Chelsea Finn. Active one-shot learning. CoRR, abs/1702.06559, 2017.
Yichen Xie, Han Lu, Junchi Yan, Xiaokang Yang, Masayoshi Tomizuka, and Wei Zhan. Active finetuning: Exploiting annotation budget in the pretraining- finttnniniinnnnigggggg paradigm. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 177224, 2023.
Yi Yang, Zhigang Ma, Feiping Nie, Xiaojun Chang, and Alexander G Hauptmann. Multi-class active learning by uncertainty sampling with diversity maximization. International Journal of Computer Vision, 113:1 13:1113-27,,00
13

---

Published as a conference paper at ICLR 2026
Tao Yu, Zhihe Lu, Xin Jin, Zhibo Chen, and Xinchao Wang. Task residual for tuning risionlangguag models. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 10899-10909, 2023.
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 11975-11986, 2023.
Kaiyang Zhou, Jingkang Yang. Chen Change Loy, and Ziwei Liu. Conditional prompt learning for vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 16816-16825, 2022a.
Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision- language models. International Journal of Computer Vision, 130(9):2337-22 2022b.
Ziqin Zhou, Yinjie Lei, Bowen Zhang, Lingqiao Liu, and Yifan Liu. Zegelip: Towards adapting clip for zero-shot semantic segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 11175-11185, 2023.
Brian D Ziebart. Modeling purposeful adaptive behavior with the principle of maximum causal entropy. Careegie Mellon University, 2010.
14

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1 EXPERIMENTAL DETAILS
Dataset. Stanford Cars (Kaasse et al. 2013) is a complex dataset known for its fine-grainee categorization. It contains 16, images spanning 196 car models. EuroSAT (Helber et al. comprises 27,000 Sentinel-2 satellite images grouped into 10 classes of land use and land cover, making it a valuable resource for remote sensing. FGVC-Aireraft (Maji et al.]|2013 consists of 10,200 images representing 102 distinct aircraft model variants. Caltech (Fei-Fei et al. 2004)
2019)
includes 9,146 images, divided into 101 object categories, plus an additional background category. DTD (Cimpoi et al., 2014) features 5,640 images across 47 texture categories, offering diverse selection of texture patterns sourced from natural environments. Flowers 101 (Nilsback & Zisserman 2008) contains 8,189 images categorized into 102 flower species, exhibiting considerable intra-class variation and inter-class similarities. Oxford Pets (Parkhi et al.] (012) consists of 7,349 images of cats and dogs, covering 37 different breeds.
Experimental details. In Algorithm [ the learning rate λv, Xo and X are all set to 3e-4. To ensure a fair comparison, we adopt ViT-B/32 as the default backbone for the student CLIP, the teacher CLIP in all experiments. Throughout all rounds, the prompt learning process for the student CLIP is optimized with the cross-entropy loss using SGD at a learning rate of 0.002, a batch size of 32, and 200 epochs across all datasets. For the text prompt, we adopt AS to realize the description augmentation for enhancing performance and set the size of the learnable tokens M to 16.
A.2 ADDITIONAL RESULTS
Analysis of the state modeling in VSSP. We are intend to incorporate the classification score information of unlabeled samples to construct gradient embeddings as state in Equation thereby enriching the state representation and facilitating better learning of the sampling policy. To verify the effectiveness of the classification score information, we conduct experiments where only the features of unlabeled samples are used as the state. The results indicate that using only features as the state in PSP is suboptimal, as the accuracy decreases from 65.66% to 63.36% when compared with using gradient embeddings. We can conclude that modeling the state with classification score information helps refine the sampling policy.
Further analysis of UST. To further analyze the effectiveness of UST, we conduct experiments on DTD and EuroSAT to compare the UST module with representative semi-supervised (Chakraborty al., 2024) and unsupervised (Huang et al., 2022) prompt learning methods, as shown in Table We include the following methods for comparison: UPI. (Huang et al.) (022). which uses CL.IP with a ResNet-50 backbone for both pseudo-labeling and inference; UPL* $ (Huang et al. 2022). , an enhanced version of UPL that leverages multiple CLIP backbones 100, VTT-B/32, ViT-B/16. and ViT-L/14) for improved pseudo-labee labeling, while retaining CLIP with Re backbone for inference; and XPL (Chakraborty al. 2024). semi-supervised prompt learning method that uses the same number of labeled samples as UST. Experimental results indicate that UST outperforms UPL., UPL,* and XPL both DTD and EuroSAT, demonstrating that filtering reliable pseudo-labeled data through UST effectively enhances prompt learning on downstream tasks.
4
Analysis of the accuracy and number of reliable pseudo-labeled data. To further study the quality of pseudo-labeled data, we report the accuracy and the number of reliable pseudo-labeled data on the DTD dataset. As shown in Table 5 both metries steadily increase as the rounds progress, with the accuracy approaching 94% in the final round. These results strongly demonstrate that UST effectively filters out reliable pseudo labels, and that the high accuracy of the lseudo-labeled data substantially mitigates the negative impact of incorrect pseudo labels.
Performance comparison with classical prompt learning methods. We evaluate the proposed PSP against classical prompt learning methods, including CoOp (Zhou et al. 2022b) and CoCoOp (Zhou et al. 2022a), using the same number of real-labeled training samples as our approach. As illustrated in Table PSP significantly outperforms CoOp and CoCoOp in terms of final accuracy across all datasets except Oxford Pets and 1alt Meanwhile, PSP achieves average accuracy improvements of 7.76% over CoOp and 7.38% over CoCoOp across these datasets. These results demonstrate that PSP effectively facilitates prompt optimization by annotating more valuable samples and extracting valuable complementary information from the remaining unlabeled data.
15

---

Published as a conference paper at ICLR 2026
Table Ablation study with semi-supervised and unsupervised prompt learning methods. We present the final accuracy on DTD and EuroS AT using ViT-B/32 as the image encoder for performance comparison with UPL and XPL.
Method
DTD
EuroSAT
UPL
Huane 2022
46.10
52.17
UPI.
Eriine ef al 2012
55.08
71.04
XPL
nakrabocry
2024
62.29
79.30
UST
62.65
81.59
Table 5: Analysis of the accuracy and the number of reliable pseudo-labeled data on the DTD dataset in each round.
Metric
Accuracy
45.47
47.99
83.30
90.67
91.35
92.06
93.25
93.93
Number
436
455
494
522
534
532
543
559
Analysis of the versatility of PSP for different VLMs. To further prove the versatility of PSP for different VLMs, we replace CLIP with SigLIP (Zhai et al., 2023). As shown in the Table PSP significantly enhances the prompt learning of SigL.P compared to the baseline PCB. Therefore, we can draw conclusion that the proposed method offers feasible solution for applying standard VLMs to a wider range of domains under limited labeling budget.
Analysis of the behavior of the learned sampling policy. policy, we compare it with BADGE, Entropy, and Coreset, and report the overlap ratio, computed as the intersection between the samples selected by both methods divided by the number of samples selected by our sampling policy. The results in Table 7 show that PSP exhibits relatively high consistency with Coreset in the early rounds (Rounds 2-4), while in later rounds (Rounds 5-8), its overlap with Entropy increases. This indicates that the sampling policy initially favors diverse samples to establish broad coverage, and gradually shifts to uncertain samples as the model' s classification capability improves.
To evaluate the behavior of our sampling
Notably, since the overlap between PSP and BADGE not consistently higher than that with the other heuristics, we conclude that PSP is not merely imitating fixed hybrid rule such BADGE. Instead, it continuously interacts with the prompt learning process and adaptively adjusts its sampling policy over rounds.
Analysis of model efficiency. We evaluate the model efficiency PSP against various PCB variants under the same backbone (ViT-B/32) by reporting training time in each round in seconds (e.g., 305, 380...) and the total training time in hours, as illustrated in Table Compared to PCB combined with BADGE, PSP requires less training time while achieving 3.33% higher final accuracy. The experimental results demonstrate that PSP strikes a balance between model efficiency and performance improvement.
Influence of various image encoder types. To explore whether different image encoder types affect the performance of PSP, we report the results various image encoder backbones (i.e., RN50, RN101, and ViT-B/16) in Table I0 It is demonstrated that PSP almost achieves performance improvements compared to other methods regardless of the image encoder backbone used. Notably, PSP achieves the highest performance on various image encoder backbones on the "Average Acc" metric, strongly proving that PSP comprehensively outperforms three PCB variants across these commonly used datasets.
Influence of various buffer thresholds. To determine the optimal buffer threshold, we compare different values in Figure 5 The results indicate that setting the buffer threshold n, to achieves the highest performance. Given the limited number of experiences, we sample one experience per gradient step for updates to fully utilize each entry in the buffer.
Performance comparison on ImageNet dataset. To evaluate our method on more advanced and large-scale datasets, we conduct performance comparison on ImageNet (Deng et al. 2009) dataset. As shown in Figure (6, PSP achieves comparable accuracy to PCB in the first two rounds but gradually outperforms it in later rounds, ultimately reaching the highest performance of 68.41% % on the final accuracy on the large-scale ImageNet dataset. These results provide powerful evidence that PSP effectively reduces the reliance large-scale labeled datasets in prompt learning.
16

---

Published as a conference paper at ICLR 2026
65.48
65.66
64.78
Accuracy
14
63.95
Figure 5: Buffer threshold analysis on DTD. We report the final accuracy for different buffer thresholds.
Accoracy P%o
Bandom (Entooy))
PUB (Coreset)
70
AS
PCB (HADGE)
20
AE
PSP
Noze
Round
Mound
(a) Learning Curve
(b) Description Augmentations
Figure 6: (a) Learning curve ImageNet dataset. Classification accuracy the ImageNet dataset using the ViT-B/32 image encoder at each round. (b) Ablation study of various description augmentations (i.e., AS, AE, and None) DTD. We report learning curves evaluate the effectiveness of various description augmentations.
INCHUASIO-OO
MEETA ENC Fdbe
ESC-E
PENTAO
Roued
Brand
Reand
(8) Prompt Size
(b) CSC
(c) Prompt Position
Figure 7: Learnable prompt analysis on DTD dataset. We report accuracy for different prompt
sizes, whether class-specific tokens are used, and varying prompt positions each round.
17

---

Published as a conference paper at ICLR 2026
Table 6: Analysis of the versatility of PSP for different Vision-Language Models.
Method
DTD
Oxford Pets
EuroSAT
Aireraft
Average Acc
PCB (SigLIP)
55.73
47.12
PSP (PigLIP)
59.75
62.99
86.18
28.65
59.39
Table 7: Analysis of the behavior of the learned sampling policy.
Round
PSP vs CoreSet
PSP vs Entropy
PSP BADGE
0,0113
0.0113
0.0113
0,1107
0.0959
0.0923
0,0920
0.0920
0.0958
4 0,1103
0.0608
0,0837
0,1094
0.1283
0,1283
0.1301
0.0892
0,0706
0,0784
0.1007
0,1045
8
0.0833
0.1023
0.1023
Influence of description augmentation. To find the optimal description augmentation suitable for PSP, we analyze the impact different description augmentations (i.e., AS, AE, and None) on DTD, and the results are presented in Figure @p. It is confirmed that PSP utilizing AS as the description augmentation significantly outperforms the cases that use AE and no augmentation. Hence, we choose AS as the default description augmentation.
Influence of increasing query size. In Table it is worth mentioning that the pre-trained zero-shot CLIP model outperforms PSP on Oxford Pets. To further analyze this anomalous phenomenon, we increase the query size n, from K to 2K in each round and report the results of PSP on Oxford Pets in Table I1 We observe that the "Final Acc" metric increases the size of the query set grows across all backbones, and PSP surpasses the performance of the pre-trained zero-shot CLIP model on various backbones when the query size reaches n, 2K.
Learnable prompt analysis. To further analyze which factors of the prompt affect the performance of PSP, conduct experiments about several variables according to the learnable prompt, e.g. the size of learnable tokens (i.e., prompt size) M, whether class-wise different tokens are allowed (CSC True or False), and the position of class token (Front, Middle, and End). For prompt size M, the results are shown in Figure 7p. It is confirmed that PSP with a larger M alternates in outperforming PSP with smaller M until the third round, after which it consistently maintains its performance lead through the final round. PSP with a larger M achieves a 2.13% improvement on the "Final Acc" metric compared PSP with smaller M. demonstrating that PSP is sensitive the prompt size, with larger M values yielding significantly better performance.
For class-wise different tokens, we analyze the performance gap between using the shared context vectors for all classes (i.e., CSC False) and using different context vectors for each class (i.e., CSC True), and the results shown Figure ?b. The results show that the initial accuracy when CSC True is lower than when CSC = False, but it eventually surpasses that of CSC = False in the second round, and then maintains its leading performance. It can be concluded that using class wise different tokens offers slight benefit to PSP. Additionally, during the first round, CSC True is more susceptible to overfitting compared to CSC = False.
Ultimately, conduct experiments investigate the impact of the class token's position (i.e., Front (Frt), Middle (Mid), and End) on active prompt learning for PSP, and the results reported in Figure 77p. The results reveal that the learning curves of PSP alternate in leading performance across different class token positions. PSP with the End position of class token slightly surpasses the others on the "Final Acc" metric. Therefore, we choose place the class token after the context vectors for all experiments. It is demonstrated that the position of the class token has minimal impact on PSP in active prompt learning.
A.3 PRELIMINARY
Active Learning identifies criteria for selecting the most informative samples under a limited labeling
budget. Active learning methods are primarily applied in three distinct scenarios: membership query
synthesis (Mahapatra et al., 2018 Mayer & Timofte 2020), stream- stream-based (NarT et al. [0166] Fang
et al., ( & Finn 2017), ) and recently most mainstream pool-based Vijayanarasimhan
& Grauman [001]] Gosselin Cord, 2008 Yang et al.] 2015)) Kapoor et al.) 2010 Bang et al.
2024) setting. In this work, we follow the pool -boe- setting and define an unlabeled data pool from
18

---

Published as a conference paper at ICLR 2026
Table 8: Ablation study with classical prompt learning methods like CoOp and CoCoOp. We report the final accuracy across seven datasets for a comprehensive comparison with CoOp and CoCoOp.
Method
DTD
Oxford Pets
EuroSAT
Flowers102
Caltech101
Stanford Cars
Aireraft
Average Ace
CoOp Zhou et 20220
58.65
87.71
68.73
88, 27
90.14
65.25
24.99
69.11
CoCooO Zhoa o an 20226
57,80
72.81
21,93
PSP
65.66
86.57
85.43
96.35
93.87
73.84
36.42
76.87
Table 9: Analysis of efficiency on DTD. All models are trained on single RTX 3090 GPU with a batch size of 32.
Method
2
5
6
Training Time (b)
F Final Acc (%)
Random
305
380
456
475
540
586
645
670
1.13
58.77
PCB(Entropy)
300
545
635
1.06
PCB(Coresee)
327
415
486
494
561
611
662
676
1.18
56, 38
PCB(BADGE)
301
944
906
919
971
998
980
991
1.95
62.33
725
869
which a subset of samples is actively selected for annotation. Based on the criteria, active learning
methods can be categorized into three main approaches: Uncertainty-bun -based sampling (Gal al.
2017: Wang et al. (2019), Diversity-based sampling (Hacohen et al. 2022 (Shu et al. 2020). Hybrid sampling (Ash et al. 20200 Parvanch et al., 2022, Caramalau et al. 2021 D. and RL-based sampling (Ash et al. 2020 Kirsch et al. 2019). Uncertainty-basee sampling, a straightforward and effective strategy, focuses on selecting samples that the model struggles to learn, employing techniques such as
2008). and Least Monte-Carde Dropout (Gal et al. 2017; Kirsch et al. 2019), Entropy (Holub et al.
2008 for object recognition, which selects the samples with the highest entropy for annotation. Diversity-based sampling focuses on selecting samples that represent the full data distribution to ensure diversity in the labeled data, including clustering (Hu al., 202I) and Coreset (Sener & Savarese 2018). Sener
Confident (Lewis& & Catlett 1994). Holub et al. proposed Entropy (Holub et al.
et al. introduced the elegant and mathematically rigorous Coreset (Sener & Savarese 2018), which
provides an approximate upper bound on the loss for feature space coverage- based active learning algorithms. Hybrid sampling (Ash et al., 2020 Parvaneh et al., (022) takes into account both diversity and uncertainty, aiming to mitigate the issue of redundancy in Uncertainty-basee sampling and the limitations of Diversity-based sampling, where basic feature coverage strategies may fall short in assessing the model's confidence its predictions. ALFA-Mix (Parvaneh et al., 2022) utilizes unlabeled data to support active learning by interpolating between the representations of labeled and unlabeled instances and identifying features the model fails to recognize through inconsistencies in predicted labels. However, Hybrid sampling relies fixed rules balance diversity and uncertainty, limiting its adaptability across tasks. RL-baaedd sampling formulates a sample selection policy, where Reinforcement Learning (RL) is applied learn policy that maximizes cumulative reward by selecting samples. Woodward et al. developed AOL (Woodward & Finn 2017) that combines meta-learning and reinforcement learning for one-shot classification tasks. Liu al. introduced DRAL (Liu et al. 2019) to guide an agent in acquiring pairwise annotated data. Notably, PAL (Fang al.) 2017) p builds a deep Q-network as an adaptive policy for sample selection. Therefore, we believe that RL-based methods have the potential to incorporate prompts for guiding sample selection. However, AOL (Woodward & Finn, 2017) and PAL (Fang et al., (017) model the decision of whether to annotate streaming unlabeled sample as a binary classification problem, while MedSelect (Vrabac) al. b022) and DARL (Liu et al. 2019) rely on pairwise data, making them unsuitable for direct application in Active Prompt Learning (APL). Therefore, we introduce Soft Actor-Crite (SAC) (Haarnoja et al.; 2018), a representative reinforcement learning algorithm known for its robustness to hyperparameters and strong performance in continuous action spaces. By designing a customized real-pseudo hybrid reward and vectorized critics, SAC can be seamlessly integrated into APL..
A.4 DESCRIPTION AUGMENTATION
Here, the new text prompt is converted below:
Pia [c]]/c/.... ... [c]ar [cls:] [which]
(14)
where di denotes the k-thh description for class i, A
k== represents e; descriptions for class 1d
i. Given the new text prompt, two possible prediction probabilities after description augmentation are expressed below:
19

---

Published as a conference paper at ICLR 2026
Table 10: Final accuracy of different image encoder architectures, including ResNet-500101 and ViT-B/16.
Mothed
DTD
Oxford Pets
FaroSAT
Flomers102
Caltecht01
Stanford Cars
Aireralt
Average Ace (*)
(Zero Shot)
$2.1
19.3
Random
56.62200.7
74.55.0.50
29.10:2.31
84:11::0775
OCIALL (Caumalou 10217
$5.26:00.1
78.2421.19
92:9210..0
85.00220.44
64.3129.76
28.23:40.59
ALEA Mix Farrane
59:02.0
52.37-0.17
01.0220..66
72.58
RNS0
13.74-1.3
83-36-0.35
70.00-4.0
25.38-0..6
[Sange
51.6350.80 30
71.15
77.74-2.13
5867:0:33
25.33:00.3
25
T3.70:0.22
17.74:3.33
96.99:0.55
57.63:0.55
25.17:0.37
65.09
53.15:81.3
75.5321.64
19.79:1.06
19.50-11.39
NT.15
6061110.54
25,58:10,10
BADOR (iAsh 10:30
$8.35
75.06:10.50
00:94:0.55
95. 56.55611..44
89 0266711330
63961:0.5
12:112:1100
70.24
PCR (1anggeeig
57.41+0.17
76 S1+1.K3
60.06-0.97
95 64.666--.24
88.050.21
65-18-077
70.25 10.33+0.35
70
PCB (aP9
90,20+1.25
76. 77-066665
81 0..0000.00
95. 72:0.31
80 10.57+0.10
62622-176
78.85+1.00 5.00
70.67
PCB (AS)
96:44=1.10
50,09+0.85
B1 10:0000
86.18+0.07
98.6:00.34
66:20:0669
20.61
71.94
PSP
62:71:4.33
85.45:0.91
67.74:00.24
95 75.57:0.77
90.70:0:45
67.350411
32.37:90.63
74.55
Fully Labeled Data
71.6
$8.0
93.6
97.6
92.8
76.8
42.6
$0.71
CEPI (Zem-Shott
43.9
k6.2
18.1
6577
65.1
623
19.5
56.
Random
53.20+1.244 1.24
7908+1.50
72.31-4.11
92 8744444
87 75
7002-0366
32.76 32.76+0.29
71 11
GCNAL (Canamaion et POOT
57.0010.54
82.0420.49
01.68
91 13.00.111
00551021
68 94
31.8210337
71.87
ALFA Mi (err01nn m0 201e
61.33.0033
85 45.4444.8
03.91.0.49
96 $2.0..44
91 22
75 00000000
31.5410.45
75.09
Irmoopy: 18500 xp 10:0
57.17
74.88
1009 16
27 1730110:55
70779
59.81 01 30
50. 14+1
79.91 2.00
26
91 K2
00870045
18 11 10.35
72.25
PCB (011)
50.81 +1 54
62.65+0.99
61:33- 1:36
06.4240 50
91. 16+0.0
1014+0 56
27 27.56+1.4 1.69
72.92
RN101
Cinasat Sange & Sarrece 2018
PCB (AS)
60.70+1.00 09
53.23 76
74.02+1.8 81
63.54+1.00 00
12.43-1.35
66.62-0.54
K7 00000002
RT 23+1 15
91. 000000200
75622-967
26. 16.37:40.42
+0.88
65.74
74.06
- PCB (bang
54.75::2.30 53
T6.43:1.61 61
75.39-1994
01 00.58
80 56:10.25
66.97:9775
27.28-10.38 33
68.75
PCB (000
56.38.31.55 55
77.11 86
76.99:0.65
80 0000000000
65. 65:3022622
27.72::0.39 0.39
69.30
PCB (AS:
57.31.22.00 m
$1.14230.24
78.49:199
96 80110.38
90X 1:0000
69.11:0.75
29.31 210.78
70.90
BADGELAShhes etal RDFM
19.92211.25
80.77:1.31
78:23:2:22
31.352
714
32.56.10.64
72.95
PCB (1310 2003
00.20::1.80 89
80,9110-42
19:50-1133
95 95.0511098
91 14-79419.444
71 11.3800.99
32,62.1.448
73.07
PCB 2007
60.59.0.84 84
K3,02200889 89
81 01:20:0000
21 11442:077
32.76.10:00
36.35
(AS)
62177:1.04
63,48-2211
81:16-1:50
964746618
18
018756188
39
02.84:0.85
75.43
PSP
63.95:8774
97.43:0.11
87.19-0.19
10
92
75-28-0.44
37.96
Fully Lubeled
93.7
$2.91
(Zemm-Sbat)
65.
65.0
626621.81
343661.54
81.14-1.8
94 06
73
58.88+0.25
(Cauaniie 2013
62.586006
52.98-0-7
95 37
70
732566021
ALFA-Ma Pereannn
66.386030 26
04.38-0.36
1540.18
34
35.55
75.95
67.06-0.00
56+0
15155-055
VIT-6166
Sever
63.81
56.0700.00
58.0350.00
62.1751.82
02.12:14 45
0016610.44
0555
23.25:0..5
70.94
59.07
63.0921.19
90.25-3.112
90
11.27::022
13.30
60,5420.86
$4.5220.23
$4.04-2.92
92. 12.15 5.555
7011011
33.2 33.36:10,03
24.24
61.9811.04
M6.77.0.19
23.85
92,97 05.4410082
72.611055
15.24.10.49
BADGILLAhh
20:20
62,8412.17
A5.5411.30
12.22-1.94
M3.77.10.51 17.9710441
16.5510077
19.64.00114
76.95
PCB [157g
64.8111.5
20.2210.71
31.53
32.10.1
93.75 28
10.20.10.30
77.32
PCB EXTT
1: 29
67.23
2..04-2.2
96, 21
94.51
19.51:00.21
77
PCB (AS)
64 95
10++1 49
63.85-22 2.45
98. 19++0.17
05.12:0.26
78 19-00-0000
00.56-10.51
78.42
PSP
67.32-
89.67-0.78
89.22 -0.16
96 95-00444
05. 10HO 90
82.06-0.37
43.50-0055
NI.54
Tolly Labelod Data
77.7
227
95.1
99.0
95.3
85.3
33.5
$5.33
Table 11: Ablation study on Oxford Pets, which evaluates the impact of increasing query size 7l.
Method
n,
RN50
RN101
ViT-B/32
VIT-1/16
CLIP EZero-Show
0
85.4
86.2
87.0
58.9
Random
37
74.65
79.08
78.30
84,36
PSP
37
85.45
87.43
86.57
89.67
PSP
74
86.40
89.34
88.72
91.44
Full Labeled Data
88.0
91.1
89.3
92.7
(1) Average Similarity (AS):
1
6
ply = =1)))
ply = d2)
(15)
ply = 2 dd) exp(cos( (i..Jiis)// stws///w)
(16)
E= exp(cos (Vv fi,ns//w)
where f7 = F+ denotes text feature corresponding to description d/_ K represents the number of classes downstream task and indicates temperature scaling parameter.
(2) Average Embedding (AE):
ply = i12) = exp(cos( fi.fi.../)/ ΣK exp(cos( fū, f/w)/w)
(17)
fh. fixs
(18)
20

---

Published as a conference paper at ICLR 2026
The main difference between the two probability scores is that AS computes the cosine similarity for each text feature before averaging, whereas AE averages the text features first and then computes the similarity.
A.5 SOFT DYNAMIC TIME WARPING
Given two vector sequences of unequal lengths: Qe = 10 Q0
10m0) and Q =
(Q ",,..,,(((2..)) Compute pairwise distance matrix D R" xmita:
D., = -
(19)
Accumulated cost matrix R. The accumulated cost matrix is initialized with Ro,o, followed by dynamic programming computation incorporating the soft minimum as described below.
Ri = D.., - min [R:-1,j+ Ha.s Hi H4-15-1)
(20)
min(a, b, b) = - log (e (6-0/7 + b/7 +e c
Path extraction. The alignment path is derived by identifying the minimum cumulative cost path within the accumulated cost matrix.
= ((i, J1), j2). (tr,jr)}
(21)
where the integer 1, E [1, n²]] indicates that the first element in the aligned Q-value Qa corresponds to an element in Qo, and the same is true for j. T' is the length of the alignment. Q₂ and Q' are obtained by expanding and repeating elements according to x.
A.6 LIMITATION AND FUTURE WORK
However, PSP has limitations, particularly in its reliance replay buffer for updating the sampling policy. If the data is highly sensitive, the security of the replay buffer becomes a critical issue, as any potential leakage could have serious consequences. On the other hand, PSP has the potential for application in more complex downstream tasks. PSP can help save resources in tasks with high annotation costs, such as Human-Object Interaction detection and semantic segmentation, through further improvements. Therefore, this will be a focus of my future work.
A.7 BROADER IMPACTS
Positive societal impacts. PSP adaptively learns a sampling policy in an End-to-End manner to select the most informative samples under a limited annotation budget, thereby reducing the reliance on large-scale labeled datasets for downstream tasks. PSP achieves comparable performance to prompt learning with a fully labeled dataset, without relying selection. This highlights its strong potential for extension to other computer vision tasks, such as object detection and semantic segmentation, where it can substantially reduce the annotation burden in these traditionally resource-intenee tasks.
Nomain-specifet knowledge for sample
Negative societal impacts. Despite its benefits, the proposed PSP further alleviates the reliance on manual labeling by more efficiently identifying informative samples. However, this advancement may bring about unforeseen socioeconomic effects. Specifically, as reliance on human annotators decreases, especially for repetitive or now-complexing labeling tasks, there is potential risk of reduced employment opportumities the data annotation industry. This shift could disproportionately affect low-skilled workers whose livelihoods depend on such roles, potentially leading to job displacement and increased economic vulnerability in regions where annotation work a key source of income.
21


Reviews:
[{"review_id": "1byyMjLSoe", "reviewer": "Reviewer_7H9h", "summary": "This paper targets the problem of sample selection in Active Prompt Learning (APL) for Vision-Language Models (VLMs). The authors argue that existing APL methods use generic active learning heuristics that do not explicitly use information from the learned prompt to guide sample selection and also ignore valuable information in the large pool of unselected data. To address this, the paper proposes PSP (Prompt-Guided Self-Training Sampling Policy), a framework that learns a sampling policy using reinforcement learning (RL). The method consists of two main parts: (1) a Vectorized Soft Actor-Critic Sampling Policy (VSSP), where an RL agent learns to select informative samples based on a state representation of gradient embeddings and a custom \"real-pseudo hybrid reward\" that reflects the prompt's performance; and (2) an Uncertainty Augmented Self-Training (UST) mechanism that generates pseudo-labels for unselected data to augment the training set. Experiments on seven image classification datasets show that PSP outperforms several active learning baselines, including variants of the state-of-the-art PCB method.", "questions": "1. The reward function is constructed as a product of the log-probability of the sampling scheme and the classification error. This objective is quite indirect. Have you experimented with a more direct reward signal, such as the negative validation accuracy on a small holdout set, or simply using $- (\\overline{r}_{s} + \\beta\\overline{r}_{p})$ as the reward? \n2. The VSSP agent is trained on a very small number of total experiences (a maximum of 8, one per round). Given the high sample complexity and potential instability of RL algorithms, how do you ensure that the learned policy is stable and generalizes beyond the few trajectories seen during training?\n3. Could you provide a more detailed justification for using Soft-DTW to align the Q-value vectors between states $s_t$ and $s_{t+1}$?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Novel Formulation for Sample Selection: The core idea of framing active sample selection for prompt tuning as a policy learning problem is novel and conceptually appealing.\n2. Explicit Integration of Prompt Information: A key strength is the explicit use of the learned prompt's performance to guide the sampling policy. \n3. Leveraging the Full Unlabeled Dataset: The UST mechanism is a sensible addition that attempts to extract more value from the entire unlabeled dataset, rather than only focusing on the small subset of samples selected for annotation\n4. Comprehensive Experimental Evaluation: The paper presents an extensive empirical study on seven datasets", "weaknesses": "1. Extreme System Complexity for Modest Performance Gains: The primary weakness of this work is the immense complexity of the proposed VSSP framework relative to the performance improvement it delivers. \n2. Convoluted and Indirect Reward Function: The reward function in Equation 4, defined as $r(s_{t},a_{t})=log(p_{m}(g))*(\\overline{r}_{s}+\\beta\\overline{r}_{p})$, is indirect and unintuitive.\n3. Potential Instability and High Sample Complexity of RL: Actor-critic methods are known to be sensitive to hyperparameters and can be unstable to train, especially with limited data.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2cY4zwPvcn", "reviewer": "Reviewer_gdcF", "summary": "This paper proposes a new method called PSP for the field of Active Prompt Learning (APL). Its main goal is to efficiently train prompts for vision-language models (such as CLIP) under a limited annotation budget by intelligently selecting the most valuable data samples for labeling, thereby adapting the prompts to downstream tasks.", "questions": "[1]  In such “low-data” or “few-episode” reinforcement learning scenarios, how does your method ensure that the Actor policy effectively converges rather than overfitting to the limited early experiences? Does your design of the Vectorized Critic play a key role here, as it can provide richer learning signals for multiple samples from a single experience?\n[2] Have you analyzed how the learned policy behaves compared to traditional active learning heuristics, such as uncertainty sampling, diversity sampling, or hybrid methods like BADGE? For instance, does the model prefer high-uncertainty samples at certain stages while prioritizing diversity at others? Can we reverse-engineer the learned “optimal policy” to extract interpretable new sampling insights that go beyond existing heuristic rules?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "[1] The experiments in the paper are comprehensive.\n[2] The challenging decision of “which samples to select for labeling” is modeled as a reinforcement learning task. This allows the sampling policy (Actor) to act like an agent that continuously “learns” and “evolves” through interaction with the environment (model performance), ultimately mastering the most efficient sampling strategy. This approach is more flexible and intelligent than traditional methods that rely on fixed rules, such as uncertainty or diversity.\n[3] The paper combines active learning techniques for vision-language models with reinforcement learning, effectively improving the efficiency of sample learning.", "weaknesses": "[1] The algorithm incurs excessive computational overhead.\n[2] The system is relatively complex, and parameter tuning may be challenging.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iC0pQ8BVbR", "reviewer": "Reviewer_aJjs", "summary": "This paper proposes PSP, an active prompt learning framework for CLIP to reduce annotation costs. PSP uses the prompt to guide sample selection and considers selected and unselected data. Moreover, PSP introduces two components: VSSP, an RL policy with a prompt-derived reward function to select informative samples, and UST, a self-training mechanism that generates and filters reliable pseudo-labels from unselected data.", "questions": "See weaknesses.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The conceptual integration of prompt learning with RL is a strength.\n\n2. The paper's claims are substantiated by extensive experiments across various aspects.", "weaknesses": "1.The paper suffers from several clarity issues that hinder understanding. Several symbols are used before definition. For example, $K$ (presumably the number of classes) is used on 213 without definition. $\\hat{y}_i^k$ (Eq. 5) is not explicitly defined, though it seems to be the teacher CLIP's prediction if $k=u$. This pattern of undefined symbols increases reading difficulty.\n\n2.The paper defaults to SAC (a 2018 algorithm) without justifying this choice over simpler selection methods (e.g., a learned Mixture-of-Experts) or newer, more advanced RL algorithms. The specific advantages of SAC for this particular selection problem are not discussed.\n\n3.The equation (2) seems redundant with the text on 215(e.g.,  $f_{V}^{t,i}=F_{V}^{t}(x_{i}^{u})$  ).\n\n4.The principle behind (2),(4),(5) is not explained.\n\n5.Table 8: Comparing PSP to CoOp and CoCoOp is problematic. (a) These are outdated baselines; newer methods like [1]MaPLe or [2]PromptSRC can be used. (b) The comparison is fundamentally unfair, as PSP uses both labeled and unlabeled data, while CoOp/CoCoOp only \"using the same number of real-labeled training samples\". A fair comparison would require augmenting CoOp/CoCoOp with a pseudo-labeling strategy or a direct comparison against unsupervised methods, like [3]PromptKD.\n\n6.Comparing a task-adapted PSP (fine-tuned on downstream data) to zero-shot VLMs is an apples-to-oranges comparison. This does not prove PSP is better. A valid comparison would be to apply the PSP to these other VLMs to demonstrate its adaptability, as was done for SigLIP (Table 7).\n\n7.The dataset (presumably DTD) in Table 9 is not explicitly named. The numbers under the rounds (e.g., \"305, 380...\") are undefined. What do they represent? I am curious about the additional training time (or computational overhead) incurred by PSP compared to methods like CoOp.\n\n8.The paper frequently refers to \"student CLIP\" and \"teacher CLIP\". The CLIP model's parameters are frozen; only the prompts are being trained. The correct terms may be \"student prompt\" and \"teacher prompt\" to reflect what is actually being updated.\n\n[1]MaPLe: Multi-modal Prompt Learning\n\n[2]Self-regulating Prompts: Foundational Model Adaptation without Forgetting\n\n[3]PromptKD: Unsupervised Prompt Distillation for Vision-Language Models", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FSbJk96lZt", "reviewer": "Reviewer_dwtc", "summary": "The paper proposes PSP, a Prompt-Guided Self-Training Sampling Policy for Active Prompt Learning with CLIP models. The method integrates a Vectorized Soft Actor-Critic Sampling Policy (VSSP) and an Uncertainty-Augmented Self-Training (UST) mechanism. VSSP introduces a real-pseudo hybrid reward and vectorized critics to guide sample selection using prompt information, while UST generates reliable pseudo-labeled data based on uncertainty and confidence. Extensive experiments on seven benchmark datasets show consistent gains over PCB and other active learning baselines.", "questions": "See Weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "1. The proposed PSP improves over strong PCB variants with clear gains in accuracy on multiple datasets, and the evaluation on on seven datasets is comprehensive.\n\n2.  The paper provides detailed mathematical formulation for VSSP and UST mechanisms.", "weaknesses": "I am not an expert in active learning, so I might defer my final score after considering other reviewers’ comments. Nonetheless, I identify two main weaknesses:\n\n1.\tWriting structure and overly complex presentation.\nThe Introduction and Method sections could be reorganized to better highlight the motivation and intuition before introducing dense equations. Currently, the paper is overloaded with mathematical notations, which obscure the core ideas. The motivation for the proposed approach is not convincing, and the discussion on the limitations of prior work lacks depth.\n\n\n2.\tPractical clarity.\nThe method appears as an overly complicated mixture of existing techniques, which raises doubts about its real-world applicability and scalability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)