VaMP: Variational Multi-Modal Prompt Learning for Vision-Language Models
Authors: Silin Cheng
Kai Han
Venue: NeurIPS 2025 poster ()
Keywords: Prompt Learning; Vision-Language Models; Multi-Modal Representation Learning
Primary area: applications

Abstract: Vision-language models (VLMs), such as CLIP, have shown strong generalization under zero-shot settings, yet adapting them to downstream tasks with limited supervision remains a significant challenge. Existing multi-modal prompt learning methods typically rely on fixed, shared prompts and deterministic parameters, which limits their ability to capture instance-level variation or model uncertainty across diverse tasks and domains. To tackle this issue, we propose a novel Variational Multi-Modal Prompt Learning (VaMP) framework that enables sample-specific, uncertainty-aware prompt tuning in multi-modal representation learning. VaMP generates instance-conditioned prompts by sampling from a learned posterior distribution, allowing the model to personalize its behavior based on input content.
To further enhance the integration of local and global semantics, we introduce a class-aware prior derived from the instance representation and class prototype. Building upon these, we formulate prompt tuning as variational inference over latent prompt representations and train the entire framework end-to-end through reparameterized sampling. Experiments on few-shot and domain generalization benchmarks show that VaMP achieves state-of-the-art performance, highlighting the benefits of modeling both uncertainty and task structure in our method. Project page: https://visual-ai.github.io/vamp

VaMP: Variational Multi-Modal Prompt Learning for Vision-Language Models
Silin Cheng Kai Han"
Visual AI Lab, The University of Hong Kong
hms.chen@ecommet.. hku.
kaihanx@hku,
Abstract
Vision- Vision-language models (VLMs), such as CLIP, have shown strong generalization under zero-shot settings, yet adapting them to downstream tasks with limited super- vision remains a significant challenge. Existing multi-modal prompt learning meth- ods typically rely fixed, shared prompts and deterministic parameters, which limits their ability to capture instance-level variation or model uncertainty across diverse tasks and domains. To tackle this issue, we propose novel Variational Multi-Modal Prompt Learning (VaMP) framework that enables sample-specific, uncertainty-awar prompt tuning in multi-modal representation learning. VaMP generates instance-condition conannconond prompts by sampling from a learned posterior dis- tribution, allowing the model personalize its behavior based input content. To further enhance the integration of local and global semantics, we introduce class-aware prior derived from the instance representation and class prototype. Building upon these, we formulate prompt tuning as variational inference over latent prompt representations and train the entire framework end-to-end through reparameterized sampling. Experiments on few-shot and domain generalization benchmarks show that VaMP achieves state-of-the-art performance, highlighting the benefits of modeling both uncertainty and task structure in our method. Project page: https: /ttppss//vvvuaaleeeeeeee github. io/vamp
Introduction
Vision-Language Models (VLMs), such as CLIP (1], have achieved impressive performance across wide range of visual recognition tasks through multi-modal representation learning. Their ability to align images and texts in shared embedding space enables strong zero-shot transfer. However, their large-scale parameters and the scarcity of training data, particularly in few-shot settings, make full model fine-tuning computationally expensive and prone to overfitting on downstream tasks.
To address this, prompt learning has emerged number of learnable tokens are prepended to the input to steer the frozen model toward task-specific behavior [2, 3, 4, 5, 6, 7, 8]. While effective, existing multi-modal prompt tuning methods typically rely on fixed, shared prompts that are applied uniformly across all samples. Such methods are inher- ently deterministic and lack the flexibility adapt instance-level variations model uncertainty, limiting their generalization to unseen tasks and domains [9, 10].
efficieet alternative, where small
While recent work has explored uncertainty-aware prompt tuning. most existing approaches remain limited in scope. Bayesian Prompt Learning introduces uncertainty modeling in text-only prompts, and Ahy-Shift Prompting leverages variational inference enhance robustness across distribution shifts. However, these methods suffer from key limitations. First, by restricting variational modeling to leput-level prompts with global latent variables, they fail to capture hierarchical feature interactions
"ooresponding Author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

or tine-grineed token-level semantic variations. Second, their text-only prompt tuning overlooks valuable visual information that could enhance cross-modal alignment. Finally, the standard Gaussian prior, shared across all classes, fails to capture inter-class variations, resulting in less discriminative prompt distributions. Consequently, these models are inadequate for capturing fine- iieegrained, input- specific variations, particularly in vision-language tasks where precise alignment is critical.
To overcome these limitations, we introduce novel Variational Multi-Modal Prompt Learning (VaMP) framework that enables sample-specific, uncertainty-awart prompt tuning for vision-language models. We make three key contributions: First, we introduce token-wise variational modeling across multiple intermediate network layers. This approach treats individual prompt tokens as latent variables, enabling the model to capture fine- -eneggriiedd semantic relationships multiple abstraction levels and improve generalization in low-data and out- distrrbbuii scenarios. Second, our multi- modal design incorporates both visual and textual signals when inferring posterior distributions, creating more aligned cross-modal representations. Third, by employing class-aware priors instead of standard Gaussian distributions, VaMP generates more discriminative prompts that better capture category-specite features and decision boundaries.
We evaluate VaMP on three challenging adaptation settings: base-to-new generalization, domain generalization and cross-dataset generalization. Our method consistently outperforms strong multi- modal prompt baselines while maintaining high parameter efficiency. Ablation studies further confirm the effectiveness of each component, including the variational modeling and task-aware prior.
2 Related Work
Pre-trained Vision-Language Models. Pre-trained vision-language models (VLMs) [1, 11, 12, 13] have gained significant attention for their strong performance across diverse vision-language tasks. These models typically follow four training paradigms: 1) masked language modeling [14, 15]: 2) masked region prediction [16, 17]:
3) image-text matching [16, 14]; and 4) contrastive learning [1, 11, 18, 19]. While VLMs provide robust, generalized representations, adapting them to downstream tasks remains challenging. Recent studies show that tailored approaches significantly improve performance in specific domains, such as few-shot image recognition [20, 21]. object detection [22, 23, 24, 25, 26, 27], semantic segmentation [28, 29, 30, 31, 32] and visual grounding [33, 34]. In this work, we focus on adapting vision- langualan models for few-shot and zero-shot visual recognition tasks.
Prompt Tuning. Instructions provided to language models as text prompts enable task-specific understanding and performance in VLMs. These prompts, either manually designed or automatically optimized through "Prompt Learning" " (originally from NLP [35, 36, 37J). have been adapted for computer vision in three primary forms: textual prompt leanring [2, 3, 38, 39, 40, 41, 42, 43, 44, 45, 46] that fine-tune CLIP's [1] by optimizing continuous prompt vectors in its language branch; visual prompt learning [4, 47, 48, 49, 50] that optimize task-specific learnable inputs in the visual input space while keeping pre-trained backbones frozen; and multi-modal prompt learning [5, 1,,51, 53, 54, 55, 6] that enhance alignment by applying prompts to both vision and language branches. Our work advances this research by introducing a variational framework for multi- modai prompt tuning, enabling sample- specific, uncertainty-awar prompt tuning with structured guidance from both visual inputs and class-level semantics.
Variational Inference. Variational inference has been widely applied to computer vision tasks, such as image generation [56, 57, 58, 59], action recognition [60], instance segmentation [61], anomaly detection [62], depth estimation [63], few-shot learning [64, 65, 66, 67], and domain generalization [68, 69]. Recently, variational inference has been applied to prompt learning to mitigate overfitting in low-shot settings and improve generalization. For example, Bayesian Prompt Learning [9] captures uncertainty by sampling prompts from learned distributions, while Prompting [10] uses a hierarchical probabilistic framework to model distribution shifts and generate adaptive prompts without test-time optimization. However, both methods are limited to the text modality and rely on globally shared prompts, which restrict their ability to capture fine-grained, input-specific variations. To overcome these limitations, we propose a probabilistic framework that combines multi-modal prompt tuning with variational modeling and sample-specifi adaptation.

---

3 Preliminary
3.1
Revisiting CLIP
Our work builds upon the pre-trained vision-language model, CLIP [1], which comprises both a text encoder and vision encoder. Following previous prompt-learning methods [2, 3, 5, 8], we adopt ViT-based CLIP model that encodes both images / E I x x3 and text descriptions.
Encoding Image. The image encoder V consists of transformer layers, denoted as {Vi) first divides the input image I into B non-overlapping patches. These patches are then projected into embeddings eo E RBxx, where d, represents the embedding dimension. Subsequently, patch embeddings, along with class token C₁, sequentially processed through the transformer blocks:
G++1l e;)) = , K-1
The final image representation x obtained by applying linear projection the last layer's class token, mapping it into the shared vision-language embedding space:
f = Fim (ck)
fa E Rdd..
Encoding Text. The text encoder converts tokenized words into embeddings w6 [wg: wō: -. w E IR Nxd, which are processed through K transformer layers:
[wa+1] = Lww,)) i= 0, 0,1........ , 1-
Similarly, the text representation t is obtained through a linear projection of the final embedding of the last token:
1= Fia LE Rd
(4)
Zero-shot Classification. For classification, hand-crafted text prompts (e.g., "a photo of <category>") with class labels E (1,2,... C} are used. The prediction ÿ for image is de- termined by the highest cosine similarity:
ÿ = arg max exp(sim( (g)/T)
(5)
Σ exp(sim(Sar) (s)))
where 7 is the temperature coefficient.
3.2 Multi-Modal Prompt Learning
Multi-modal prompt learning methods [5, 70, 8] extend text-only prompt learning approaches [2, 3] by jointly tuning the text and image prompts to achieve improved alignment for downstream tasks. These methods typically specify H consecutive transformer layers, beginning from the J-th layer, for prompt tuning, while leaving the remaining transformer layers fixed (where 2 and < K + 1). For example, MaPLe [5] focuses prompt tuning the shallow layers (J 0, H 9), whereas MMRL [8] applies prompt tuning to deeper layers (J = 5, H 7).
Deep Language Prompting. In the text branch, we introduce learnable prompts 2; E Mxd, consisting M tokens, into the i-th transformer layer for prompt tuning. For each layer from J to J J fl 1, these tokens are concatenated with the original token embeddings and fed into the i-th transformer layer to generate inputs for the next layer:
L, w L ([2,, wil),
(6)
while layers outside this range (from 0 to J 1 and from J + H to K 1) remain unchanged:
[w++1] = T.+11 ([w=])-
Finally, the text representation is derived via Eq. 4.
L. w++1] L,+1 1 (1=,, wil)
(8)
Meanwhile, layers outside this range (from to - and from J H K remain unchanged:
[wa+1]= = T++11 ([wd)).
9
The final text representation is obtained through Eq. 4.
3

---

(a) Class-Aware Prior Construction
(b) Variational Multi-Modal Prompt Adaptation (VMPA)
(x/
Cat
Image Encoder
Entrrres Kavunk
Lut
Pa
Bird
Ps. (Zi ov)
(c) Training pipeline
Eucoder
Fist
phoooo cal
Soooda Lagne
Thacode
Lee
Pioode Eayw St
Boode Layve V/
Push Patvaling
FizE
Kancdee
Image SauR-11
cx--1 6x
Prototype embodking
Instance embedding
Sampling
Utilizing CLIP's frozen image encoder to process training samples, generating offlie class prototypes for subsequent adaptation. (b) Variational Multi-Modal Prompt Adaptation (VMPA): Variational modeling mechanism where image- conditioned posterior (z x) and class prototype-based prior py (2. Gy) are aligned through KL divergence regularization of latent prompt distributions. (c) Training Pipeline: Full architecture of our proposed VaMP framework.
Figure 1: Overview of the VaMP framework. (a) Class-Award Prior Construction:
Deep Vision Prompting. In the visual branch, M learnable tokens ž, Mx inserted into the i-th transformer layer for prompt tuning. The exact computation these visual tokens depends on the specific multi-modal prompt learning method being used. For instance, in MaPLe [5], these visual tokens are generated from the language prompts via a linear transformation implemented with an MLP. In contrast, MMRL [8] obtains visual tokens from shared latent space-a of learnable tokens initialized by sampling from a Gaussian distribution and then uses separate linear projection functions (also implemented with MLPs) generate modality prompts.
For each layer from to J + H 1, these generated tokens are concatenated with the original patch token embeddings and fed into the i-th transformer layer to produce inputs for the next layer.
[G+11 6:41, ] V4+1 (Ic,, 2,1)-
(10)
For the remaining layers, the original operation without prompts is preserved:
[c++1 1+ = Vi+1 ex]).
(11)
The final image representation follows the same projection process outlined Eq. During inference, predictions follow standard classification procedures based on similarity.
4 Method
We propose VaMP-a variational multi-modal prompt learning framework that enables sample- specific, umcertainty-awart and structured tuning within vision-language models. Our method

---

consists of three key components: (i) sample-specific multi-modar prompr generation, where image- conditioned prompts are injected across multiple transformer layers; (ii) variational prompt adaptation for multi-modal representation learning. which models the text-side prompts as latent variables to capture instance-level uncertainty; and (iii) class-aware prior construction, which regularizes the latent space using semantic information from both the input instance and its class prototype. An overview of the full framework is illustrated Figure I, which highlights the generation of text prompts from image features, the variational posterior, and the class-conditioned prior used for regularization.
4.1 Sample-specific Multi-Modal Prompt Generation
In multi-modal prompt learning, prior work typically learns a fixed set of prompt tokens shared across all input samples [5, 8]. While effective, such fixed prompts cannot adapt to instance -speciffc variations, which are crucial for robust downstream performance under distribution shifts.
To overcome this limitation, we propose sample-specific prompi generation, where prompts in the text encoder are dynamically generated based on the input image. Unlike previous methods that insert fixed prompts single layer, generate and inject prompts across multiple transformer layers, enabling hierarchical and fine-grained modulation of the language stream.
Specifically, given an input image we first extract its global visual representation far using the frozen CLIP image encoder. To generate text-side prompts, we define a set of layer-specific prompt generators { 1 where each $ is lightweight MI.P that maps the image feature to a set of M prompt tokens for the i-th transformer layer:
= (f E R Mxd
i= i= J= H
(12)
where d is the token embedding dimension. Although all Φ, share the same architecture, they have independent parameters to allow for layer-specific adaptation.
The resulting prompts 2; are concatenated with the frozen text token embeddings W; at each layer i of the CLIP text encoder, as formalized in Eq. 8, following the multi-layer prompt insertion strategy used in MaPLe [5] and MMRL [8].
In parallel, we also adopt deep vision prompting in the CLIP image encoder, where set of learnable vision-side prompt tokens 2 E IR Mxd is inserted at each selected transformer layer i. These vision prompts are shared across samples and optimized independently from the input x. They are not generated dynamically. nor modeled as latent variables.
In summary, method introduces dynamic, sample-specific text prompts =; conditioned on image features and static, shared vision prompts 2, aligning both modalities through a structured and hierarchical prompting design. Only the text-side prompts 2, modeled probabilistically in variational framework.
4.2 Variational Multi-Modal Prompt Adaptation
While our sample-specific prompts z; increase flexibility, they remain deterministic and lack the ability to capture uncertainty-a key factor few-shot and distribution-siie settings. To address this, formulate prompt adaptation prompt tokens with latent prompt tokens learned via variational inference.
probabilistic latent variable model, replacing deterministic
For each input image æ and fixed text template t, define {z))/++111 where each 2; E xx is a latent variable representing the prompt tokens inserted at layer of the text encoder. To model the posterior distribution over these latent variables, we introduce a set of layer-specific MLPs {ox). --+ where each é, predicts the parameters of Gaussian distribution:
[n o.] F.).
(13)
where f is the frozen CLIP image embedding. Using these predicted parameters, the posterior distribution is formulated as a product of layer-wise Gaussians, conditioned solely on the image x:
qs (2s x)) N (Mi diag(o?))
(14)
5

---

Given label we aim to maximize the marginal likelihood:
logp(y 22,) log p(y x, t, (,,,2) p(z x) dz.
(15)
As the integral is intractable, we maximize the variational evidence lower bound (ELBO):
LELBO (=(x)) [logp(y x (,, KL (go(= x)) [[=)))
(16)
Here, p(2) represents the prior distribution over the latent prompts, which we initially set to standard Gaussian (O I). In implementation, we apply the reparameterization trick [71] to enable gradient-based optimization:
2, 14: + O C2 (O.
(17)
Each sampled z; replaces deterministic 2,, and is concatenated with the frozen tokens w; to form the layer input, as formalized in Eq. 8. This variational formulation introduces structured, uncertainty-aud distribution over prompts and allows the model to adaptively control prompt behavior per input.
4.3 Class-Aware Prior Construction
In variational inference, the prior distribution p(2) serves as a erucial regularizer for the learned posterior (s z). While standard choice p(2) = NN(,, I) provides a generic reference, it lacks semantic structure and offers task-specific guidance. To incorporate class-level semanties into the latent prompt space, we introduce class-aware prior that conditions on prototype representation for each class.
During training, we assume access to the class label for each sample. We compute a class prototype 0y by averaging the posterior means of training samples class yy:
Σ Sap IrEDy
(18)
Dy
where Dy is the set training instances labeled y,-
To model layer-wise latent prompts = = { we introduce a set of layer-specific prior
networks (bb))/// Each prior network maps the class prototype cy to the parameters of Gaussian prior at layer
=
(p oy) = = diag(();)))).
(19)
The resulting ELBO objective now sums across layers:
J+
CELBO
Σ
x, (,,) (oog
KL (go(z: xr) pp oy)))
(20)
This class-aware prior construction provides global semantic anchoring for each layer's prompt distribution. It encourages prompts from the same class to lie in nearby regions of the latent space, improving intra-class consistency and few-shot generalization. At test time, when is unavailable, we revert the standard prior p(z+) (0, for all layers.
4.4 Inference Procedure
During inference, our method follows single forward pass through the VaMP framework. Given an input image and fixed text template (e.g., "A photo of [CLASS]") "), first extract the frozen CLIP image feature J
For probabilistic inference, we leverage Monte Carlo sampling over the latent prompt distribution. Specifically, given input image x, we draw S samples { from the learned posterior (s(2, x) for each layer é:
[u4 o.) =d(fw), 24,8 pi + o ① 64,0, 6,0 10.
(21)

---

Each sampled latent prompt 2i.s is concatenated with the frozen text tokens wi at each transformer layer. Simultaneously, a set of shared vision-side prompts { is injected into the correspond- ing layers of the CLIP image encoder. The model then computes the image and text features using the frozen CLIP encoders with inserted prompts, and generates a prediction (s(y x. L, { - based on the similarity between the projected image and text representations.
Finally, the predictions are averaged across samples to form the final output:
ply x,()
Σ poly x. (z )
(22)
In our experiments, we use S = 10 samples for all evaluations. This inference-time ensembling preserves the expressiveness of the variational model while improving robustness and stability.
5
Experiments
5.1 Experiments Setup
We evaluate the performance VaMP under three different settings: base-to-novel generalization. cross-dataset evaluation, domain generalization, and few-shot learning, All conducted under 16-shot setting, where each category has only 16 training examples.
Base-to-Novel Generalization. In this setting, dataset classes are split into base and novel classes. The model is trained exclusively on base classes and tested on both base and novel classes, enabling an assessment its transfer learning performance on base classes and its ability to preserve the inherent generalization and zero-shot capabilities of pre-trained VLMSs for novel classes. This evaluation is conducted across 11 diverse classification datasets: ImageNet [72], Caltech101 [73], OxfordPets [74], StanfordCars [75], Flowers 102 [76], Food101 [77], FGVCAireraft [78], SUN397 [79], UCF101 [80], DTD [81] J. and EuroSAT [82].
Evaluation. This evaluation examines the model's ability to generalize to unseen datasets. Following CoCoOp [3], the model is trained on all 1000 ImageNet classes few-shot setting and directly tested, without further fine-tuning, the same datasets used for base-to-novel generalization, allowing us to assess cross-dataset transferability.
Datss-Dtaset
Domain Generalization. To evaluate the model's robustness to domain shifts and its generalization to out-of-distribuutinn data, we train it on ImageNet and test on four domain-varion datasets: ImageNetV2 [83], ImageNet-Sketch [84], ImageNet-A [85], and ImageNet-R [86]. each introducing distinct types of domain variation.
Implementation Details. We follow prior studies [5, 8] and adopt a 16-shot training setting for all experiments unless otherwise noted. We build on the ViT-B/I6 variant of CLIP [89] the visual backbone and apply multi-layer prompt tuning on the text and vision encoders starting from the J-th transformer layer. For MMRL-style settings, we set representation tokens per layer. For MaPLe-style configurations, we adopt J = 0. H = 9 with prompt length M for both modalities. All experiments are conducted on single NVIDIA V100 GPU.
5,
and insert M learnable
5.2 Main Results
Base-to-Novel Generalization. We evaluate VaMP against recent prompt tuning methods across 11 diverse datasets under the base-to-new generalization protocol. As shown in Table 1, VaMP achieves competitive performance on base classes while consistently outperforming all baselines novel classes. In particular, VaMP attains the highest average novel accuracy 78.67%, outperforming the best previous method (MMRL) by 1.51% % and demonstrating strong generalization to unseen categories. The advantage is especially pronounced on challenging datasets with large domain shifts. For example, DTD-aa dataset characterized by fine-grained textures rather than semantic vategories-VVMP achieves a novel accuracy of 75.50%, surpassing MMRL by 2.67%. These results highlight the strength of our structured, variational modeling in adapting to unfamiliar domains without sacrificing base class performance.
Domain Generalization.
We further evaluate VaMP in domain generalization setting,
7

---

Table 1: Comparison of VaMP with previous state-of-the-art methods on base-to-novel generalization across II datasets. Bold values indicate the best results. VaMP consistently enhances base class performance without compromising generalization.
Method
Average Novel
ImageNet Novel
Caltech101 Novel
OxfordPets Novel
Base
H
Base
H
Base
H
Base
CLI [i]
69.31
74,22
71.70
72.13
68.11
70.22
95.85
94.00
95.40
91.17
97.26
94.12
CoOp [2]
82.69
63.22
71.66
76.47
67.88
71.92
93.73
93.67
Co0pOp [1]
80.47
71.69
75.85
75.98
70:41
73.10
97.96
93.81
95.84
95.20
97.69
96.41
ProDA [36]
81.56
72.30
76.65 65
75.40
70.23
72.72
98.27
93.23
95.68
95,43
97.83
96.62
KgCoOp [4]]
80.73
73.60
75.83
69.96
72.78
96.03
MaPle [s]
82.28
75,
78.55 55
76.66
70.54
73.47
97.74
94, 56
96.02
95.43
97.76
96.58
PRC (6]
84.26
76.10
79.97
77.60
70.73
74.01
98.10
94.03
96.02
95.33
97.30
96.30
TCP [42]
84.13
75.36
79.51
77.27
69:87
73.38
98.25
94.67
96.42
94.67
97.22
95.92
MMA (70
83,20
76,80
79.85 19.87
77.31
71.00
74.02
98.40
94,00
96.15
95.40
98.07
96.72
2SFS [87]
85.55
75.48
80.20
77.71
70.99
74.20
95.71
94.43
96.52
95.32
97,82
96.55
SkipT [89]
85.04
77.53
81.11
77.73
70:40
73.89
98.50
96.89
95.70
97.87
96.77
MMRL [6]
85 65.68
77.16
81.20
77:90
71.30
74.45
98.97
94,50
68
95.90
97.60
96,74
VaMP
86,45
78.67
82.37
78.98
73.45
76.11
98.95
95.96
97.43
96,95
95,24
96.08
Method
StanfoniCars
Flowers102
Food101
PCVCCAineran
Base
Novel
Base
Novel
H
Base
Novel
Base
Novel
CLIP []]
63.37
74.89
68.65
72.08
77.80
74.83
90.10
91.22
90.66
27.19
36.29
31.09
CoOp
78.12
60.40
68.13
97.60
59.67
74.06
88.33
82.26
85.19
40.44
22.30
28.75
CpOpOp
70.49
73.59
72.01
94.87
71.25
81.71
90.70
91.29
90.99
23.71
27.74
ProDA [18]
74.70
71.20
72.91
97.70
68. 68
80.66
90.30
88.57
89.43
36,00 90
34.13
35.46
KgCoOp [11]
71,76
75.04
73.36
95.00
74.73
83.65
90.50
91.70
91.09
36.21
33.55
34,83
MaPLe
72.94
74,00
73.47
95.92
72.46
92.56
36.50
PRompeSRC [6]
78.27
74.97
76.58
98.07
76.50
85.95
90.67
91.53
91.10
42.73
37.87
40.15
TCP [12
80.80
74,13
77.32
97.73
75.57
85.23
90.57
91.37
90.97
11.97
34,43
37,83
MMA [70]
78.50
73.10
75.70
97.27
75.93
95.45
90.71
40.57
36.33
28F8 [87]
82,50
74,80
78 .64
99. 79
76.17
85.88
89.11
91.5 54
90.21
47.48
35.51
40.63
SkipT [88]
82,93
72.50
77.37
98.57
75.80
85.70
90.67
92.03
91.34
15.37
37.13
40.84
MMRL [8]
81.30
75.07
78.06
98.97
77.27
86.78
91.50
91.03
46.30
37.03
41.15
VaMP
83.78
80.14
81.91
98.96
83.97
90.85
92.77
93.16
92.96
46,77
41.13
43.76
Method
SUNN99
DTTD>
EanSSAT
UCP101
Base
Novel
H
Basc
Novel
H
Basc
Novel
11
Base
Novol
H
CLIP
69.36
75.35
72.25
53.24
59.90
56.37
70.53
77.50
CoOp
80.60
65.89
72:51
79:44
41.18
54.24
92.19
54.74
68.60
84.69
56.05
67.46
CpOpOp
74.74
76.86
78.27
77.01
56.00
61.85
87.49
60.01
71.21
82.33
73:45
77.64
ProDA [38]
78.67
76.93
77.79
80.67
56.48
83:90
66.00
73.88
71.97
78.04
KgCoOp [1]]
80.29
76.53
78.36
77.55
$1.99
61.35
$5.64
61.34
73.18
82,89
76.67
79.65
MaPLe
80.82
78.70
79.75
80.36
59.18
68.16
94.07
73:23
82.35
83.00
78.66
80.77
PRC
82.67
78.47
80.52
X3.37
62.97
71.75
92.90
73.90
87.10
78.80
82.74
TCP [12]
82.63
76.20
80.35
82.77
$8.07
68.25
91.63
74.75
52.52
87.13
80.77
83.83
MMA [70]
82.27
78.57
$0.38
83.20
65.66
73.38
85.46
82.34
83.87
86.23
60.03
82.20
2SPS [87]
82.59
78:91
80.70
65.01
73.52
96.91
67.09
79.29
87.85
78.19
82.74
SkipT [88]
82,40
79.03
80.60 68
83.77
67.23
74.59
92:47
83.00
57.48
87.30
82.47
84.81
MMRL
83.20
79.30
81.20
85.6
65.00
73.82
.0
80.17
87.21
88.10
60.07
63.89
VMMP
83.37
78.95
81
86.14
67.20
75.50
95.78
77.21
N8.52
78.99
83.48
where prompts are optimized on ImageNet and directly applied its variants (-V2, -S,
Table 2: Comparison of VaMP with previous state-of- methods on domain generalization across 4 datasets.
-A, -R) without any target supervision. As shown in Table 2, VaMP achieves the high- est accuracy across all four target domains, with an average accuracy of 61.73% out- performing the best baseline (MMRL) by 1.20%. The gain is particularly significant on Sketch, a challenging domain due to its abstract and textureless nature. On this subset, VaMP achieves 49.69%, improving over MMRL by 0.52%. These results vali- date the effécctivnness of our variational prompt modeling and class-aware regularization in enhancing out-of-distribution generalization, even in low-texture, cross-modality scenarios.
Source
Target
ImageNet
-V2
-S
CLIP [I]
66.73
60.83
46.15
47.77
73.96
CoOp 2
71.51
64.20
47.99
49.71
CoOpOp 3
71.02
64.07
48.75
50.63
76.18
MaPLe
70.72
64.07
49.15
50.90
76.98
PromptSRC [6]
71.27
64.35
49.55
50.90
77.80
MMA [70)
71.00
64.33
49.13
51.12
77.32
MMRL [8]
72.03
64.47
49.17
51.20
77.53
VaMP 72.83
64.96
49.69
51.97
78.01
Poos-DDttsett Generalization. To assess robustness under domain shift, we evaluate VaMP on cross-datase transfer, where prompts are tuned on one source dataset (ImageNet) and evaluated directly on unseen target datasets. As shown in Table 3, VaMP achieves the best average performance across 10 diverse target datasets, with an average accuracy of 67.74%, outperforming the strongest baseline (MMRL) by 0.49%. The improvement is particularly notable on challenging datasets with large domain gaps. For example, on EuroS remote sensing dataset with significant visual discrepancy from natural images-VaMe achieves target accuracy of 53,82%, improving over
8

---

Table 3: Comparison of VaMP with previous state-of-the-art methods cross-dataset evaluation across 10 datasets.
Target
POVCAiraalt
StanforCooon
Calteeht11
OxfondPets
InsgeeNer
Arorago
Foof10
SUN3Y7
EoooSAA AT
UCFR0
DTD
CoOp
71.51
63.88
93.70
89.14
61.51
68.71
85.30
18.47
64.15
41.92
46.39
66.55
CoOgOp
MaPl.c
70.72
66.30
93.53
90.49
63.57
72.23
86.20
24.74
67.01
46.49
48.06
68.09
PrempiSRC
[6
71.27
65.81
93.60
90.25
65.70
70.25
86.15
23.90
67.10
46.87
45.50
66.75
TCP
MMA 70
71.00
66.51
93.80
90.30
66.13
72.07
86.12
25.33
68.17
46.57
49.21
68.32
MMRL
72.03
67.25
94.67
91.43
66.
72.77
86.40
26.30
67.57
45.90
53.10
68.27
VaMP 72.83
67.74
94.96
91.79
66.10
73.18
86.97
26.76
46.82
53.82
68.93
Table 4: Ablation studies: impact of sample-specific multi-modal prompt generation, variational prompt adaptation and class-aware prior base-to-new generalization performance, averaged across 11 datasets.
(a) Effects of sample-specific multi-modal prompt generation
(b) Effects of variational multi-modal prompt gen- eration
Method
Prompt Type
Base
New
H
Method
Prompt Type
Base
New
H
task- lask-specific
82.28
75.14
78.55
Deterministic prompt
82.95
76.95
79.83
MaPLe [5]
sample-specific
82.95
76.95
79.83
MaPLe [5]
Variational prompt
84.77
77.32
80.87
task task- specispecific
85.68
77.16
81.20
Deterministic prompt
85.93
78.13
81.84
MMRL [8]
sample-specific
85,93
78.13
81.84
MMRL [8]
Variational prompt
86.11
78.45
82.10
(c) Effects of class-aware prior base-to-new generalization
Method
Prompt Type
Base
New
Normal gaussian prior
84.77
77.32
80.87
MaPLe
Class-aware prior
85.13
78.07
81.45
Normal gaussian prior
86,11
78.45
82.10
MMRL [8]
Class-aware prior
86.45
78.67
82.37
MMRL by 0.72%. These results demonstrate that structured and uncertainty- aware adaptation generalizes well across domains, even when the target distribution differs substantially from the source.
5.3 Ablation Study
Effects of Sample-specific Multi-Modal Prompt Generation. We begin by assessing the effect of samplc-specific multi-modal prompt generation. As shown in Table 4a, previous methods such as MaPLe and MMRL use task-specific prompts shared across all inputs. In contrast, specific design generates prompts conditioned on each input instance and injected across multiple layers. This structured, instance-aware formulation consistently improves generalization to novel categories, demonstrating the advantage of personalized multi-modas adaptation over fixed prompt configurations.
sample-
Effects of Variational Prompt Adaptation. Next, we evaluate the effect of variational modeling on the prompt tokens. While prior work adopts deterministic prompt embeddings, our approach treats the text-side prompts as latent variables inferred per instance. This formulation introduces uncertainty modeling into the prompt space, enabling better adaptation ambiguous out-of-distributio inputs. As shown Table 4b, applying variational prompt learning improves generalization across both MaPLe and MMRL backbones, validating the benefit of latent, sample-specific modeling.
Effects of Class-Aware Prior. We further assess the role of structured priors in regularizing the latent prompt space. Unlike standard variational methods that use an isotropic Gaussian prior, our method constructs class-aware prior from prototype representations computed over training data. This provides global semantic guidance for latent prompt inference. As shown in Table 4c, replacing the normal prior with class-aware one consistently improves performance, highlighting the importance of semantic regularization for class-consistent prompt adaptation.

---

Layer
Layer 9
Layer
Layer
Layer 9
Layer 12
Orldes keplower
Ferzlan
AMoon tlepkert
Shorken Husk
Atskan Melomuts
berhston
(a) ImageNet
(b) Flowers102
Figure 2: Qualitative analysis. Layer-wise visualization of aggregated posterior mean distributions go (o(z(z))
for sample images from (a) ImageNet and (b) Flowers102.
5.4 Qualitative Analysis
To gain intuitive insights into the learned latent prompt space, visualize the posterior distributions across text encoder layers. Figure 2 illustrates this process for representative samples from ImageNet and Oxford Flowers datasets. For each input image æ, we extract the image-conditionee posterior (z(a)(z)) at layers 6, 9, and 12. Since VaMP generates M prompt tokens per layer, we aggregate M their means and covariances Magg, (xr) MI Ej=1 1 (1) and (x) x7 Σド= d (z), then project these high- figh-ddmeeniioal vectors 2D using PCA. Contours represent uncertainty regions at 1.5g and 2.5o standard deviations.
These visualizations provide insights into how VaMP achieves uncertainty- prompt learning through its variational design. First, the varying distributional characteristics across layers and samples-reflecter in the spatial positions, orientations, and covariance structures of the posterior distributions- demonstrate that VaMP successfully models input-dependent uncertainty through its variational framework, moving beyond deterministic point estimates. Second, the layer-wise evolution reveals hierarchical uncertainty refinement: early layers (Layer 6) exhibit broader posterior distributions with larger variance to accommodate diverse prompt adaptations, while deeper layers (Layer 12) manifest tighter, more concentrated distributions with reduced uncertainty reflecting task-specific specialization. This progressive variance reduction enables adaptive representation learning at different semantic abstraction levels. Third, the consistent inter-class separation of posterior distributions across all depths demonstrates the effectiveness of the p=(z)c), which regularizes the variational posterior toward discriminative regions of the latent space. This class-conditional guidance ensures that prompts encode category-specific information from early feature extraction stages. Finally, VaMP's stochastic sampling mechanism, manifested through the distributional spread and overlap patterns, provides inherent robustness against overfitting to singular prompt configurations. By parameterizing prompts as probability distributions rather than fixed embeddings, VaMP balances representation flexibility with discriminative capacity throughout the hierarchical architecture.
prior
Conclusion
We presented VaMP, a variational framework for prompt adaptation in multi-modal representation learning. Our approach addresses the limitations of fixed, shared prompts by introducing structured and uncertainty-aware mechanism that adapts to individual input instances. VaMP comprises three key components: (i) sample-specific multi-modal prompt generation, where visual features condition prompt tokens across multiple transformer layers; (ii) variational modeling of text-side prompts as latent variables, enabling setancc-sppciffee and probabilistic adaptation; and (iii) class-aware prior constructed from semantic prototypes, which regularizes the latent space with global class-level information. Through extensive experiments few-shot and domain generalization benchmarks, we demonstrate that VaMP achieves state-of-the-art performance while maintaining high parameter efficiency. Our findings highlight the importance of modeling both instance-level variability and task structure in posset-based adaptation for languaee models.
10

---

Acknowledgements
This work is supported by the Hong Kong Research Grants Council General Research Fund (Grant No.: 17211024).
References
[1] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Míshkin, and Jack Clark. Learning transferable visual models from natural language supervision. In ICML, 2021.
[2] Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-language models. IJCV, 2022.
[3] Kaiyang Zhou, Jingkang Yang. Chen Change Loy, and Ziwei Liu. Conditional prompt learning for vision-language models. In CVPR, 2022.
[4] Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim. Visual prompt tuning. In ECCV, 2022.
[5] Muhammad Uzair Khattak, Hanoona Rasheed, Muhammad Maaz, Salman Khan, and Fa- had Shahbaz Khan. Maple: Multi-modal prompt learning. In CVPR, 2023.
[6] Muhammad Uzair Khattak, Syed Talal Wasim, Muzammal Naseer, Salman Khan, Ming-Hsuan Yang, and Fahad Shahbaz Khan. Self-regulating prompts: Foundational model adaptation without forgetting. In ICCV, 2023.
[7] Yingjun Du, Gaowen Liu, Yuzhang Shang, Yuguang Yao, Ramana Kompella, and Cees GM Snoek. Prompt diffusion robustifies any-modality prompt learning. arXiv preprint arXiv:2410.20164 2024.
[8] Yuncheng Guo and Xiaodong Gu. Mmrl: Multi-modal representation learning for vision- language models. In CVPR, 2025.
[9] Mohammad Mahdi Derakhshani, Enrique Sanchez, Adrian Bulat, Victor G Turrisi da Costa, Cees GM Snoek, Georgios Tzimiropoulos. and Brais Martinez. Bayesian prompt learning for image-language model generalization. In ICCV, 2023.
[10] Zehao Xiao, Jiayi Shen, Mohammad Mahdi Derakhshani, Shengeai Liao, and Cees GM Snoek. Any-shift prompting for generalization over distributions. In CVPR, 2024.
[11] Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung. Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. In ICML, 2021.
[12] Lewei Yao, Runhui Huang, Lu Hou, Guansong Lu, Minzhe Niu, Hang Xu, Xiaodan Liang, Zhenguo Li, Xin Jiang, and Chunjing Xu. Filip: Fine-grained interactive language-image pre-training. In ICLR, 2022.
[13] Juncheng Li, Xin He, Longhui Wei, Long Qian, Linchao Zhu, Lingxi Xie, Yueting Zhuang, Qi Tian, and Siliang Tang. Fine-grained semantically aligned vision-language pre-training. In NeurlPS, 2022.
[14] Wonjae Kim, Bokyung Son, and Ildoo Kim. Vilt: Vision-and-languag. transformer without convolution or region supervision. In ICMI., 2021.
[15] Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee. Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks. In NewrIPS, 2019.
[16] Hao Tan and Mohit Bansal. Lxmert: Learning cross-modality encoder representations from transformers. In EMNLP-IJCNLP, 2019.
[17] Weijie Su, Xizhou Zhu, Yue Cao, Bin Li, Lewei Lu, Furu Wei, and Jifeng Dai. Vl-bert: Pre-training of generic visual-linguistic representations. In ICLR, 2019.

---

[18] Junnan Li, Ramprasaath Selvaraju, Akhilesh Gotmare, Shafiq Joty, Caiming Xiong, and Steven Chu Hong Hoi. Align before fuse: Vision and language representation learning with momentum distillation. In NeurIPS, 2021.
[19] Yuqi Huo, Manli Zhang, Guangzhen Liu, Haoyu Lu, Yizhao Gao, Guoxing Yang. Jingyuan Wen, Heng Zhang, Baogui Xu, and Weihao Zheng. Wenlan: Bridging vision and language by large-scale multi-modal pre-training. arXiv preprint arXhv:2103.06661. 2021.
[20] Peng Gao, Shijie Geng, Renrui Zhang, Teli Ma, Rongyao Fang, Yongfeng Zhang, Hongsheng Li, and Yu Qiao. Clip-adapter: Better vision-language models with feature adapters. IJC, 2024.
[21] Renrui Zhang, Rongyao Fang, Peng Gao, Wei Zhang, Kunchang Li, Jifeng Dai, Yu Qiao, and Hongsheng Li. Tip-adapter: Training-free clip-adapted for better vision laiguala modeling, In ECCV, 2022.
[22] Jiaming Li, Jiacheng Zhang, Jichang Li, Ge Li. Si Liu, Liang Lin, and Guanbin Li. Learning background prompts discover implicit knowledge for open vocabulary object detection. In CVPR, 2024.
[23] Muhammad Maaz, Hanoona Rasheed, Salman Khan, Fahad Shahbaz Khan, Rao Muhammad Anwer, and Kinnn-suaan Yang. Class-agnostie object detection with multi-modal transformer. In ECCV, 2022.
[24] Xingyi Zhou, Rohit Girdhar, Armand Joulin, Philipp Krähenbühl, and Ishan Misra. Detecting twonty-thousand classes using image-level supervision. In ECCV, 2022.
[25] Xiuye Gu, Tsung-Ye Yi Lin, Weicheng Kuo, and Yin Cui. Open-vocabulary object detection via vision and language knowledge distillation. In ICLR, 2022.
[26] Yuhang Zang, Wei Li, Kaiyang Zhou, Chen Huang, and Chen Change Loy. Open-vocabulary detr with conditional matching. In ECCV, 2022.
[27] Tianheng Cheng, Lin Song, Yixiao Ge, Wenyu Liu, Xinggang Wang, and Ying Shan. Yolo-world: Real-time open-vocabulary object detection. In CVPR, 2024.
[28] Xiangtai Li, Haobo Yuan, Wei Li, Henghui Ding, Size Wu, Wenwei Zhang, Yining Li, Kai Chen, and Chen Change Loy. Omg-seg: Is one model good enough for all segmentation? In CVPR, 2024.
[29] Yongming Rao, Wenliang Zhao, Guangyi Chen, Yansong Tang. Zheng Zhu, Guan Huang, Jie Zhou, and Jiwen Lu. Denseclip: Language-guide dense prediction with context-aware prompting. In CVPR, 2022.
[30] Xiangtai Li, Henghui Ding, Haobo Yuan, Wenwei Zhang, Jiangmiao Pang, Guangliang Cheng, Kai Chen, Ziwei Liu, and Chen Change Loy. Transformer-hasee visual segmentation: A survey. IEEE TPAMI, 2024.
[31] Timo Lüddecke and Alexander Ecker. Image segmentation using text and image prompts. In CVPR, 2022.
[32] Wenfang Sun, Yingjun Du, Gaowen Liu, Ramana Kompella, and Cees GM Snoek. Training-free semantic segmentation via llm-supervision. arXiv preprint arXiv:2404.00701, 2024.
[33] Zeyu Han, Fangrui Zhu, Qianru Lao, and Huaizu Jiang. Zero-shot referring expression compre- hension via structural similarity between images and captions. In CVPR, 2024.
[34] Silin Cheng, Yang Liu, Xinwei He, Sebastien Ourselin, Lei Tan, and Gen Luo. Weakmen: Multi-task collaborative network for weakly supervised referring expression comprehension and segmentation. In CVPR, 2025.
[35] Taylor Shin, Yasaman Razeghi, Robert I. Logan IV, Eric Wallace, and Sameer Singh. Auto- prompt: Eliciting knowledge from language models with automatically generated prompts. In EMNLP, 2020.
12

---

[36] Zhengbao Jiang, Frank F Xu, Jun Araki, and Graham Neubig. How can we know what language models know? TACL, 2020.
[37] Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 2023.
[38] Yuning Lu, Jianzhuang Liu,, Yonggang Zhang, Yajing Liu, and Xinmei Tian. Prompt distribution learning. In CVPR, 2022.
[39] Beier Zhu, Yulei Niu, Yucheng Han, Yue Wu, and Hanwang Zhang. Prompt-aligned gradient for prompt tuning. In ICCV, 2023.
[40] Xinyang Liu, Dongsheng Wang. Miaoge Li, Zhibin Duan, Yishi Xu, Bo Chen, and Mingyuan Zhou. Patch-token aligned bayesian prompt learning for vision- lision-language models, arXiv preprint arXiv:2303.09100 2023.
[41] Hantao Yao. Rui Zhang, and Changsheng Xu. Visual-language prompt tuning with knowledge- guided context optimization. In CVPR, 2023.
[42] Hantao Yao, Rui Zhang, and Changsheng Xu. Tep: Textual-based class-aware prompt tuning for visual-language model. In CVPR, 2024.
[43] Jinyoung Park, Juyeon Ko, and Hyunwoo J Kim. Prompt learning via meta-regularizztionn In CVPR, 2024.
[44] Xinyu Tian, Shu Zou, Zhaoyuan Yang, and Jing Zhang. Argue: Attribute-gudde prompt tuning for vision-language models. In CVPR, 2024.
[45] Adrian Bulat and Georgios Tzimiropoulos. Lasp: Text-to-text optimization for language-aware soft prompting of vision & language models. In CVPR, 2023.
[46] Yingjun Du, Wenfang Sun, and Cees Snoek. Ipo: Interpretable prompt optimization for vision-language models. In NeurIPS, 2024.
[47] Zifeng Wang, Zizhao Zhang, Chen- Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Pfister. Learning to prompt for continual learning. In CVPR, 2022.
[48] Hyojin Bahng, Ali Jahanian, Swami Sankaranarayanan, and Phillip Isola. Exploring visual prompts for adapting large-scale models. arXiv preprint 2.t3::2003.12774, 2022.
[49] Feng Li, Qing Jiang, Hao Zhang, Tianhe Ren, Shilong Liu, Xueyan Zou, Huaizhe Xu, Hongyang Li, Jianwei Yang. Chunyuan Li, et al. Visual in-context prompting. In CVPR, 2024.
[50] Lingfeng Yang. Yueze Wang, Xiang Li, Xinlong Wang, and Jian Yang. Fine-grained visual prompting. In NeurlPS, 2024.
[51] Yi-Lun Lee, Yi-Hsuan Tsai, Wei-Chen Chiu, and Chen Yu Lee. Multimodal prompting with missing modalities for visual recognition. CVPR, 2023.
[52] Yaowei Li, Ruijic Quan, Linchao Zhu, and Yi Yang. Efficient multimodal fusion via interactive prompting. In CVPR, 2023.
[53] Shuvendu Roy and Ali Etemad. Consistency-guided prompt learning for vasion-language models. In ICLR, 2024.
[54] Zheng Li, Xiang Li, Xinyi Fu, Xin Zhang, Weiqiang Wang, Shuo Chen, and Jian Yang. Promptkd: Unsupervised prompt distillation for vasion-language models. In CVPR, 2024.
[55] Haoyang Li, Liang Wang. Chao Wang, Jing Jiang, Yan Peng, and Guodong Long. Dpc: Dual-prompt collaboration for tuning vision-language models. In CVPR, 2025.
[56] Ming Ding, Zhuoyi Yang. Wenyi Hong, Wendi Zheng, Chang Zhou, Da Yin, Junyang Lin, Xu Zou, Zhou Shao, Hongxia Yang, et al. Cogview: Mastering text-to-image generation via transformers. In NewrIPS, 2021.
13

---

[57] Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Lya Sutskever. Zero-shot text-to-image generation. ICML, 2021.
[58]
Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Pro- lificdreamer: High-fidelity and diverse text-to-3d generation with variational score distillation. In NewrIPS, 2023.
[59] Peihao Wang. Dejia Xu, Zhiwen Fan, Dilin Wang. Sreyas Mohan, Forrest landola, Rakesh Ranjan, Yilei Li, Qiang Liu, Zhangyang Wang, et al. Taming mode collapse in score distillation for text-to-3d generation. In CVPR, 2024.
[60] Hyung-gun Chi, Myoung Hoon Ha, Seunggeun Chi, Sang Wan Lec, Qixing Huang, and Karthik Ramani. Infogen: Representation learning for human skeleton-based action recognition. In CVPR, 2022.
[61] Namdar Homayounfar, Yuwen Xiong, Justin Liang, Wei-Chiu Ma, and Raquel Urtasun. Levelset r-enn: A deep variational method for instance segmentation. In ECCV, 2020.
[62] Mingyu Lee and Jongwon Choi. Text-guided variational image generation for industrial anomaly detection and segmentation. In CVPR, 2024.
[63] Ziyao Zeng, Daniel Wang, Fengyu Yang, Hyoungseob Park, Stefano Soatto, Dong Lao, and Alex Wong. Wordepth: Variational language prior for monocular depth estimation. In CVPR, 2024.
[64] Jian Zhang, Chenglong Zhao, Bingbing Ni, Minghao Xu, and Xiaokang Yang. Variational few-shot learning. In CVPR, 2019.
[65] Jingyi Xu and Hieu Le. Generating representative samples for few-shot classification. In CVPR. 2022.
[66] Wenfang Sun, Yingjun Du, Xiantong Zhen, Fan Wang, Ling Wang, and Cees GM Snoek. Metamodulation: Learning variational feature hierarchies for few-shot learning with fewer tasks. In ICML, 2023.
[67] Jie Liu, Yingjun Du, Zehao Xiao, Cees GM Snoek, Jan-Jakob Sonke, and Efstratios Gavves. Memory-augmented variational adaptation for online few-shot segmentation. In CVPR, 2023.
[68] Yingjun Du, Jun Xu, Huan Xiong, Qiang Qiu, Xiantong Zhen, Cees GM Snoek, and Ling Shao. Learning to learn with variational information bottleneck for domain generalization. In ECCV, 2020.
[69] Pan Li, Da Li, Wei Li, Shaogang Gong, Yanwei Fu, and Timothy M Hospedales. simple feature augmentation for domain generalization. In ICCV, 2021.
[70] Lingxiao Yang, Yu-Yuan Zhang, Yanchen Wang, and Xiaohua Xie. Mma: Multi-modal adapter for vision-language models. In CVPR, 2024.
[71] Diederik P Kingma, Max Welling, et al. Auto-encoding variational bayes. In ICLR, 2014.
[72] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, 2009.
[73] Li Fei-Fei, Rob Fergus, and Pietro Perona. Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. In CVPR workshop. 2004.
[74] Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar. Cats and dogs. In CVPR, 2012.
[75] Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine-grained categorization. In ICCV, 2013.
[76] Maria-Flena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In ICVGIP, 2008
14

---

[77] Lukas Bossard. Matthieu Guillaumin, and Luc Van Gool. mood-101-mining discriminative components with random forests. In ECCV, 2014.
[78]
Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi. Fine- grained visual classification of aircraft. arXiv preprint 1rXiv:1306.5500 2013.
[79]
Jianxiong Xiao, James Hays, Krista A Ehinger, Aude Oliva, and Antonio Torralba. Sun database: Large-scale scene recognition from abbey to zoo. In CVPR, 2010.
[80] Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. Ucf101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv: 2012.
[81] Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describing textures the wild. In CVPR, 2014.
[82] Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2019.
[83] Benjamin Recht. Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do imagenet classifiers generalize to imagenet? In ICML, 2019.
[84] Haohan Wang, songwei Ge, Zachary Lipton, and Eric P Xing. Learning robust global represen- tations by penalizing local predictive power. In NeurlPS, 2019.
[85] Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial examples. In CVPR, 2021.
[86] Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, and Mike Guo. The many faces of robustness: A critical analysis of out-of-distribution generalization. In ICCV, 2021.
[87] Matteo Farina, Massimiliano Mancini, Giovanni Iacca, and Elisa Ricci. Rethinking few-shot adaptation of vision-language models in two stages. In CVPR, 2025.
[88] Shihan Wu, Ji Zhang, Pengpeng Zeng, Lianli Gao, Jingkuan Song, and Heng Tao Shen. Skip tuning: Pre-trained vision-language models are effective and efficient adapters themselves. In CVPR, 2025.
[89] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. OpenAl blog, 2019.
[90] Seokju Cho, Heeseong Shin, Sunghwan Hong, Anurag Arnab, Paul Hongsuck Seo, and Seun- gryong Kim. Cat-seg: Cost aggregation for open-vocabulayy semantic segmentation. In CVPR, 2024.
[91] Xiaohu Huang, Hao Zhou, Kun Yao, and Kai Han. Froster: Frozen clip a strong teacher for open- vpec-vocabulary action recognition. In ICLR. 2024.
[92] Jian Ding, Nan Xue, Gui-Song Xia, and Dengxin Dai. Decoupling zero-shot semantic segmen- tation. In CVPR, 2022.
[93] Feng Liang, Bichen Wu, Xiaoliang Dai, Kunpeng Li, Yinan Zhao, Hang Zhang, Peizhao Zhang, Peter Vajda, and Diana Mareulescu. Open-vocabulary semantic segmentation with mask-adapted clip. In CVPR, 2023.
[94] Mengde Xu, Zheng Zhang, Fangyun Wei, Han Hu, and Xiang Bai. Side adapter network for open-vocabulary semantic segmentation. In CVPR, 2023.
[95] Hanoona Rasheed, Muhammad Uzair Khattak, Muhammad Maaz, Salman Khan, and Fa- had Shahbaz Khan. Fine-tuned clip models are efficient video learners. In CVPR, 2023,
(96] Zejia Weng, Xitong Yang, Ang Li, Zuxuan Wu, and Yu-Gang Jiang. Open-velip: Transforming clip to an open-vocabulary video model via interpolated weight optimization. In ICML, 2023,
15

---

[97] Quan Sun, Yuxin Fang, Ledell Wu, Xinlong Wang, and Yue Cao. Eva-clip: Improved training techniques for clip scale. arXiv preprint arXiv: 15389, 2023.
[98] Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In ICCV, 2023.
[99] Michael Tschannen, Alexey Gritsenko, Xiao Wang, Muhammad Ferjad Naeem, Ibrahim Alab- dulmohsin, Nikhil Parthasarathy, Talfan Evans, Lucas Beyer, Ye Xia, Basil Mustafa, et al. Siglip 2: Multilingual vision-language encoders with improved semantic understanding, localization, and dense features. arXiv preprint arXiv:2502. 14786, 2025.
16

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We clearly state contributions in Abstract and also Section 1. These contributions are well validated by our experimental results Section 5.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We provide "limitation" subsection in the supplementary material.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
17

---

Justification:
We have provided the theoretical proof in the appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We have specified all the training details in Section 5.1 and supplemental materials.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
18

---

reproducible, they Answer: [Yes]
Justification: (1) Data: All the datasets we used in this paper are publicly available online, and all the readers are free download them. We list the statistics of all the used datasets in the supplementary material. (2) Code: Code will be available after paper got accepted.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (kttps://nips.co
Sublic/guides //c//o/Sumiissionn//i
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: //nips. wttpccccpppcccccccccccccccccccct wwwpc//guss/.eee/ee SCt///oodiiissaeeeeeeeee ssionPolicy) for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Following the standard experimental setup, we repeat each experiment over random seeds and report the mean of the results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
19

---

The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative emrr rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide the computing resources in experiments 5,
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: //weurrurs....// cccpublicc/eeccccccccccccc Caabblic/Etiice htwp:/tttihiiieeeeeeeeeeeeeeeeee iiinees?dllle??
Answer: [Yes]
Justification:
We reviewed and followed the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer:
[Yes]
Justification: provide the potential broader impacts supplemental materials.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
20

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The data and models pose no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We cite the original papers that produced the code package and datasets.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/dataseta has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
21

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: Details of the datasets/model are provided in the supplemental materials. The code will be released after the paper get accepted.
Guidelines:
The answer NA means that the paper does not release assets.
Researchers should communicate the details the dataset/code/mmodll as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
This paper does not involve crowdsourcing nor research with human subjects
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions. do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
22

---

Appendix
A Dataset Details
Details of 14 datasets are shown in Table Al.
Table A1: Summary of the 14 datasets.
Dataset
Classes
Train
Val
Test
Description
ImageNet [72]
1,000
1.28M
50,000
Recognition of generic objects
Caltech101 [73]
100
4,128
1,649
2,465
Recognition of generic objects
OxfordPets [74]
2,944
3,669
Fine-grained classification of pets Fine aeeeggriied classification of cars Fine- tine-grained classification of flowers
StanfondCars [75]
196
6,509
1,635
8,041
Flowers 02 [76]
102
4,093
1,633
Food101 [77]
101
50,500
20,200
30,300
Fine- mee-ggained classification of foods
FGVCAircraft [78]
100
3,334
3,333
3,333
Fine-grat Fine-grained classification of aircrafts
SUN397 [79]
397
15,880
3,970
19,850
Scene classification
DID [81]
47
2,820
1,128
1,692
Texture classification
EuroSAT [82]
10
13,500
5,400
8,100
Land use & cover classification with satellite images
UCF101 [80]
101
7,639
1,898
3,783
Action recognition
Ieag [83]
1,000
10,000
New test data for ImageNet
122 2
ImageNet-Sketch [84]
1,000
50,889
Sketch-style images of ImageNet classes
ImageNet-A [85]
200
7,500
Natural adversarial examples of 200 ImageNet classes
ImageNet-R [86]
200
30,000
Renditions 200 ImageNet classes
B More Implementation Details
All models are trained using the AdamW optimizer with a learning rate of 0.001 and a weight decay of 0.01. The batch size is set 32 for ImageNet and for all other datasets. We apply automatic mixed-precte preccipeeiioon training throughout to improve efficiency. For base-to-novel generalization on ImageNet, we train for 5 epochs; for other datasets train for 10 epochs. For cross-datasel and domain generalization tasks, we train on ImageNet for a single epoch. Few-shot learning tasks use 5 training epochs ImageNet and 50 epochs on target datasets. All reported results are averaged over three independent runs. All prompts and representation tokens are initialized from a zero-mean Gaussian distribution with a standard deviation of 0.02. For EuroSAT, we follow MMRL [8] and set the representation token dimension d, = 2048; for all other datasets, we use d, = 512. The fusion parameter a in MMRL-style classifiers fixed to 0.7. The average accuracy reported over three independent runs. For variational modeling, we use a two-layer MLP with GELU activation to parameterize both the posterior network and the prior network 3/, outputting mean and log-variance vectors per layer. The latent variables 2 are sampled using the reparameterization trick, and we perform = 10 Monte Carlo samples at inference time. Class prototypes og are computed offline at the start of training.
C Efficiency Analysis
We analyze the efficiency of our variational multi-modal prompt framework, focusing on the trade- off between inference cost and generalization ability. As detailed in Table A2, VaMP introduces only a modest overhead compared to the strong MMRL baseline. For instance, with = 10, our method improves the Harmonic Mean (HM) from 81. $1.20 to 82.37 with a minimal latency increase of just 0.8ms per image on an NVIDIA V100 GPU. The performance gains saturate quickly as increases, demonstrating that VaMP achieves an effective balance between accuracy and efficiency even with limited sampling. Furthermore, our approach is parameter-efficient, increasing the number of learnable parameters by less than 6% when integrated into MMRL (from 4.992M to 5.12M.). This balance demonstrates that VaMP can be deployed with low overhead while still leveraging uncertainty modeling for improved generalization.
23

---

Table A2: Analysis of performance efficiency trade-off.
Method
S
Inference Time (ms/img)
Base
Novel
HM
MMRL
5.3
85.68
77.16
81.20
MMRL+ VaMP
5.5
86.10 10
77.35
81.44
MMRL+ VaMP
5
5.8
86.37
78.26
82.03
MMRL+ VaMP
10
6.1
86.45
78.67
82.37
MMRL+ VaMP
20
6.5
86.47
78.37
82.22
MMRL+ VaMP
50
9.3
86.43
78.15
82.01
D More Ablation Studies
We conduct ablation studies by integrating our variational prompt adaptation into the MMRL [8] framework, analyzing the impact of key hyperparameters such as the prompt depth, and prompt length. All results are reported on the base-to-novel generalization benchmark (11 datasets), using average accuracy across base and novel splits.
Effect of Prompt Insertion Depth J and H. We vary the transformer layer index J where prompt tuning begins, and the number of consecutive layers H to which prompts added. As shown in Table A3, inserting prompts deeper into the encoder (e.g., starting from layer = 5) and extending them to more layers (e.g., H 7) leads to better performance both base and novel classes. This confirms the benefit of hierarchical prompt injection for deeper vision-language alignment.
Table A3: Effect prompt insertion depth (J, H).
J
H
Base
Novel
H
3
85.57
77.14
81.13
6
5
86.28
78.02
82.01
7
86.45
78.67
82.37
Effect of Prompt Token Length M. We analyze the sensitivity to the number of prompt tokens M injected per layer. Table A4 shows that increasing M from to 5 improves accuracy, the model benefits from higher representational capacity. However, further increasing to M = 8 slightly reduces generalization, likely due redundancy and overfitting. Hence, set M as the default in main experiments.
Table A4: Effect of prompt token length per layer.
M
Base
Novel
H
1
85.76
77.21
81.30
3
86.18
78.05
82.02
5
86.45
78.67
82.37
8
86.19
78.12
82.00
E
Detailed Derivation of the Variational Lower Bound (ELBO)
We derive the evidence lower bound (ELBO) used in our variational prompt learning framework. Our goal is to estimate the conditional likelihood of the class label y given image and text prompt template f, where the latent variables represent the sample-specific text prompt tokens injected across multiple layers of the language encoder.
The marginal likelihood is obtained by integrating out the latent variables 2:
log æ. t) = log p(y x, t, p(z xx) dz.
(23)
24

---

Since this integral is generally intractable, we introduce variational posterior 96(2 x) and apply Jensen's inequality:
logp(y ax,t) log / g0(2 xx) ply [x,t, 2)
de
(24)
( xx)
= (x(()))) log ply x., 2) p(2 x))
(25)
g0(= x))
=Byo(s)a)) [log ply
x; L 2)]
(T x)) 1| x))
(26)
expected log-likelitood
diverreee
We model z as a collection of layer-wise latent prompt embeddings (=) one for each of the
H transformer layers in the text encoder. We assume the posterior and prior factorize across layers:
J+H-1
( 2))
II
g
[x),
(27)
imJ
J+
p( x))
II
p(z) x).
(28)
This leads the following form of the ELBO:
J+H-1
LELBO
(xula)) logp(y 2,,,, 2:)
KL(gg(=: [x) x))]
(29)
where is injected at layer i of the frozen text encoder, modulating the token representations via concatenation.
During training, we replace p(z: x) with a class- aware prior:
Po (2; 0y) = - (A,, diag((),))))
= (cy)).
(30)
where the class prototype oy is the mean of posterior means f over all training samples class y. The final training objective maximizes the ELBO in Eq. 29.
This variational formulation enables model to learn expressive, uncertainty-aware, sample-specific prompts while regularizing the latent space with class-level semantic structure.
Extension to Other Tasks
To assess the generalizability of VaMP beyond standard image classification, we extended our evaluation to the more complex domains of opea-vocabylary segmentation and action recognition. Our approach was integrated into two state-of-the-art frameworks: CAT-Seg [90] for segmentation and FROSTER [91] for action recognition, both of which utilize CLIP ViT-B/16 backbone. This compatibility allowed for seamless and direct evaluation of VaMP's effectiveness these diverse tasks.
For the open-vocabulary segmentation task, adhered the established CAT-Seg protocol. The model was trained on the COCO-Stuff dataset (118K images, 171 categories) and subsequently evaluated on several challenging benchmarks, including ADE20K. PASCAL-Contes and PASCAL VOC, which feature wide range of category scales (59-847 classes). The results are detailed in Table A5.
For -pcabuocab action recognition, we adopted the base-to-nove evaluation protocol from FROSTER. This setup involves training the model exclusively the base classes from widely used video benchmarks, Kinetics-400 and UCF-101. The primary metric the model's ability to generalize novel, unseen action categories during testing. Table A6 summarizes these results.
The consistent performance gains observed across both segmentation and action recognition tasks underscore the scalability and robust generalization capabilities of our proposed framework.
25

---

Table A5: Performance for open-vocabulary segmentation.
Method
Prompt Tuning
ADE-847
PC-459
ADE: 150
PC-59
VOC-20
Former [92]
5.6
10.4
18.0
45.5
89.5
OVSeg [93]
7.1
11.0
24.8
53.3
92.6
SAN [94]
10.1
12.6
27.5
53.8
94.0
CAT-Seg [90]
12.0
19.0
31.8
57.5
94.6
CAT-Seg
MMRL
12.8
18.7
32.4
57.9
94.3
CAT-Seg
MMRL+VaMP
13.9 (+1.1)
20.3 (+1.6)
33.3 (+0.9)
58.6 (+0.7)
95.2 (+0.9)
Table A6: Performance open-vocabulary action recognition.
Base classes, Novel classes, HM:
Harmonic mean.
Method
Prompt Tuning
K400(B)
K400N)
K400(HM)
UCR(B)
UCF(N)
UCR(HM)
ViFi-CLIP [95]
76.4
61.1
67.9
92.9
67.7
78.3
Open-VCLIn [96]
62.6
68.9
94.8
77.5
85.3
FROSTER [91]
77.8
64.3
70.4
95.3
80.0
87.0
FROSTER
MMRL MMRL+VaMM
78.3
64.1
70.5
95.5
80.2
87.2
FROSTER
78.8 (+0.5)
64.8 (+0.7)
71.1 (+0.6)
96.1 (+0.6)
81.0 (+0.8)
87.9 (+0.7)
G Scalability to Other VLM Architectures
To demonstrate the architecture-agnue agnnostic nature of our method, we validated VaMP on several state- of-the-art VLMs that represent the latest advances in CLIP-style architectures, Our method can be readily integrated into more complex VLMs, as it only requires access to intermediate transformer layers and the ability to inject prompt tokens. The variational adaptation module is lightweight (MLPs per layer) and does not assume any specific backbone structure. We therefore integrated VaMP into several prominent models, including EVA-CLIP [97]. SigLIP [98], and SigLIP 2 [99]. These models share similar structural designs with CLIP, which enabled their rapid adaptation for our experiments. As shown in Table A7, VaMP obtains clear and consistent performance improvements across all VLMMs, demonstrating its strong generalization capability to unseen categories- critical challenge in prompt learning. The consistent improvements across different VLMs further substantiate the effectiveness and generalizability our approach to different architectures.
Table A7: Performance comparison of different VLMs on ViT-B/16 backbone under base-to-novel generalization sctting.
VLM
Method
Base
Novel
H
CLIP [1]
MMRL
85.68
77.16
$1.20
CLIP [1]
MMRL+ VaMP 86.45 (+0.77)
78.67 (+1.51)
82.37 (+1.17)
EVA-CLIP [97]
MMRL
85.97
77.69
81.62
EVA-CLIP [97]
MMRL+ VaMP
86.59
(+0.62)
79.18 (+1.49)
82.71 (+1.09)
Ii [98]
MMRL
86.
78.21
81.97
SigLIP [98]
MMRL+ AMRL+VaMP
86.88
(+0.76)
79.45 (+1.24)
83.00 (+1.03)
SigLIP 2 [99]
MMRL
86.64
78.97
82.62
SigLIP 2 [99]
MMRL+VaMP
87.09 (+0.45)
80.02 (+1.05)
83.40 (+0.78)
H
Broader Impact and Limitations
Our work presents variational framework for sample-specific, uncertainty-aware prompt adaptation in vision-language models, aiming to improve robustness under distribution shifts and limited super- vision. The proposed method has the potential to benefit wide range of downstream applications where multi-moda understanding and generalization are essential, such as assistive Al systems, open-world recognition, or low-resource domain transfer. The probabilistic modeling component can further inspire future efforts in calibrated and interpretable multi-modal adaptation. At present, we have not identified any ethical concerns associated with the real-world applications of this technology. However, strongly recommend continuous monitoring and evaluation to ensure its responsible
26

---

and ethical deployment. Our approach also has certain limitations. First, our class-aware prior construction relies on access class prototypes computed from training data, which may not be available in zero-label scenarios. Extending our method to work under fully unsupervised or few-label conditions remains an open direction. Second, our experiments focus on classification tasks; the extension to generative or structured prediction settings (e.g., image captioning. VQA) requires further investigation and architectural adaptation. Despite these challenges, we believe our framework takes an important step toward principled and efficient multi-modal prompt learning, and hope it provides useful insights for future research in uncertainty-awar adaptation, vision-language alignment, and lightweight tuning strategies.
27


Reviews:
[{"review_id": "eyLzsoN1VF", "reviewer": "Reviewer_hikd", "summary": "This paper proposes a novel framework for adapting vision-language models (VLMs), like CLIP, to new tasks with limited supervision. It introduces sample-specific, uncertainty-aware prompt tuning by modeling text-side prompts as latent variables conditioned on image inputs, and regularizing them with a class-aware prior that encodes semantic structure. Experiments on few-shot, cross-dataset, and domain generalization tasks demonstrate that VaMP outperforms state-of-the-art baselines while maintaining high parameter efficiency.", "questions": "1. Can you provide more insight into the computational overhead introduced by variational sampling during inference, and whether there is a trade-off between accuracy and efficiency?\n2. Would it be better if put efficiency analysis in the main paper rather than being placed in the appendix? The current experiments primarily highlight the effectiveness of the method, and incorporating efficiency results would provide a more practical perspective.\n3. Can you provide qualitative examples to illustrate the generated prompts' structure or diversity?\n4. Can VaMP be scaled or adapted to newer multi-modal models like GPT-4V, which use more complex fusion mechanisms?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strength:\n1. VaMP generates prompts dynamically based on each input image, which allows the model to adapt better to individual samples and handle uncertainty—especially beneficial in few-shot or out-of-distribution scenarios.\n2. By incorporating a class-aware prior based on semantic prototypes, the method enhances intra-class consistency and improves generalization, especially across domains.\n3. Extensive experiments and good results.\n\nWeakness:\n1. Constructing the class-aware prior depends on class labels to compute prototypes, which may not generalize well to unsupervised or label-scarce settings.\n2. Only classification tasks, need to show the potential to extend this method to other tasks, to show the impact.\n3. The use of variational sampling introduces additional computational overhead during inference. Although there's efficiency analysis in appendix, the trade-off between improved generalization and increased latency has not been fully quantified or explored in depth.\n4. Scalability to more complex architectures is not clearly stated.", "was_revised": true, "final_justification": "Thanks authors for the detailed rebuttal, which make sense to me and will make paper stronger if final version could be revised. I'll suggest accept this paper.", "rebuttal": "We appreciate the reviewers’ valuable feedback and their recognition of our framework’s novelty, robust generalization, and good performance. We have carefully addressed the comments below.\n\n> W1: Constructing the class-aware prior depends on class labels to compute prototypes, which may not generalize well to unsupervised or label-scarce settings.\n\n**R:** Our experiments demonstrate that VaMP performs well in label-scarce settings. In the base-to-novel setting, we only have access to base class labels during training with merely 16 shots per class, representing a highly label-scarce scenario. For novel classes, no label information is available during training. Under this constraint, prototypes are computed only for base classes, yet VaMP achieves the best novel class performance (Table 1), showing better generalization to unseen categories. To further validate this finding, we conducted additional experiments under the extreme 1-shot constraint as suggested by Reviewer EaPS (Table A5), where VaMP continues to outperform existing methods, thereby corroborating its effectiveness under severe label scarcity. Moreover, in the cross-dataset evaluation (Table 2), the model is trained solely on ImageNet using 16 shots per class and directly tested on unrelated datasets without using any label or prototype information, effectively operating in a zero-shot setting. Despite these limitations, VaMP still consistently outperforms prior methods. At test time, we revert to a standard Gaussian prior, ensuring applicability without label access.\n\nWe believe the key reason for our method’s strong generalization is its variational formulation. It has been shown in the literature (e.g., [a]) that VAE-like designs in traditional generative models exhibit robust generalization capabilities because the latent variable acts as a structured bottleneck that captures salient, disentangled factors of variation. Similarly, in VaMP, the latent prompt distribution encourages the model to encode task-relevant variations in a sample-specific yet class-agnostic manner, enabling strong zero-shot transfer. This design makes VaMP particularly suitable for real-world applications where labeled data is scarce or expensive to obtain.\n\n\nMeanwhile, we agree that our approach still requires at least some labeled data to compute meaningful class prototypes, which is a practical assumption and normally easily accessible. It remains an open challenge to obtain reliable prototypes in completely unsupervised scenarios, which we consider as the future work.\n\n[a] Bayesian Invariant Risk Minimization. CVPR 2022.\n\n> W2: Extension to other tasks.\n\n**R:** Following this suggestion, we evaluated VaMP's generalizability beyond image classification task by conducting experiments on open-vocabulary segmentation and action recognition tasks. We integrated VaMP into state-of-the-art frameworks of two tasks: CAT-Seg[a] for segmentation and FROSTER[b] for action recognition, utilizing CLIP ViT-B/16 as the backbone encoder. Both baselines are extensions built upon CLIP, allowing our method to be seamlessly integrated into these approaches. \n\nFor open-vocabulary segmentation, we followed CAT-Seg's protocol, training on COCO-Stuff (118K images, 171 categories) and evaluating across multiple benchmarks including ADE20K, PASCAL-Context, and PASCAL VOC with varying category scales (59-847 classes). The results are shown below in Table A1:\n\n**Table A1: Performance for open-vocabulary segmentation.**\n\n| Method | Prompt Tuning | ADE-847 | PC-459 | ADE-150 | PC-59 | VOC-20 |\n|--------|---------------|---------|--------|---------|-------|--------|\n| ZegFormer | - | 5.6 | 10.4 | 18.0 | 45.5 | 89.5 |\n| OVSeg | - | 7.1 | 11.0 | 24.8 | 53.3 | 92.6 |\n| SAN | - | 10.1 | 12.6 | 27.5 | 53.8 | 94.0 |\n| CAT-Seg | - | 12.0 | 19.0 | 31.8 | 57.5 | 94.6 |\n| CAT-Seg | MMRL | 12.8 | 18.7 | 32.4 | 57.9 | 94.3 |\n| CAT-Seg | MMRL+VaMP | **13.9** (+1.1) | **20.3** (+1.6) | **33.3** (+0.9) | **58.6** (+0.7) | **95.2** (+0.9) |\n\nFor open-vocabulary action recognition, we adopted FROSTER's base-to-novel evaluation protocol on two widely used video benchmarks, i.e, Kinetics-400 and UCF-101, where models trained exclusively on base classes are required to generalize to unseen novel action categories. The results are shown below in Table A2:\n\n**Table A2: Performance for open-vocabulary action recognition. B: Base classes, N: Novel classes, HM: Harmonic mean.**\n\n| Method | Prompt Tuning | K400(B) | K400(N) | K400(HM) | UCF(B) | UCF(N) | UCF(HM) |\n|--------|---------------|---------|---------|----------|--------|--------|---------|\n| ViFi-CLIP | - | 76.4 | 61.1 | 67.9 | 92.9 | 67.7 | 78.3 |\n| Open-VCLIP | - | 76.5 | 62.6 | 68.9 | 94.8 | 77.5 | 85.3 |\n| FROSTER | - | 77.8 | 64.3 | 70.4 | 95.3 | 80.0 | 87.0 |\n| FROSTER | MMRL | 78.3 | 64.1 | 70.5 | 95.5 | 80.2 | 87.2 |\n| FROSTER | MMRL+VaMP | **78.8** (+0.5) | **64.8** (+0.7) | **71.1** (+0.6) | **96.1** (+0.6) | **81.0** (+0.8) | **87.9** (+0.7) |\n\nAs shown, VaMP achieves consistent improvements across different tasks, demonstrating the scalability of our approach and its strong generalization capabilities.\n\n[b] CAT-Seg: Cost Aggregation for Open-Vocabulary Semantic Segmentation. CVPR 2024\n\n[c] FROSTER: Frozen CLIP Is A Strong Teacher for Open-Vocabulary Action Recognition. ICLR 2024.\n\n> W3/Q1: Analysis on the trade-off between improved generalization and increased latency.\n\n**R:** We appreciate the reviewer's suggestion and have conducted a more detailed quantitative analysis of the trade-off between inference cost and accuracy under different numbers of Monte Carlo samples S. As shown in Table A3, even with a modest number of samples (e.g., S=5 or S=10), VaMP achieves clear gains in harmonic mean accuracy over the strong MMRL baseline, with only marginal increases in inference time (e.g., 5.3ms/image → 6.1ms/image). Specifically, at S=10, VaMP improves HM from 81.20 to 82.37 with a latency increase of only 0.8ms per image—demonstrating an efficient balance between generalization and overhead. Notably, the performance saturates quickly with increasing S, suggesting that our method remains highly efficient even with limited sampling. This confirms that VaMP can be deployed with low overhead while still leveraging uncertainty modeling for improved generalization.\n\n**Table A3: Comparison of inference time**\n\n| Method | S | Inference Time (ms/img) | Base | Novel | HM |\n|--------|---|------------------------|------|-------|-----|\n| MMRL | - | **5.3** | 85.68 | 77.16 | 81.20 |\n| MMRL+VaMP | 1 | 5.5 | 86.10 | 77.35 | 81.44 |\n| MMRL+VaMP | 5 | 5.8 | 86.37 | 78.26 | 82.03 |\n| MMRL+VaMP | 10 | 6.1 | 86.45 | **78.67** | **82.37** |\n| MMRL+VaMP | 20 | 6.5 | **86.47** | 78.37 | 82.22 |\n| MMRL+VaMP | 50 | 9.3 | 86.43 | 78.15 | 82.01 |\n\n> W4/Q4: Scalability to newer multi-modal models like GPT-4V.\n\n**R:** Our method is architecture-agnostic and can be readily integrated into more complex VLMs, as it only requires access to intermediate transformer layers and the ability to inject prompt tokens. The variational adaptation module is lightweight (MLPs per layer) and does not assume any specific backbone structure. However, due to time and computational resource constraints during the rebuttal period, we were unable to conduct experiments on large-scale GPT-4V like MLLMs, such as QWen-VL and InternVL. GPT-4V is proprietary and we can not apply to it directly. However, technically, it is feasible. We will explore this afterwards and include it in our revision. \n\nDespite these constraints, we have validated VaMP on several state-of-the-art VLMs that represent the latest advances in CLIP-style architectures, including EVA-CLIP [d], SigLIP [e] and SigLIP 2 [f]. These models share similar structural designs with CLIP, enabling rapid adaptation during the rebuttal period. As shown in Table A4, VaMP obtains clear and consistent  performance improvements across all VLMs, demonstrating its strong generalization capability to unseen categories—a critical challenge in prompt learning. The consistent improvements across different VLMs further substantiate the effectiveness and generalizability of our approach to different architectures.\n\n**Table A4: Performance comparison of different VLMs on ViT-B/16 backbone under base-to-novel generalization setting**\n\n| VLM | Method | Base | Novel | H |\n|-----|--------|------|-------|---|\n| CLIP | MMRL | 85.68 | 77.16 | 81.20 |\n| CLIP | MMRL+VaMP | **86.45** (+0.77) | **78.67** (+1.51) | **82.37** (+1.17) |\n| EVA-CLIP | MMRL | 85.97 | 77.69 | 81.62 |\n| EVA-CLIP | MMRL+VaMP | **86.59** (+0.62) | **79.18** (+1.49) | **82.71** (+1.09) |\n| SigLIP | MMRL | 86.12 | 78.21 | 81.97 |\n| SigLIP | MMRL+VaMP | **86.88** (+0.76) | **79.45** (+1.24) | **83.00** (+1.03) |\n| SigLIP 2 | MMRL | 86.64 | 78.97 | 82.62 |\n| SigLIP 2 | MMRL+VaMP | **87.09** (+0.45) | **80.02** (+1.05) | **83.40** (+0.78) |\n\n[d] EVA-CLIP: Improved Training Techniques for CLIP at Scale. Arxiv 2023.\n\n[e] Sigmoid Loss for Language Image Pre-Training. ICCV 2023.\n\n[f] SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features. Arxiv 2025.\n\n> Q2: Put efficiency analysis in the main paper.\n\n**R:** Thanks for the suggestion. We will move it to the main paper in revision.\n\n> Q3: Qualitative examples to illustrate the generated prompts' structure or diversity.\n\n**R:** While we are unable to include image visualizations in the rebuttal due to format constraints, we will provide such examples in the revised manuscript. Specifically, we visualize the learned prompt distributions via t-SNE for different image categories. The results show clear separation across categories, suggesting that VaMP generates semantically distinct prompts conditioned on image content. This inter-class diversity is significantly more pronounced than that of deterministic prompt methods, validating the effectiveness of our uncertainty-aware prompt generation."}, {"review_id": "YTaFRHUyVX", "reviewer": "Reviewer_EaPS", "summary": "This paper proposes VaMP, a novel framework for multi-modal prompt learning with vision-language models. It introduces sample-specific prompt generation conditioned on visual input,  a variational inference formulation where prompt tokens are treated as latent variables, and  a class-aware prior to guide prompt regularization. The method is evaluated under various settings including base-to-novel generalization, cross-dataset generalization, and domain generalization. VaMP outperforms strong baselines like MMRL and PromptSRC across all settings.", "questions": "- The current experiments use a fixed 16-shot setting. Have the authors explored how VaMP performs under more limited (e.g., 1-shot) or more generous supervision?\n\n- Since the posterior over prompt tokens is conditioned only on the image, have the authors considered whether adding textual input (e.g., class name or prompt template) could further inform the distribution? \n\n- In Section 4.4, the authors mention drawing 10 Monte Carlo samples from the posterior to compute the final prediction. Could the authors clarify whether the same set of sampled prompts is shared across all classes for computing similarities, or whether separate prompts are sampled for each class template (e.g., “a photo of a [CLASS]”)?\n\n- In Section 4.3, class prototypes are computed by averaging the posterior means of image embeddings within each class (Eq. 18). Are these computed once before training (e.g., with frozen CLIP features), or are they updated dynamically during training as the posterior evolves?", "limitations": "Yes", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n- The paper is well-motivated, addressing a clear gap in current multi-modal prompt tuning methods by enabling instance-level adaptation and uncertainty modeling. \n\n- The proposed framework is conceptually elegant, formulating prompt tuning as variational inference with a class-aware prior, which is both novel and intuitive. \n\n- The writing is clear and well-structured, making the method easy to understand despite its technical depth. \n\n- The authors provide solid empirical results across a wide range of benchmarks, consistently outperforming strong baselines. The ablation studies are particularly thorough and directly support the value of each design component.\n\nWeaknesses\n\nWhile the paper is novel and interesting, there are a couple of points where a bit more discussion could make the work even more complete. \n\n- First, the class-aware prior is only used during training, and replaced by a standard Gaussian at test time—some reflection on the potential impact of this train-test mismatch would be helpful. \n\n- Second, although the variational approach introduces uncertainty into prompt learning, the paper doesn’t delve into how this uncertainty might be utilized in practice, such as for calibration or confidence-aware decision making, which could be an interesting extension.", "was_revised": true, "final_justification": "Thanks to the authors for the thoughtful rebuttal. I’ve re-read the paper and discussion carefully, and I believe my concerns have been addressed. I keep my original score.", "rebuttal": "We appreciate the reviewer's valuable feedback and recognition of VaMP's clear motivation, conceptual elegance, methodological noveltyand thorough empirical validation across diverse benchmarks. We address the reviewer's comments below carefully.\n\n> W1: The class-aware prior is only used during training, and replaced by a standard Gaussian at test time—some reflection on the potential impact of this train-test mismatch would be helpful.\n\n**R:** We appreciate this suggestion. Indeed, class-aware priors are used only during training to regularize the posterior and encourage semantically structured latent spaces. At test time, we revert to standard Gaussian priors due to the absence of labels. While this introduces a train-test mismatch in the prior, the learned posterior remains image-conditioned and captures instance-specific variations. Empirically, we observe strong generalization on novel classes (Table 1) and unseen datasets (Table 2), indicating that the semantic guidance provided during training improves prompt consistency without overfitting to label-specific priors.\n\n> W2: Second, although the variational approach introduces uncertainty into prompt learning, the paper doesn't delve into how this uncertainty might be utilized in practice, such as for calibration or confidence-aware decision making, which could be an interesting extension.\n\n**R:** Thanks for the suggestion. In VaMP, uncertainty is primarily leveraged to enable diverse, instance-conditioned prompt generation during training and inference. While we do not explicitly use the uncertainty estimates for downstream decision-making, we agree this is a promising direction—e.g., using posterior variance for selective prediction, reweighting, or out-of-distribution detection. We will add a brief discussion on this point in the conclusion to highlight it as a potential avenue for future work.\n\n> Q1: The current experiments use a fixed 16-shot setting. Have the authors explored how VaMP performs under more limited (e.g., 1-shot) or more generous supervision?\n\n**R:** Following the suggestion, we evaluated VaMP under the challenging 1-shot Base-to-Novel Generalization setting across 11 diverse classification datasets. The results are reported in Table A5 below. In this demanding scenario, models are trained on base classes with only a single example per class and must generalize to previously unseen novel classes. As demonstrated in Table A5, VaMP achieves superior performance with the highest harmonic mean (71.23) and novel class accuracy (72.48), substantially outperforming existing methods in this cross-category generalization task. Most notably, compared to the current state-of-the-art MMRL method, VaMP demonstrates remarkable improvements with a +4.05% gain on novel classes (72.48 vs. 68.43) and a +2.11% increase in harmonic mean (71.23 vs. 69.12). This significant performance improvement over the previous state-of-the-art highlights the effectiveness of our variational prompt learning approach, which better captures the uncertainty inherent in prompt distributions when transferring knowledge from base to novel categories, demonstrating robust performance and sample efficiency even under such extremely data-constrained conditions.\n\n**Table A5: 1-shot Base-to-Novel generalization results on 11 datasets with ViT-B/16 backbone.**\n\n| Method | Base | Novel | H |\n|--------|------|-------|---|\n| CoOp | 70.82 | 60.20 | 65.09 |\n| CoCoOp | 70.27 | 65.32 | 67.71 |\n| MaPLe | 69.01 | 67.18 | 68.08 |\n| MMRL | 69.82 | 68.43 | 69.12 |\n| VaMP | **70.02** | **72.48** | **71.23** |\n\n> Q2: Since the posterior over prompt tokens is conditioned only on the image, have the authors considered whether adding textual input (e.g., class name or prompt template) could further inform the distribution?\n\n**R:** Our goal is to generate *image-specific* prompts that adapt to the visual content of each instance, thereby capturing intra-class variability and improving generalization to novel or ambiguous samples. Conditioning on the image is therefore essential for enabling this instance-level adaptation. In contrast, conditioning on textual input such as class names would shift the modeling focus toward class-specific or template-guided prompts, which could reduce the diversity and flexibility of the learned prompts. Notably, the class label is already indirectly incorporated via the prototype-based prior during training, which allows us to guide the posterior distribution without explicitly relying on textual input as a conditioning signal.\n\n> Q3: In Section 4.4, the authors mention drawing 10 Monte Carlo samples from the posterior to compute the final prediction. Could the authors clarify whether the same set of sampled prompts is shared across all classes for computing similarities, or whether separate prompts are sampled for each class template (e.g., \"a photo of a [CLASS]\")?\n\n**R:** The sampled prompts are image-conditioned and shared across all class templates. That is, for a given image, we draw 10 samples from the posterior $q_φ(z|x)$, and compute cosine similarities between the resulting text features (with each class name inserted) and the frozen image embedding. This design ensures efficiency and avoids leakage of label-specific information into the sampling process.\n\n> Q4: In Section 4.3, class prototypes are computed by averaging the posterior means of image embeddings within each class (Eq. 18). Are these computed once before training (e.g., with frozen CLIP features), or are they updated dynamically during training as the posterior evolves?\n\n**R:** The class prototypes are computed once before training using frozen CLIP image features. They remain fixed throughout training and serve as stable semantic anchors for constructing the class-aware prior. This avoids potential instability from dynamic updates and ensures consistency across training iterations. We will clarify this in our revised manuscript."}, {"review_id": "mVqpJMfH6D", "reviewer": "Reviewer_VFp2", "summary": "This paper proposes a variational prompt learning technique designed to overcome the limitations of data-scarce fine-tuning in downstream vision-language tasks (e.g., CLIP). The proposed method enables sample-specific prompt learning by modeling uncertainty through posterior sampling, making the model more adaptable to input content.\nThe key contributions of the paper include:\n1.\tIntroduction of a sample-specific, multi-modal prompt generation strategy.\n2.\tA mechanism for prompt adaptation.\n3.\tA class-aware prior, which integrates both instance-level representations and class prototypes.\nThe method is framed as a variational inference-based prompt adaptation (VMPA) approach, enabling uncertainty-aware learning for vision-language models.", "questions": "Much of the paper is well-written. However, some parts are confusing:\no\tThe role of $\\tilde{z}_{i-1}$ is unclear: is it computed from the residual stream of the image encoder? Is it passed through a trainable layer and concatenated with transformer inputs? \n\n1.\tUnclear implementation: How exactly is $\\tilde{z}_{i-1}$ calculated? What input is used to compute it? The process is vague and could hinder reproducibility.\n2.\tOverlaps with prior work: Please clarify the key technical differences from [24] and [15], beyond what has already been published.\n3.\tLimitations not stated: The paper should explicitly discuss its limitations in the main body, not only in the supplementary material.", "limitations": "The paper shows technical competence and incremental improvements, but lacks novelty, clear explanation of core mechanisms, and sufficient performance gains to justify acceptance.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\no\tThe paper builds upon recent developments in prompt learning and variational inference in the context of vision-language models.\no\tThe proposed architecture appears technically sound and is consistent with methods in prior work (e.g., MaPLe [24], MMRL [15]).\n\nWeaknesses:\no\tThe implementation details are insufficiently explained, particularly regarding the computation of $\\tilde{z}_{i-1}$, which seems critical for reproducibility.\t\no\tThere is no discussion of the method's limitations in the main paper, although it is stated to be present in the supplementary material.\no\tThe performance improvement over the state-of-the-art is marginal, raising concerns about the impact.\no\tOverlap with prior work (notably [15] and [24]) is there, but after the rebuttal I agree that: one key novelty of VaMP lies in the reformulation of prompt learning as a variational inference problem for multi-modal prompt learning. \n\n\nSignificance\n•\tWhile the idea of integrating uncertainty into prompt learning is interesting, the actual improvements are limited and lack justification.\n•\tThe method is only evaluated on a narrow range of models (e.g., CLIP), with no extension to more recent large-scale vision-language models.\n•\tThe contribution feels incremental, as it adapts known techniques with minor modifications.\n \nOriginality\n•\tThe proposed method borrows heavily from prior work:\n   o\tSimilar prompt insertion strategy as MaPLe [24] and MMRL [15].\n   o\tThe variational adaptation idea has been previously explored in the literature.\n•\tThere is no clear justification of why this method is novel or more effective than the prior works.\n•\tThe paper does not sufficiently distinguish itself from existing methods.", "was_revised": true, "final_justification": "I agree with the author's rebuttal and clarification of main concerns to a large degree, mainly: one key novelty of VaMP lies in the reformulation of prompt learning as a variational inference problem for multi-modal prompt learning. I reassessed the paper accordingly.", "rebuttal": "We appreciate the reviewer's valuable feedback and recognition of VaMP's technical soundness. We carefully address the reviewer's comments below.\n\n> W1/Q1: Unclear implementation: How exactly is $z_{i-1}$ calculated? What input is used to compute it? The process is vague and could hinder reproducibility.\n\n**R:** The term $z_{i-1}$ represents the learnable tokens in the visual branch, which we have briefly introduced in L118. Its exact computation depends on the multi-modal prompt learning method. In MaPLe, these visual tokens are generated from the language prompts via a linear transformation implemented with an MLP. Meanwhile, MMRL obtains visual tokens from a shared latent space that is a set of learnable tokens, which are initialized by sampling from a Gaussian distribution, and then uses separate linear projection functions (also implemented with MLPs) to generate modality-specific prompts. We will provide more details in the revision to further clarify and release our code to ensure reproducibility.\n\n> W2/Q3: There is no discussion of the method's limitations in the main paper, although it is stated to be present in the supplementary material.\n\n**R:** Thanks for the suggestion. We will move this discussion into the main paper in the revision.\n\n\n> W3: The performance improvement over the state-of-the-art is marginal, raising concerns about the impact.\n\n**R:** We respectfully disagree with this assessment. VaMP consistently improves upon strong baselines across 11 datasets in both few-shot and cross-dataset generalization settings. In the base-to-novel generalization setting, VaMP outperforms the year-25 state-of-the-art method MMRL by achieving 78.67 (+1.51) on novel classes and 82.37 (+1.17) on the harmonic mean (HM) metric. These gains are non-trivial: they represent **average improvements across 11 diverse datasets**, and the HM metric, computed as **HM = 2 × base_acc × novel_acc / (base_acc + novel_acc)**, is inherently challenging to improve due to its non-linear nature. \n\nFor context, MMRL's improvement over the previous year 2024's SOTA method, MMA, was 0.36 for novel class accuracy and 1.33 for HM, indicating that our results are neither marginal nor negligible. Likewise, in cross-dataset evaluation, the improvement VaMP achieves over MMRL in the source dataset (72.83 vs. 72.03) is comparable to MMRL's improvement over MMA (72.03 vs. 71.00). Moreover, on the target dataset, VaMP's **average improvement over MMRL across 10 datasets (67.74 vs. 67.25)** parallels MMRL's gain over MMA (67.25 vs. 66.61).\n\nBeyond accuracy, VaMP introduces a principled framework for sample-specific, uncertainty-aware adaptation, which offers broader benefits beyond raw performance, as also recognized by other reviewers. We believe these results and contributions together underscore the superiority of the method.\n\n> W4/Q2: Overlaps with prior work. Please clarify the key technical differences from [24] and [15], beyond what has already been published. \n\n**R:** We respectfully but firmly disagree that our contributions merely overlap with prior work. Firstly, MMRL [15] and MaPLe [24] serve as baseline methods upon which our approach builds, as stated in the preliminary section (Lines 108-109); we do not claim any of them as our own contributions. Second, one key novelty of VaMP lies in the reformulation of prompt learning as a variational inference problem for multi-modal prompt learning (as also recognized by other reviewers). Through this formulation, VaMP enables sample-specific and token-level uncertainty modeling, a capability not offered by any existing methods. We further introduce a class-aware prior derived from semantic prototypes to guide adaptation and enhance generalization. To the best of our knowledge, there is no existing methods providing such a novel framework containing these ideas for multi-modal prompt learning.\n\nMoreover, the originality and rigour of our framework are also widely recognized by other reviewers. Reviewer hikd noted that it represents “a novel framework”, Reviewer EaPS described it as “well-motivated, addressing a clear gap in current multi-modal prompt tuning methods”, “conceptually elegant”, “novel and intuitive,” and Reviewer R47nX confirmed that “the mathematical formulations are well-defined, and the reasoning is sound.” We believe these endorsements underscore the conceptual and technical novelty and depth of VaMP.\n\n> W5: The method is only evaluated on a narrow range of models (e.g., CLIP), with no extension to more recent large-scale vision-language models.\n\n**R:** Our method is architecture-agnostic and can be easily integrated into more complex Vision-Language Models (VLMs), as it only requires access to intermediate transformer layers and the ability to inject prompt tokens. The variational adaptation module is lightweight (MLPs per layer) and does not assume any specific backbone structure. \nFollowing this suggestion, we have validated VaMP on several state-of-the-art VLMs that represent the latest advances in CLIP-style architectures, including EVA-CLIP [a], SigLIP [b], and SigLIP 2 [c]. These models share structural similarities with CLIP, allowing quick adaptation within the rebuttal timeframe. As shown in Table A6, VaMP consistently improves performance across all tested VLMs. Notably, these improvements on novel classes underscore VaMP's strong generalization capability, which is an essential factor in prompt learning. The consistent gains across these diverse architectures highlight the effectiveness and generalizability of VaMP.\n\n**Table A6: Performance comparison of different methods on ViT-B/16 backbone for base, novel classes, and harmonic mean (H).**\n\n| VLM | Method | Base | Novel | H |\n|-----|--------|------|-------|---|\n| CLIP | MMRL | 85.68 | 77.16 | 81.20 |\n| CLIP | MMRL+VaMP | **86.45** (+0.77) | **78.67** (+1.51) | **82.37** (+1.17) |\n| EVA-CLIP | MMRL | 85.97 | 77.69 | 81.62 |\n| EVA-CLIP | MMRL+VaMP | **86.59** (+0.62) | **79.18** (+1.49) | **82.71** (+1.09) |\n| SigLIP | MMRL | 86.12 | 78.21 | 81.97 |\n| SigLIP | MMRL+VaMP | **86.88** (+0.76) | **79.45** (+1.24) | **83.00** (+1.03) |\n| SigLIP 2 | MMRL | 86.64 | 78.97 | 82.62 |\n| SigLIP 2 | MMRL+VaMP | **87.09** (+0.45) | **80.02** (+1.05) | **83.40** (+0.78) |\n\nFurthermore, we have also applied our method to different tasks and different models, as demonstrated in Table A1 and Table A2 in our reply to Reviewer hikd. The consistent improvements observed across diverse tasks and models clearly illustrate the generalization capability and versatility of our approach, and we hope the reviewer can re-consider the benefits of VaMP.\n\nAdditionally, our method is also technically feasible to be integrated with MLLMs, such as QWen-VL and InternVL or other GPT-4V like models. Due to limited time and computational resources during the rebuttal period, we could not conduct additional experiments on such models, but we are confident on outcome given all our experiments executed on all VLMs so far.\n\n[a] EVA-CLIP: Improved Training Techniques for CLIP at Scale. Arxiv 2023.\n\n[b] Sigmoid Loss for Language Image Pre-Training. ICCV 2023.\n\n[c] SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features. Arxiv 2025."}, {"review_id": "OPkSmvpJXN", "reviewer": "Reviewer_47nX", "summary": "This paper transforms deterministic multimodal prompts into statistical ones to capture instance-level variations and model uncertainty, achieving performance that surpasses state-of-the-art methods across multiple settings and benchmarks.", "questions": "Please refer to the Weaknesses section above.", "limitations": "yes", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1.The writing is clear and easy to follow.\n\n2.The mathematical formulations are well-defined, and the reasoning is sound.\n\n3.The method demonstrates superior performance, surpassing state-of-the-art prompting strategies.\n\nWeaknesses:\n\n1.I feel that uncertainty-based modeling may increase computational overhead. Could the authors compare the models and computational complexity of deterministic and uncertainty-based prompting strategies?\n\n2.The current benchmarks and settings are relatively simple. Can the proposed method be extended to dense prediction or video-based benchmarks?\n\n3.There are many strategies for uncertainty modeling, such as Monte Carlo sampling and evidence learning. Why did the authors choose to model the Gaussian mean and variance using MLPs?\n\n4.According to Figure 1, the prompts only operate at the encoding stage. Why not consider integrating them at the decoding stage as well?", "was_revised": true, "final_justification": "I have carefully reviewed comments and considered the feedback from the other reviewers, especially Reviewer VFp2. I remain concerned about the novelty of this work. Furthermore, after a deeper investigation into recent advances in this field, I found that the core idea of this work appears to be quite similar to that of [1]. \n\n[1]. Variational prompt tuning improves generalization of vision-language models, 2022", "rebuttal": "We appreciate the reviewer's valuable feedback and recognition of VaMP's clear presentation, well-defined mathematical formulations, sound reasoning, and superior performance across multiple benchmarks. We carefully address the reviewer's comments below.\n\n> W1: I feel that uncertainty-based modeling may increase computational overhead. Could the authors compare the models and computational complexity of deterministic and uncertainty-based prompting strategies?\n\n**R:** We thank the reviewer for raising this important practical consideration. As mentioned in Section S3 of our supplementary material, we have provided a quantitative analysis of computational efficiency. Here, we further provide a more detailed quantitative analysis of the trade-off between inference cost and accuracy under different numbers of Monte Carlo samples S. As shown in Table A7, even with a modest number of samples (e.g., S=5 or S=10), VaMP achieves clear gains in harmonic mean accuracy over the strong MMRL baseline, with only marginal increases in inference time (e.g., 5.3ms/image → 6.1ms/image). Specifically, at S=10, VaMP improves HM from 81.20 to 82.37 with a latency increase of only 0.8ms per image—demonstrating an efficient balance between generalization and overhead. Notably, the performance saturates quickly with increasing S, suggesting that our method remains highly efficient even with limited sampling. This confirms that VaMP can be deployed with low overhead while still leveraging uncertainty modeling for improved generalization.\n\n**Table A7: Comparison of inference cost and parameter count.**\n\n| Method | S | Inference Time (ms/img) | Base | Novel | HM |\n|--------|---|------------------------|------|-------|-----|\n| MMRL | - | **5.3** | 85.68 | 77.16 | 81.20 |\n| MMRL+VaMP | 1 | 5.5 | 86.10 | 77.35 | 81.44 |\n| MMRL+VaMP | 5 | 5.8 | 86.37 | 78.26 | 82.03 |\n| MMRL+VaMP | 10 | 6.1 | 86.45 | **78.67** | **82.37** |\n| MMRL+VaMP | 20 | 6.5 | **86.47** | 78.37 | 82.22 |\n| MMRL+VaMP | 50 | 9.3 | 86.43 | 78.15 | 82.01 |\n\n> W2: The current benchmarks and settings are relatively simple. Can the proposed method be extended to dense prediction or video-based benchmarks?\n\n**R:** Thank you for this valuable suggestion. We evaluated VaMP's generalizability beyond classification by conducting experiments on open-vocabulary segmentation and action recognition tasks. We integrated VaMP into state-of-the-art frameworks of two tasks: CAT-Seg [a] for segmentation and FROSTER [b] for action recognition, utilizing CLIP ViT-B/16 as the backbone encoder. Both baselines are extensions built upon CLIP, allowing our method to be seamlessly integrated into these approaches. \n\nFor open-vocabulary segmentation, we followed CAT-Seg's protocol, training on COCO-Stuff (118K images, 171 categories) and evaluating across multiple benchmarks including ADE20K, PASCAL-Context, and PASCAL VOC with varying category scales (59-847 classes). The results are shown in Table A8:\n\n**Table A8: Performance comparison of different methods on ViT-B/16 backbone for open-vocabulary segmentation.**\n\n| Method | Prompt Tuning | ADE-847 | PC-459 | ADE-150 | PC-59 | VOC-20 |\n|--------|---------------|---------|--------|---------|-------|--------|\n| ZegFormer | - | 5.6 | 10.4 | 18.0 | 45.5 | 89.5 |\n| OVSeg | - | 7.1 | 11.0 | 24.8 | 53.3 | 92.6 |\n| SAN | - | 10.1 | 12.6 | 27.5 | 53.8 | 94.0 |\n| CAT-Seg | - | 12.0 | 19.0 | 31.8 | 57.5 | 94.6 |\n| CAT-Seg | MMRL | 12.8 | 18.7 | 32.4 | 57.9 | 94.3 |\n| CAT-Seg | MMRL+VaMP | **13.9** (+1.1) | **20.3** (+1.6) | **33.3** (+0.9) | **58.6** (+0.7) | **95.2** (+0.9) |\n\nFor open-vocabulary action recognition, we adopted FROSTER's base-to-novel evaluation protocol on Kinetics-400 and UCF-101, where models trained exclusively on base classes are required to generalize to unseen novel action categories. The results are shown in Table A9:\n\n**Table A9: Performance comparison under base-to-novel setting on two widely-used video benchmarks, i.e., Kinetics-400 and UCF-101. B: Base classes, N: Novel classes, HM: Harmonic mean.**\n\n| Method | Prompt Tuning | K400(B) | K400(N) | K400(HM) | UCF(B) | UCF(N) | UCF(HM) |\n|--------|---------------|---------|---------|----------|--------|--------|---------|\n| ViFi-CLIP | - | 76.4 | 61.1 | 67.9 | 92.9 | 67.7 | 78.3 |\n| Open-VCLIP | - | 76.5 | 62.6 | 68.9 | 94.8 | 77.5 | 85.3 |\n| FROSTER | - | 77.8 | 64.3 | 70.4 | 95.3 | 80.0 | 87.0 |\n| FROSTER | MMRL | 78.3 | 64.1 | 70.5 | 95.5 | 80.2 | 87.2 |\n| FROSTER | MMRL+VaMP | **78.8** (+0.5) | **64.8** (+0.7) | **71.1** (+0.6) | **96.1** (+0.6) | **81.0** (+0.8) | **87.9** (+0.7) |\n\nAs shown, VaMP achieves consistent improvements across different tasks, demonstrating the scalability of our approach and its strong generalization capabilities.\n\n[a] CAT-Seg: Cost Aggregation for Open-Vocabulary Semantic Segmentation. CVPR 2024\n\n[b] FROSTER: Frozen CLIP Is A Strong Teacher for Open-Vocabulary Action Recognition. ICLR 2024.\n\n> W3: There are many strategies for uncertainty modeling, such as Monte Carlo sampling and evidence learning. Why did the authors choose to model the Gaussian mean and variance using MLPs?\n\n**R:** We adopt MLPs to model the posterior mean and variance due to their simplicity, efficiency, and compatibility with variational inference. This design choice follows the well-established principles of Variational Autoencoders (VAEs)[c], which have been widely adopted in the machine learning community for modeling uncertainty. Similar to VAEs, our approach allows lightweight, layer-wise modeling of uncertainty and integrates naturally with the reparameterization trick for stable end-to-end training. Compared to alternatives like evidence learning, our approach does not require additional supervision or calibration loss, and directly optimizes downstream performance. Empirically, Table 4(b) in our main manuscript shows consistent improvements over deterministic prompts, validating our choice. Exploring richer uncertainty modeling strategies remains a promising direction for future work.\n\n[c] Auto-Encoding Variational Bayes. ICLR 2014.\n\n> W4: According to Figure 1, the prompts only operate at the encoding stage. Why not consider integrating them at the decoding stage as well?\n\n**R:**  We would like to clarify that, our method builds on CLIP-like architectures following previous methods [79, 80, 24, 71, 11, 66, 15], which employ a dual encoder architecture—a vision encoder and a text encoder—without a decoding stage. As such, there is no separate decoder module where prompts could be inserted. All computations are encoder-based, and predictions are made via similarity in the joint embedding space. Our prompt adaptation targets the transformer layers within the text encoder, which directly influence the representation used for image-text matching. For specific models or applications when the decoders are needed, our prompt design can be equally applicable."}]

Author rebuttal:

Meta-review: The authors have thoroughly addressed all reviewer concerns through extensive additional experiments and detailed clarifications. They demonstrated VaMP’s strong generalization across multiple tasks (including segmentation and action recognition), quantified the computational trade-offs of variational sampling, and validated the method on recent VLMs (EVA-CLIP, SigLIP, SigLIP 2). The rebuttal effectively clarified the novelty relative to prior variational prompt methods by emphasizing multi-modal, token-level, and class-aware design differences. While one reviewer remained skeptical about originality, the majority recognized the conceptual and empirical contributions, supported by consistent SOTA results and a well-motivated variational framework. The paper is technically sound, well-evaluated, and makes a valuable contribution to multi-modal prompt learning.
Decision: Accept (poster)