Understanding and Mitigating Miscalibration in Prompt Tuning for Vision-Language Models

Authors: Shuoyuan Wang, Yixuan Li, Hongxin Wei
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: Vision-Language Models, Confidence calibration, Outlier Regularization, Prompt Tuning

Abstract:
Confidence calibration is critical for the safe deployment of machine learning models in the real world.
However, such issue in vision-language models like CLIP, particularly after fine-tuning, has not been fully addressed.
In this work, we demonstrate that existing prompt tuning methods usually lead to a trade-off of calibration between base and new classes:
the cross-entropy loss in CoOp causes overconfidence in new classes by increasing textual label divergence, whereas the regularization of KgCoOp maintains the confidence level but results in underconfidence in base classes due to the improved accuracy.
Inspired by the observations, we introduce Dynamic Outlier Regularization (DOR) to ensure the confidence calibration on both base and new classes after fine-tuning. 
In particular, we propose to minimize the feature deviation of novel textual labels (instead of base classes) sampled from a large vocabulary.
In effect, DOR prevents the increase in textual divergence for new labels while easing restrictions on base classes.
Extensive experiments demonstrate that DOR can enhance the calibration performance of current fine-tuning methods on base and new classes.

Decision: reject
Reviewer scores: 6, 3, 6, 6

Reviews and discussion:
<thread>
creation date:
1730448447673

thread_type:
official_review

<thread_object>
creator: reviewer_wlz3

<content>
summary:
This paper investigates the calibration performance of CLIP following fine-tuning. The author observes a trade-off in calibration between base and new classes and proposes a method called Dynamic Outlier Regularization (DOR). DOR samples categories unrelated to the base class from a large vocabulary to minimize the feature deviation of novel textual labels. Empirical results demonstrate that this approach outperforms the standard fine-tuning method across various settings.

strengths:
1. The author conducted robust experiments that demonstrate how prompt fine-tuning prior to happiness can lead to a decline in the model's calibration performance.

2. The paper introduces an efficient normalization method designed to enhance the calibration performance of both base and novel classes.

3. The paper provides performance results across multiple calibration evaluation metrics and a range of experimental settings.

4. The writing is clear and well-structured, making it easy to read.

weaknesses:
1. Previous works [1,2] have examined the calibration performance of pre-trained CLIP after fine-tuning. However, your paper lacks experimental results comparing your method with these studies. We recommend that you include such comparisons in your work.
2. Given that your method is based on experimental observations from CoOp and KgCoOp, we have concerns about its generalizability. For example, in Table 1, your method underperforms compared to Vanilla TCP in half of the settings.
3. This article primarily selects outliers from WordNet. We are curious whether using different lexical databases significantly affects the results.

If you can include relevant experiments and address my questions, I will consider increasing the score.

[1] Enabling Calibration In The Zero-Shot Inference of Large Vision-Language Models.

[2] Towards Calibrated Robust Fine-Tuning of Vision-Language Models.

questions:
See above.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730530585647

thread_type:
official_review

<thread_object>
creator: reviewer_5mjq

<content>
summary:
This paper introduces Dynamic Outlier Regularization (DOR), a method to improve confidence calibration in fine-tuned VLMs by controlling textual feature divergence (also can be extended to visual tuning) through the use of selected outliers, thereby enhancing model reliability on both base and novel classes.

strengths:
1. DOR integrates smoothly with popular prompt-tuning methods (e.g., CoOp, KgCoOp) without requiring major architectural changes, making it easy to adopt in existing pipelines. 
2. The authors provided insightful analysis of how current prompt-tuning methods impact confidence calibration, with clear explanations for why overconfidence or underconfidence arises in certain settings. The motivation is clear.
3. The manuscript is well-written and in a good logic.

weaknesses:
1. The ablation study is insufficient, e.g. how sensitive DOR is to the choice of outliers and whether different selection strategies could yield better results?
2.  The experiments are largely limited to standard benchmarks without applying the method to domain-specific tasks (e.g., medical imaging or autonomous systems as the authors mentioned in the Introduction), where calibration is especially critical.

questions:
1. When you calculate the semantic similarity between textual labels in WordNet and the base classes, do you use cosine similarity? Please Clarify. Would using a different metric (e.g. Euclidean distance) impact results?
2. The outliers​ are selected based on the top-K, but there’s no mention of a specific similarity threshold. Would setting a threshold affect performance?
3. How frequently are the outliers updated during training? Does the frequency affect DOR’s calibration performance?
4. In page 8, line 407-408, is it a typo (should be "CoCoOp" ranther than "CoOp") or a wrong statement (the number is wrong if you campred to the zero-shot CLIP)? Please correct it.
5. What criteria were used to select visual outliers from ImageNet-1K? How to ensure these outliers are sufficiently distinct from base classes without introducing irrelevant noise?
6.  How does DOR influence the feature space of base classes when incorporating visual outliers?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730543536452

thread_type:
official_review

<thread_object>
creator: reviewer_pueg

<content>
summary:
This paper proposes Dynamic Outlier Regularization (DOR) to improve confidence calibration in prompt-tuned vision-language models (VLMs), particularly in CLIP. The authors argue that current prompt-tuning methods lead to miscalibration by creating a trade-off between base and new classes, with CoOp overconfident on new classes and KgCoOp underconfident on base classes. DOR aims to resolve this trade-off by introducing textual outliers to regularize model predictions, promoting consistent calibration across both base and new classes.

strengths:
- The paper identifies a relevant issue in the domain of prompt tuning and confidence calibration in VLMs, an area of growing importance.
- The authors introduce a novel idea of using dynamically sampled textual outliers to address calibration inconsistencies, and the approach shows effectiveness across various datasets.
- DOR’s flexibility in working with multiple prompt-tuning methods is a potential advantage.

weaknesses:
- The paper does not include comparisons with several recent and closely related methods, such as CoPrompt and PromptSRC, which also address calibration issues and trade-offs between base and new classes. Without these comparisons, it is unclear whether DOR provides any substantial advantage over the state of the art, especially since these methods were specifically designed to tackle the same calibration challenges.

- The primary claim—that prompt-tuning methods like CoOp and KgCoOp introduce calibration trade-offs between base and new classes—has already been extensively studied in prior works. For instance, CoPrompt effectively handles these issues and includes mechanisms specifically designed to manage calibration across both class types. As such, the problem statement lacks novelty, and the paper provides insufficient rationale for why DOR would be preferable to these existing methods.


- While the paper offers some empirical evidence for DOR’s effectiveness, it lacks analysis that explains why the use of textual outliers should systematically address calibration trade-offs. 

- The proposed solution, while conceptually interesting, lacks practical guidelines on how to effectively select and implement outliers in a real-world setting. Given that the efficacy of DOR relies on appropriate textual outlier selection, more detailed criteria or algorithms for selecting these outliers would be necessary for practitioners to adopt this method.

- Interestingly, some of the latest methods show less improvement with the proposed solution compared to some of the earlier methods like CoOp. This indicates that the latest methods are already capable of handling the problem and don't require such a solution proposed in this paper. Again, the paper lacks a comparison to the latest method, making it difficult to understand if it has any usage.

questions:
See the weakness section.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730618945721

thread_type:
official_review

<thread_object>
creator: reviewer_2y4k

<content>
summary:
This paper, through CoOp and KgCoOp, observe that when a model undergoes adaptation via prompt tuning, CoOp tends to be overconfident in novel classes, while KgCoOp, on the contrary, becomes underconfident. To address this tendency towards overconfidence/underconfidence, this paper propose a Dynamic Outlier Regularizer (DOR) term, which demonstrates performance improvements when the proposed regularizer is added to the algrorithms for prompt learning of VLMs.

strengths:
1. A new blind point was identified (CoOp and KgCoOp becomes over/under confident)
2. Experimental results show performance improvement when the proposed regularizer is added.

weaknesses:
1. The motivation was inferred from CoOp and KgCoOp, but in fact, these two algorithms lack adequate consideration for novel classes. For an effective analysis of this phenomenon, algorithms that directly account for novel classes should be utilized, such as CoCoOp, MaPLe, PromptSRC, DEPT, and TCP. If similar tendencies are observed in these algorithms, it would strongly support the authors' claim.

2. It appears that further explanation is needed regarding the concept of "Texture divergence." From my understanding, this divergence is due to the diversity of textual representation arising from CoOp's prompt learning method. However, finding concrete evidence to confirm this explanation is challenging. Additionally, a more detailed explanation of the notation, particularly the keywords emphasized in the paper, would enhance reader's understanding if the study.

3. More comparisons with other algorithms are necessary. Given the large number of prompt learning algorithms, further experimental comparisons are needed to confirm whether the effect of this regularizer is generalizable. (Similar with the first weakness statement)

questions:
Please refer to the weakness part.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732180790515

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General Response

comment:
We thank all the reviewers for their valuable comments and insightful suggestions.
We are glad that reviewers find this work focuses on an **relevant** and **important issue** (pueg), which is a **new blind point** (2Y4k) in vision-language models. We are also encouraged that reviewers appreciate that our motivation is **clear** (wLz3) with **insightful analysis** (5mjQ),  and the proposed DOR is **flexible** (pueg) on existing prompt tuning methods.  Besides, reviewers recognize that the writing is **well-structured** (5mjQ, wLz3) and  **in good logic** (5mjQ).

In the following responses, we provide point-by-point responses to all reviewers’ comments and concerns. We list the main concerns of the reviewers and the changes $^1$  below:

- Added experiments for DOR on more latest prompt tuning methods in **Table 1-2** and **Appendix F**. [2Y4k, pueg]
- Added experiments for textual divergence on more methods in **Appendix B.1**. [2Y4k]
- Added explanation for DOR's contribution and motivation in **Appendix B.2**. [pueg]
- Added ablations on DOR including selection strategy [pueg, 5mjQ], similarity metric [5mjQ], outlier numbers [5mjQ], update frequency [5mjQ], and outliers databases [wLz3] in **Table 4 and Appendix G**.
- Added experiments on domain-specific tasks in **Appendix H**. [5mjQ]
- Added experiments on existing calibration for CLIP in **Appendix I**. [wLz3]
- Added explanation on DOR-V in **Line 510-513** and **Appendix J**. [5mjQ]
- Clarified the calibration evaluation in experiments in **Line 313**. [wLz3]
- Fixed typo in **Line 406**. [5mjQ]


We sincerely hope the following responses can address the concerns. We look forward to your response and are willing to provide clarifications to any concerns you may have.

 $^1$ For clarity, we highlight the revised part of the manuscript in blue color.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732294195355

thread_type:
official_comment

<thread_object>
creator: area_chair_mhot

<content>
title:
Interactive Discussions

comment:
Dear Reviewers,

Thank you for your efforts in reviewing this paper. We highly encourage you to participate in interactive discussions with the authors before November 26, fostering a more dynamic exchange of ideas rather than a one-sided rebuttal.

Please feel free to share your thoughts and engage with the authors at your earliest convenience.

Thank you for your collaboration.

Best regards,
ICLR 2025 Area Chair
</content>
</thread_object>

</thread>


Metareview:
metareview:
This submission introduces Dynamic Outlier Regularization (DOR) to enhance confidence calibration in prompt-tuned vision-language models (VLMs), with a specific focus on CLIP. DOR primarily leverages textual outliers from WordNet to regularize model predictions, aiming to ensure consistent calibration across both base and novel classes. However, after reviewing the feedback and author rebuttals, the area chair is inclined to reject the paper, as the experimental improvements are marginal and the integration of WordNet-derived outliers may not be practical for real-world applications.