Reflexive Guidance: Improving OoDD in Vision-Language Models via Self-Guided Image-Adaptive Concept Generation

Authors: Jihyo Kim, Seulbi Lee, Sangheum Hwang
Primary area: foundation or frontier models, including LLMs
Keywords: Out-of-distribution detection, Trustworthiness, Vision-language models, Foundation models

Abstract:
With the recent emergence of foundation models trained on internet-scale data and demonstrating remarkable generalization capabilities, such foundation models have become more widely adopted, leading to an expanding range of application domains. Despite this rapid proliferation, the trustworthiness of foundation models remains underexplored. Specifically, the out-of-distribution detection (OoDD) capabilities of large vision-language models (LVLMs), such as GPT-4o, which are trained on massive multi-modal data, have not been sufficiently addressed. The disparity between their demonstrated potential and practical reliability raises concerns regarding the safe and trustworthy deployment of foundation models. To address this gap, we evaluate and analyze the OoDD capabilities of various proprietary and open-source LVLMs. Our investigation contributes to a better understanding of how these foundation models represent confidence scores through their generated natural language responses. Furthermore, we propose a self-guided prompting approach, termed Reflexive Guidance (ReGuide), aimed at enhancing the OoDD capability of LVLMs by leveraging self-generated image-adaptive concept suggestions. Experimental results demonstrate that our ReGuide enhances the performance of current LVLMs in both image classification and OoDD tasks. The lists of sampled images, along with the prompts and responses for each sample are available at https://github.com/daintlab/ReGuide.

Decision: accept (poster)
Reviewer scores: 3, 6, 6, 6

Reviews and discussion:
<thread>
creation date:
1730115729537

thread_type:
official_review

<thread_object>
creator: reviewer_wywf

<content>
summary:
This paper explores to enhance out-of-distribution detection (OoDD) in large vision-language models (LVLMs). The authors emphasize the increasing importance of these models in various domains and highlight the challenges related to their trustworthiness, specifically in dealing with out-of-distribution (OoD) inputs. To address this, they propose a self-guided prompting approach called Reflexive Guidance (ReGuide), which leverages LVLMs’ visual interpretation capabilities to improve OoD detection by generating image-adaptive concept suggestions. The experiments demonstrate that ReGuide significantly boosts the performance of both open-source and proprietary models in OoD detection tasks.

strengths:
1. The introduction of the ReGuide method is interesting. By using image-adaptive concept generation, the authors provide a unique solution to enhance OoD detection in LVLMs. This approach leverages the strong visual recognition capabilities of these models, offering a novel way to bridge gaps in OoD detectability.

2. Comprehensive Evaluation: This paper comprehensively evaluates both proprietary and open-source LVLMs, providing a balanced comparison across different models. This may be helpful for future research.

3. Detailed Analysis: The authors provide a well-rounded analysis, including experimental results, visual feature interpretation, and the effectiveness of ReGuide on near- and far-OoD datasets. This depth of analysis gives credibility to their findings and demonstrates the robustness of their proposed method. Furthermore, it demonstrates the potential of introducing LVLMs to the area,

weaknesses:
1. Limited Explanation of Failures: Although the paper provides examples of failure cases, it does not fully delve into the underlying causes or offer detailed solutions for these issues. While the authors acknowledge the problem of overconfidence in models such as GPT-4o with ReGuide, further exploration of these limitations could strengthen the study. There is still not a robust method to effectively introduce LVLMs to the OoD tasks.

2. Model-Specific Insights: The paper focuses on generic findings across models, but a deeper investigation into how specific models (e.g., GPT-4o vs. InternVL2) behave differently when ReGuide is applied could add nuance to the conclusions. For example, the differences in false positive rates (FPR) between models with and without ReGuide should be presented for a better comparison. 

3. Scalability and Practicality: While the ReGuide method shows a promising direction, the computational overhead and API limitations mentioned in the paper could present challenges for practical, large-scale implementation. This issue is touched upon but not sufficiently addressed in terms of how ReGuide might be optimized for deployment at scale. Meanwhile, the inference cost analysis can further improve the paper's quality and inspire further work.

questions:
N/A
</content>
</thread_object>

</thread>
<thread>
creation date:
1730630647594

thread_type:
official_review

<thread_object>
creator: reviewer_bdm6

<content>
summary:
Overall this is a solid paper. This paper addresses the underexplored trustworthiness of foundation models, particularly focusing on the out-of-distribution detection (OoDD) capabilities of large vision-language models (LVLMs). The authors highlight the gap between the demonstrated potential and practical reliability of LVLMs, raising concerns about their safe deployment. To address this, the researchers evaluate and analyze the OoDD capabilities of various proprietary and open-source LVLMs. They propose a novel approach called Reflexive Guidance (ReGuide), which aims to enhance the OoDD capability of LVLMs by using self-generated image-adaptive concept suggestions. Experimental results show that ReGuide improves the performance of current LVLMs in both image classification and OoDD tasks.

strengths:
[Promising research topic] This paper focuses on an under-explored topic, the reliability of large vision-language models. [Benchmark contribution] This paper define the concept of Out-of-Distribution Detection on LVLMs, and conduct extensive experiments to build a benchmark and give insightful analysis. [Method contribution] Furthermore, they propose a method ReGuide on the prompt to boost the performance of LVLMs. They verify the effectiveness of the proposed method on GPT-4o and InternVL.

weaknesses:
Since the out-of-distribution detection on vision-language foundation models is a new definition proposed in this paper, I would suggest to add a demonstration figure to show this concept in a visually straightforward way. You can compare the conventional OOD, OoDD on CLIP and OoDD on LVLMs.

questions:
Please see Weaknesses.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730689020395

thread_type:
official_review

<thread_object>
creator: reviewer_kmc5

<content>
summary:
This work proposes a self-guided prompting approach, termed Reflexive Guidance (ReGuide), aimed at enhancing the OoDD capability of LVLMs by leveraging self-generated image-adaptive concept suggestions. Experimental results demonstrate that our ReGuide enhances the performance of current LVLMs in both image classification and OoDD tasks

strengths:
Strengths:
•This paper represents the first effort to extend zero-shot Out-of-Distribution Detection (OoDD) to generative Large Vision-Language Models (LVLMs) in reference to CLIP. Building upon this foundation, the authors define the task and propose corresponding evaluation methodologies, which offer valuable insights for future research；
•The paper evaluates and analyzes the OoDD capabilities of various proprietary and open-source LVLMs. Furthermore, the authors propose a two-stage self-guided prompting approach, termed Reflexive Guidance (ReGuide), to unlock the potential of LVLMs. Experimental results demonstrate that the ReGuide enhances the performance of LVLMs in both image classification and OoDD tasks.

weaknesses:
Weaknesses:
•The paper introduces new tasks and evaluation metrics. However, certain details in this regard are somewhat lacking. For instance, the paper mentions using only 25% of the benchmark subsets but does not specify which portions are utilized. It would be preferable to provide a list of these subsets to facilitate the replication and comparison of subsequent work. Additionally, other similar details need to be verified.

•In the experimental evaluation, the ‘valid ratio’ varies across different methods. And only valid responses are included when measuring performance, which results in inconsistencies in the samples used to calculate metrics for each method. This may not be justifiable. Have you considered adopting a more consistent evaluation approach?

•The novelty of this paper is somewhat limited.  Firstly, it introduces the OoDD task for LVLMs; however, the rationale behind this task remains debatable. Considering the robust capabilities of LVLMs and their ongoing advancements, LVLMs are anticipated to directly recognize objects within images. In comparison, the method proposed in this paper is overly complex and excessively reliant on prompts. Secondly, the approach presented in the paper appears to be more akin to prompt engineering, lacking substantial innovative contributions.

questions:
see weakness
</content>
</thread_object>

</thread>
<thread>
creation date:
1730715815155

thread_type:
official_review

<thread_object>
creator: reviewer_mv3d

<content>
summary:
The goal for the paper is to evaluate the out-of-distribution detection (OoDD) capabilities of LLMs (some private and open models)

strengths:
- easy to read
- easy to duplicate

weaknesses:
- motivation for the work
- limited impacts
- extending from a prior work in text-based method.

questions:
- Why we need to evaluate the capabilities 
- as LLM gets huge weekly, the measurement still makes sense?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732258716793

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Response to all reviewers

comment:
We sincerely thank the reviewers for their thoughtful and constructive feedback, which has significantly contributed to improving the quality and clarity of our work. In response to the comments, we have made several updates and carefully addressed all points raised.

To support and facilitate future research, we have uploaded the data lists sampled for the experiments, along with the prompt-response pairs obtained from the main experiments. These resources are available at https://drive.google.com/file/d/1x-0PcPrYlZ0biKALOMg3nN6cCIxRXmEu/view?usp=sharing. **They will be made publicly available in a GitHub repository after the final decision regarding the manuscript**.


Regarding the term "overconfidence" used in the original manuscript, we recognize that it may convey an unintended negative meaning. To avoid misunderstandings, we have replaced this term throughout the paper with "model behavior predicting with extremely high confidence," which more accurately reflects our intended meaning.

During the review process, we identified and corrected a few minor errors in the manuscript. For example, the valid ratio of InternVL2-76B in Table 4 was updated from 99.63 to 97.26, and the OoD input ratio of GPT-4o+ReGuide in Table 5 was corrected from 85.97 to 95.97. We initially used an overly strict method for AUROC and FPR@95%TPR, prioritizing OoD inputs lower than ID inputs with the same score, which could lead to misinterpretation. To improve clarity, we recalculated these metrics using a standard interpolation method commonly used in libraries like scikit-learn. These corrections are reflected in the corresponding discussions, but they do not impact the overall results or conclusions of the paper.

Below, we provide a detailed list of the sections that have been updated in response to the comments:
- Sec. 1: Clear motivation and insights regarding our study
- Sec. 3.1: Comparison of the OoDD framework for single-modal image classifiers, CLIP, and LVLMs including a figure and the corresponding description (reviewer BDm6)
- Sec. 4.1: Detailed analysis on degraded FPR@95%TPR for large models (i.e., InternVL2-76B and GPT-4o) and addition FPR@90%TPR to Table 4 (reviewer wyWF)
- Appendix B
  - B.1 Details of the dataset, including the number of samples and their distributions (reviewer kMC5)
  - B.2 Performance on the shared valid query set for a more rigorous comparison on the 25% ImageNet200 benchmark (reviewer kMC5)
  - B.5 Ablation results on the effects of guidelines and examples for mitigating failure cases (reviewer kMC5)
- Appendix C
  - C.2 Performance on the shared valid query set for a more rigorous comparison on the 5% ImageNet200 benchmark (reviewer kMC5)
  - C.3 Detailed analysis on degraded FPR@95%TPR with ROC curves, examples at the two extremes of the ROC curve, and OoD score distributions (reviewer wyWF)
  - C.6 Inference cost analysis including computational complexity and API costs (reviewer wyWF)

Please refer to the updated manuscript. We hope our responses sufficiently address all your concerns. If you have any additional concerns or require further clarification, please do not hesitate to let us know. We would be happy to address any questions.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper aims at improving the OoDD capability of large language models by adopting a self-guided prompting approach, which is somewhat solid.

Reviewers propose many concerns, such as the defination of this new task, the details of experimental evaluations, and these concerns are well addressed during th rebuttal period. The authors are required to update/revise their paper according to the reviewer's suggestions.

Three reviewers lean to accept this paper and one reviewer remain suggest to reject this work; however, their confidence scores were not very high (scores: 3, 4, 3, 3). After carefully reading the discussion content and the paper, I decide to ignore the negative score from Reviewer Mv3D due to its low quality and finally make the acceptance recommendation to this work.