Title: Unveiling the Tapestry of Consistency in Large Vision-Language Models
Venue: NeurIPS 2024
Accepted: yes
Keywords: Consistency

ConBench

Large Vision-Language Models

Analysis
Abstract: Large vision-language models (LVLMs) have recently achieved rapid progress, exhibiting great perception and reasoning abilities concerning visual information. However, when faced with prompts in different sizes of solution spaces, LVLMs fail to always give consistent answers regarding the same knowledge point. This inconsistency of answers between different solution spaces is prevalent in LVLMs and erodes trust. To this end, we provide a multi-modal benchmark ConBench, to intuitively analyze how LVLMs perform when the solution space of a prompt revolves around a knowledge point. Based on the ConBench tool, we are the first to reveal the tapestry and get the following findings: (1) In the discriminate realm, the larger the solution space of the prompt, the lower the accuracy of the answers. 
(2) Establish the relationship between the discriminative and generative realms: the accuracy of the discriminative question type exhibits a strong positive correlation with its Consistency with the caption. (3) Compared to open-source models, closed-source models exhibit a pronounced bias advantage in terms of Consistency. Eventually, we ameliorate the consistency of LVLMs by trigger-based diagnostic refinement, indirectly improving the performance of their caption. We hope this paper will accelerate the research community in better evaluating their models and encourage future advancements in the consistency domain.


Peer reviews:
comments: SUMMARY:
This paper introduces a multimodal Consistency Benchmark (ConBench) to systematically evaluate the capabilities of LVLMs via diverse question formats. 
ConBench has a total of 4k questions on 1k images and corresponding 3k discriminative ground truths, as well as two special metrics to evaluate the consistency of LVLMs. 
Based on ConBench, this work conducts a comprehensive analysis of inconsistency in LVLMs.

STRENGTHS:
- The constructed ConBench can better evaluate LVLMs and encourage further advancements in the consistency domain. 
- Their findings provide insight for future community research.
- This paper proposes a trigger-based diagnostic refinement method (TDR) to ameliorate the generation skill of LVLMs without any additional training.

WEAKNESSES:
- The results in Table 4 suggest that TDR can markedly improve the consistency of baseline LVLMs. In addition to ConScore[C], how does TDR affect LVLMs' performance on comprehensive multimodal benchmarks?

QUESTIONS:
See Weakness

LIMITATIONS:
Yes, the authors discuss limitations in the Appendix.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2024-07-11

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents ConBench, a multi-modal benchmark to intuitively analyze how LVLMs perform when different prompts are used for one model around a knowledge point. Based on the proposed benchmark, several interesting findings are pointed out, such as the relationships between the prompt space and the answer accuracy in both the discrimitive and generative realm. Based on the findings, the paper proposes a way to improve  the consistency of LVLMs by trigger-based diagnostic refinement. This is specifically conducted on improving the caption capabilities of LVLMs.\n\nSTRENGTHS:\n1. The benchmark is comprehensive and the related findings are interesting.\n2. The motivation for the proposed method to improve the consistency of LVLMs is straightforward based on the findings.\n3. The evaluations conducted on the designed benchmarks provide evidence of the effectiveness. However, there are some concerns regarding the experimental results, which will be further discussed in the weaknesses section.\n\nWEAKNESSES:\n1. Some of the presentations are unclear. For example, the definition of the discriminative and generative domains should be clearly explained at the beginning to avoid  obscure understandings.\n2. Some metrics are not clearly explained, including but not limited to ConScore[C], metric[C] and metric[D] in Sec 4.1.\n3. For the generative problem, the paper uses GPT4 to compare the consistency of the LVLM output answer and the ground truth. However, several issues arise here: (1) Does GPT4 have a certain bias; (2) Does GPT4 also have inconsistent judgment issues; (3) How to eliminate the impact of GPT4's inconsistent judgments, for example, is it feasible to conduct several experiments and provide results variation.\n4. About the Trigger-based Diagnostic Refinement: it seems that previous work [a] also used the same way to give more accurate answers in LLM realm, that is prompting the models with previous answers and the same-meaning questions. Further explanations are suggested to clarify on the common and different things between the proposed Trigger-based Diagnostic Refinement and other works (e.g., [a]) in LLM realm.\n\n[a] Chuanyang Zheng, etc.  Progressive-Hint Prompting Improves Reasoning in Large Language Models， 2023.\n\nQUESTIONS:\nShown in the weaknesses part\n\nJustification of final rating:\n\nI appreciate the opportunity to evaluate this submission. Overall, the proposed benchmark is interesting. The findings based on the benchmark are reasonable. The proposed methods based on the findings are practical. I give my rating as boardline accept.\n\nLIMITATIONS:\nThe author has claimed it in the checklist.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-07-14"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presented a comprehensive study LVLMs on their inconsistent answers given different prompt solution spaces (true/false, multiple choice, and limited QA). Specifically, the authors introduced the ConBench benchmark to evaluate the performance of various models. They investigated the relationship between the size of the solution space and the accuracy of answers provided by these models, and analysed the consistency between answers and the generated captions. They also presented a simple yet effective method to ameliorate the consistency.\n\nSTRENGTHS:\n* It is an interesting perspective to construct a benchmark to evaluate the model's performance when prompt with different solution spaces are provided. In fact, this is a common problem that we might encounter when we used LVLMs to answer our questions.\n* The analysis was comprehensive with some insightful findings. In particular, the positive correlation between the accuracy and consistency, and the relationship between the confidence and consistency.\n* The authors also provided a simple yet effective method, i.e., picking those words with lower confidence scores for re-prompting, to generate more accurate captions. The method did not require re-training the model.\n\nWEAKNESSES:\n* The proposed benchmark was not sufficient to evaluate the model, with the size of only 1K images, and three types of questions. It would be great if the bench can be more diverse and representative for various domains.\n* It was not clear what was the insight for Fact 4.4.1. Did the authors want to show that closed-source models put more efforts on improving the consistency compared with open-sourced ones?\n* The current method to improve consistency was ad-hoc and based on such an idea, it is likely that simply changing the generation arguments like temperature, top_p or top_k might have similar effects.\n* The paper seems more suitable to be submitted to the Datasets & Benchmarks track instead of the main track.\n\nQUESTIONS:\nSee above for questions.\n\nLIMITATIONS:\nThe authors discussed potential social impacts as well as limitations of their paper in the appendix. I agreed with them that using GPTs for evaluation might pose a bias and a more rigorous human evaluation was preferred.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-21"}