Decompose and Compare Consistency: Measuring VLM s’ Answer Reliability via Task-Decomposition Consistency Comparison (2024)

Abstract: Despite tremendous advancements, current state-of-the-art Vision-Language Models (VLMs) are still far from perfect. They tend to hallucinate and may generate biased responses. In such circumstances, having a way to assess the reliability of a given response generated by a VLM is quite useful. Existing methods, such as estimating uncertainty using answer likelihoods or prompt-based confidence generation, often suffer from overconfidence. Other methods use self-consistency comparison but are affected by confirmation biases. To alleviate these, we propose Decompose and Compare Consistency (DeCC) for reliability measurement. By comparing the consistency between the direct answer generated using the VLM’s internal reasoning process, and the indirect answers obtained by decomposing the question into sub-questions and reasoning over the sub-answers produced by the VLM, DeCC measures the reliability of VLM’s direct answer. Experiments across six vision-language tasks with three VLMs show DeCC’s reliability estimation achieves better correlation with task accuracy compared to the existing methods.

## introduction automatic measurement of reliability of responses generated by ai systems such as vision-language models (vlms) is useful for deciding whether to trust a response or not, which in turn is necessary to build secure systems and enable further improvements @xcite . existing reliability estimation methods often estimate the model's uncertainty using answer likelihoods or prompt the model to generate a confidence value @xcite @xcite . these methods often fail to correlate well with task accuracy because models are not well-calibrated and tend to be overconfident @xcite inconsistent, the answer is unreliable! which person is everyone here staring at now? question: catcher vlm decomposer what is the batter's focus on? sub-question 1 is the umpire visible in the image? sub-question 2 are the other players and the umpire looking in the same direction as the batter? sub-question 4 is the catcher visible in the image? sub-question 3 pitcher. sub-answer 1 yes sub-answer 2 yes sub-answer 3 yes sub-answer 4 decomposed sub-qa pairs vlm vlm llm for reasoning is the answer reliable? 🧐 which person is everyone here staring at now? original question pitcher pitcher catcher vlm's reasoned answer llm's reasoned answer vlm's direct answer ## related work existing methods use uncertainty-based metrics for reliability measurement, such as setting a reliability threshold on answer likelihoods @xcite @xcite , or prompting the model to generate a confidence value @xcite @xcite . however, uncertainty-based metrics often lead to overconfidence since confidence calibration is not a training goal @xcite . but retraining models to generate calibrated confidence @xcite @xcite is impractical for evaluating multiple vlms. self-consistency methods generate multiple responses to assess reliability @xcite @xcite but suffer from confirmation biases @xcite . multi-agent collaboration can mitigate this. @xcite use multiple llms to interact in cooperative and competitive settings to evaluate reliability. @xcite use llms to generate related questions about the image and use high-confidence qa pairs as premises, with the original qa as the hypothesis, to determine reliability. our approach differs by decomposing the question into simpler sub-questions. we also conduct extensive experiments to explore the effectiveness of different consistency comparison settings on reliability measurement. ## method for a question q, an image i, and an answer a from a candidate vlm, decc obtains a binary reliability score r indicating whether a is reliable. as shown in fig 1 , decc contains two components: task decomposition and consistency comparison. ## conclusion we use consistency comparison based on task decomposition for measuring vlms answer reliability. by decomposing complex questions into simpler sub-questions, we achieve more accurate and robust reliability estimation. we find the performance of reliability measurement and the effectiveness of different consistency comparison settings correlate with candidate vlm's capabilities.