F aith S core: Fine-grained Evaluations of Hallucinations in Large Vision-Language Models (2024)

Abstract: We introduce FaithScore (Faithfulness to Atomic Image Facts Score), a reference-free and fine-grained evaluation metric that measures the faithfulness of the generated free-form answers from large vision-language models (LVLMs). The FaithScore evaluation first identifies sub-sentences containing descriptive statements that need to be verified, then extracts a comprehensive list of atomic facts from these sub-sentences, and finally conducts consistency verification between fine-grained atomic facts and the input image. Meta-evaluation demonstrates that our metric highly correlates with human judgments of faithfulness. We collect two benchmark datasets (i.e. LLaVA-1k and MSCOCO-Cap) for evaluating LVLMs instruction-following hallucinations. We measure hallucinations in state-of-the-art LVLMs with FaithScore on the datasets. Results reveal that current systems are prone to generate hallucinated content unfaithful to the image, which leaves room for future improvements. We hope our metric FaithScore can help evaluate future LVLMs in terms of faithfulness and provide insightful advice for enhancing LVLMs’ faithfulness.

## introduction large language models (llms), such as @xcite and @xcite , have demonstrated various language modeling capabilities. despite their achievements, they still lack the capacity to handle multimodal inputs effectively. as a result, a significant amount of research has shifted its focus towards large vision-language models (lvlms) @xcite @xcite by incorporating powerful llms @xcite and vision foundation models (vfms) (dosovitskiy et al., 2021; bommasani 1 https://github.com/bcdnlp/faithscore . ## estimating faithscore in this section, we begin by clearly defining the research problem in section 3.1, followed by a detailed framework of estimating faithscore in section 3.2. ## meta-evaluate faithscore to verify that our automatic evaluation correlates with human judgment, we conduct human evaluations in terms of hallucination. we select the test dataset from the llava paper @xcite (llava-bench) for human evaluation, which is constructed based on the mscoco dataset. this test set is a visual instruction following dataset comprising three distinct question types: detailed description, conversation, and complex question. for each type, this dataset includes 90 questions. we select answers from llava @xcite and instructblip @xcite ) models for evaluation. ## conclusion we introduce a novel metric called faithscore for evaluating free-form and open-domain answers generated by large vision-language models. compared to previous metrics, faithscore offers a finer level of granularity, interpretability, and closer alignment with human judgments. our quantitative analysis demonstrates that current lvlms are prone to visual hallucination problems. we also find that the answer length and number of objects could affect the faithfulness of lvlms. in addition, the faithfulness performance of lvlms on different types of atomic facts varies. we expect that faithscore will be of great value for evaluating forthcoming advanced lvlms.