Evaluating Object Hallucination in Large Vision-Language Models (2023)

Abstract: Inspired by the superior language abilities of large language models (LLM), large vision-language models (LVLM) have been recently proposed by integrating powerful LLMs for improving the performance on complex multimodal tasks. Despite the promising progress on LVLMs, we find that they suffer from object hallucinations, i.e., they tend to generate objects inconsistent with the target images in the descriptions. To investigate it, this work presents the first systematic study on object hallucination of LVLMs. We conduct the evaluation experiments on several representative LVLMs, and show that they mostly suffer from severe object hallucination issues. We further discuss that the visual instructions may influence the hallucination, and find that: objects that frequently appear in the visual instructions or co-occur with the image objects are obviously prone to be hallucinated by LVLMs. Besides, we further design a polling-based query method called POPE for better evaluation of object hallucination. Experiment results show that our POPE can evaluate object hallucination in a more stable and flexible way.

## introduction large language models (llms) @xcite have shown remarkable abilities to solve various complex tasks by following human instructions in a zero-shot manner. the success of llms drives the researchers to devise more powerful multimodal models based on the superior capacity of llms, to enhance the understanding of visual semantics @xcite . as an exemplified work, @xcite has exhibited the exciting performance of llms on multimodal tasks and scenarios. following this line of research, a surge of studies @xcite @xcite have been proposed to enhance the visionlanguage pre-trained model (vlpm) @xcite by incorporating powerful llms @xcite , which are called large vision-language model (lvlm). typically, existing work reuses the visual encoder in vlpms to handle image data, while replacing the original language encoder with llms. after vision-language pre-training @xcite and visual instruction tuning @xcite , lvlms can fulfill complex tasks according to human instructions, demonstrating strong capacities in solving various vision-language tasks, e.g., image captioning @xcite @xcite and visual question answering @xcite @xcite . despite the success of lvlms, previous work has revealed that their main components, i.e., llms and vlpms, both suffer from hallucination. especially, llms tend to hallucinate unintended text @xcite , and vlpms might generate nonexistent objects in the image @xcite (termed as object hallucination). it is generally believed that the hallucination would degrade the model performance and greatly harm the user experiences in real-world applications @xcite . therefore, it is natural to ask the question: does hallucination still exist in lvlms? in this paper, we systematically evaluate the issue of object hallucination in existing lvlms, which refers to generating contents that are inconsistent with ground-truth objects in the given image. to conduct our study, we first use the chair (caption hallucination assessment with image relevance) metric @xcite , and examine the hallucination degree of several representative lvlms on the mscoco dataset. our preliminary experiments ( provide a detailed description of the given image. ## object hallucination in lvlms in this section, we evaluate the object hallucination problem in popular lvlms using an existing method. we first introduce the evaluation settings and then analyze the experimental results. ## influence of instruction data on object hallucination considering their impressive performance on complex vision-language tasks @xcite @xcite , it is counter-intuitive that the hallucination problem of lvlms is so severe. since smaller vlpms suffer less from object hallucination, it is possible that the visual instruction-tuning process of lvlms exacerbates object hallucination. in this section, we investigate the influence of the visual instruction data. we first make two basic hypotheses in section 4.1 and then conduct qualitative and quantitative analysis to verify them in section 4.2 and section 4.3. ## pope in this section, we devise polling-based object probing evaluation (pope), a simple yet effective approach for evaluating hallucination in lvlms. we first provide an overview of pope, and then evaluate the representative lvlms with pope. finally, we discuss the stability and scalability of our method, and also analyze the impact of hallucina- ## conclusion in this work, we conducted evaluation experiments on several lvlms and examined how they suffer from the object hallucination issue. by investigating the reasons for object hallucination, we empirically revealed that the object distributions of the visual instructions would affect the object hallucination of lvlms. besides, we also found that the existing hallucination evaluation methods might be affected by the input instructions and the generated text of lvlms, thus leading to less reliable evaluation results. to address this issue, we proposed a polling-based query method called pope, to provide an improved evaluation approach for the object hallucination of lvlms. experimental results have shown that our proposed pope can better evaluate the object hallucination issue of lvlms.