AUTOHALLUSION: Automatic Generation of Hallucination Benchmarks for
  Vision-Language Models
Authors: name: Xiyang Wu; name: Tianrui Guan; name: Dianqi Li; name: Shuaiyi Huang; name: Xiaoyu Liu; name: Xijun Wang; name: Ruiqi Xian; name: Abhinav Shrivastava; name: Furong Huang; name: Jordan Lee Boyd-Graber; name: Tianyi Zhou; name: Dinesh Manocha
Published: 2024-06-16 11:44:43

Abstract: Large vision-language models (LVLMs) hallucinate: certain context cues in an
image may trigger the language module's overconfident and incorrect reasoning
on abnormal or hypothetical objects. Though a few benchmarks have been
developed to investigate LVLM hallucinations, they mainly rely on hand-crafted
corner cases whose fail patterns may hardly generalize, and finetuning on them
could undermine their validity. These motivate us to develop the first
automatic benchmark generation approach, AUTOHALLUSION, that harnesses a few
principal strategies to create diverse hallucination examples. It probes the
language modules in LVLMs for context cues and uses them to synthesize images
by: (1) adding objects abnormal to the context cues; (2) for two co-occurring
objects, keeping one and excluding the other; or (3) removing objects closely
tied to the context cues. It then generates image-based questions whose
ground-truth answers contradict the language module's prior. A model has to
overcome contextual biases and distractions to reach correct answers, while
incorrect or inconsistent answers indicate hallucinations. AUTOHALLUSION
enables us to create new benchmarks at the minimum cost and thus overcomes the
fragility of hand-crafted benchmarks. It also reveals common failure patterns
and reasons, providing key insights to detect, avoid, or control
hallucinations. Comprehensive evaluations of top-tier LVLMs, e.g.,
GPT-4V(ision), Gemini Pro Vision, Claude 3, and LLaVA-1.5, show a 97.7% and
98.7% success rate of hallucination induction on synthetic and real-world
datasets of AUTOHALLUSION, paving the way for a long battle against
hallucinations.