MHALO : Evaluating MLLM s as Fine-grained Hallucination Detectors (2025)

Abstract: Hallucination remains a critical challenge for multimodal large language models (MLLMs), undermining their reliability in real-world applications. While fine-grained hallucination detection (FHD) holds promise for enhancing high-quality vision-language data construction and model alignment through enriched feedback signals, automated solutions for this task have yet to be systematically explored. Inspired by the concept of “MLLM as a Judge”, we introduce MHALO, the first comprehensive benchmark specifically designed for evaluating MLLMs’ capability in performing token-level FHD. Our benchmark encompasses 12 distinct hallucination types spanning both multimodal perception and reasoning domains. Through extensive evaluations of 9 selected MLLMs, we reveal substantial performance limitations, with the leading model achieving an average F1_{IoU} of only 40.59%. To address this limitation, we develop HaloDet-4B, a specialized model trained on our curated training data, which significantly outperforms existing models. We hope the benchmark can provide valuable insights for future research on hallucination mitigation in MLLMs. The code and dataset will be publicly available.

## introduction the advancement of multimodal large language models (mllms; @xcite @xcite ) represents a groundbreaking achievement in the field of ai, demonstrating exceptional capabilities in perception and reasoning @xcite @xcite @xcite . despite their promise, mllms are still plagued by hallucination, a phenomenon that involves generating erroneous or fabricated responses contradicting the actual visual content or language context @xcite @xcite . therefore, to address this issue and enhance the reliability of mllms, fine-grained hallucination detection (fhd), which offers enriched token-level feedback signals, emerges as a crucial solution to mitigate the generation of erroneous or fabricated responses. unlike coarse-grained feedback that penalizes hallucinations at the expense of suppressing correct content @xcite , fhd accelerates human annotation and data refinement by pinpointing hallucinations @xcite , thereby facilitating efficient acquisition of high-quality data. furthermore, fhd offers more informative signals, leading to effective model alignment. @xcite @xcite . despite its advantages, current research on multimodal hallucination detection still exhibits limitations in granularity. @xcite was one of the first to conduct fine-grained hallucinations evaluations by verifying the extracted atomic facts in responses against the input image. @xcite proposed a unified detection framework using external tools to validate hallucinations. both of them operate detection at the claim level and lack the ability to precisely localize hallucinations. automated hallucination detection at a more finegrained level, token level, remains unexplored. inspired by the concept of mllms as a judge @xcite @xcite , a natural question emerges: "can mllms serve as reliable judges for fhd?" this necessitates establishing a meta-evaluation benchmark that can effectively assess the performance of mllms on fhd. building such a benchmark presents two key challenges: (1) construct a tailored dataset ensuring comprehensive coverage of hallucination types across diverse scenarios. (2) developing quantitative and objective evaluation metrics that align with human judgment. to bridge these research gaps, we introduce mhalo, a novel fhd benchmark consisting of 2,155 carefully curated instances with token-level annotations. it features the following aspects: on the one hand, prior work has mainly focused on natural scenes and contains only a small proportion of questions requiring mathematical reasoning @xcite @xcite , leaving a comprehensive investigation of hallucination detection within vision-language reasoning largely unexplored. thus, we present a comprehensive taxonomy covering hallucinations in both multimodal perception and reasoning processes, categorizing hallucinations into 12 distinct types (see the pie chart in figure 1 ). on the other hand, mhalo defines fhd as a task requiring models to provide token-level hallucination annotations (see examples in figure 1 ), taking into account both recognition and localization aspects, and we propose the corresponding metrics f 1 m and f 1 iou , the latter inspired by object detection to objectively assess the accuracy of detection. we demonstrate their effectiveness through rigorous validation. we evaluate multiple well-known mllms (ope-nai, 2024; anthropic, 2024) on mhalo and in-vestigate the impact of different prompting strategies on their performance. it can be observed that fhd poses significant challenges for state-ofthe-art (sota) mllms, with the leading mllm on mhalo, gpt-4o, achieving an average f 1 iou of only 40.59%. in order to build a highperformance fine-grained hallucination detector, we adopt a data-driven strategy to fine-tune a specialized model halodet-4b, which achieves sota performance on mhalo. our contributions are as follows: type definition object incorrect identification of objects in visual content. ## mhalo we present mhalo, a novel benchmark encompassing 2,155 meticulously curated entries. the benchmark construction addresses three core challenges: (1) granular annotation framework, (2) comprehensive hallucination taxonomy, and (3) detection-oriented metrics. we begin by describing its unique features compared to previous works (section 2.1). next, we outline the data curation pipeline (section 2.2). finally, we show the design of token-level metrics (section 2.3) for the automated and quantitative assessment of precise detection performance. ## fine-tuning an mllm as a detector in our preliminary experiments, we observed that leading mllms @xcite are not particularly effective at detecting hallucinatory segments (see ## conclusion in this paper, we introduce a novel meta-evaluation benchmark, mhalo, designed to assess different mllms ' capability in performing fhd. by systematically evaluating 9 well-known mllms, we highlight the significant performance gaps, none of the models exceeded 50% f 1 iou . to address this limitation, we develop halodet-4b, a specialized model that significantly outperforms existing models. this benchmark, along with the trained detector, provides valuable tools for improving hallucination detection in mllms and can guide future research in model alignment.