ALOH a: A New Measure for Hallucination in Captioning Models (2024)

Abstract: Despite recent advances in multimodal pre-training for visual description, state-of-the-art models still produce captions containing errors, such as hallucinating objects not present in a scene. The existing prominent metric for object hallucination, CHAIR, is limited to a fixed set of MS COCO objects and synonyms. In this work, we propose a modernized open-vocabulary metric, ALOHa, which leverages large language models (LLMs) to measure object hallucinations. Specifically, we use an LLM to extract groundable objects from a candidate caption, measure their semantic similarity to reference objects from captions and object detections, and use Hungarian matching to produce a final hallucination score. We show that ALOHa correctly identifies 13.6% more hallucinated objects than CHAIR on HAT, a new gold-standard subset of MS COCO Captions annotated for hallucinations, and 30.8% more on nocaps, where objects extend beyond MS COCO categories.

## introduction and background in recent years, vision-language models have demonstrated remarkable performance. unfortunately, even state-of-the-art models for visual description still generate captions with object hallucinations -objects or entities that are present in the caption yet are not explicitly supported by visual evidence in the image @xcite . in order to reduce the occurrence of object hallucinations in vision-language models, it is helpful to understand and quantify the problem through reliable, localizable, and generalizable measures of object hallucination. reliable measures are capable of correctly indicating if a given caption contains an object hallucination. localizable measures are capable of indicating which object in a particular caption is hallucinated. generalizable measures are capable of eval-figure 1: (top) the sota prior object hallucination metric, chair, is limited to ms coco objects, and fails to detect the hallucinations in this image caption while aloha (ours, bottom) correctly assigns low similarity scores to the hallucinations "baseball player" and "bat". aloha does not penalize the caption for "catcher", "umpire", and "bass drum", as the caption indicates uncertainty of their presence. recent works that measure object hallucinations in generated text generally fall into two categories: measures that find hallucinations by explicitly matching from a set of objects, and measures that compute distances between latent image and/or text embeddings, indicating a hallucination if the embeddings are too distant. in the first category, chair @xcite is a measure that explicitly extracts objects from candidate sentences using simple string matching against ms coco classes and a small set of synonyms. it compares these extracted objects against the ground truth detections and objects extracted from the ground truth reference captions. chair is both reliable, as string matching on a fixed set of objects is accurate, consistent, and localizable, as individual non-matching strings are identified. however, as seen in figure 1 , chair is not generalizable, as it can only handle a fixed set of predetermined objects. other uni-modal measures in this category include those for abstractive summarization @xcite @xcite @xcite , dialogue @xcite , and structured knowledge @xcite . these often generalize poorly to vision-language tasks as they require grounding the generated text into inputs of the same modality. in the second category, clipscore @xcite employs clip @xcite embeddings to assess image-text matches. while it is generalizable and reliable, it lacks localization as it does not pinpoint incorrect spans of text. clipberts @xcite and ref-clipscore (an extension of clipscore accounting for reference captions) face similar limitations. pope @xcite evaluates vision-language models' likelihood to hallucinate objects with machine-generated queries consisting of samples extracted from both reference object detections and nonexistent objects, but addresses a different problem from that which we investigate here -it measures how often models hallucinate rather than localizes and detects issues within a single caption. inspired by recent successes using llms for evaluation in language-only tasks @xcite @xcite @xcite , we introduce assessment with language models for object hallucination (aloha), a modernized measure for object hallucination detection that is reliable, localizable, and generalizable. aloha extends the reliability and localization of chair to new input domains by leveraging in-context learning of llms combined with semantically rich text embeddings for object parsing and matching (figure 1 ). for a given image caption, we generate two measures: aloha o , a numeric score for each object rating the degree to which that object is a hallucination, and aloha, an aggregated score rating the degree to which the whole caption contains a hallucination. we demonstrate aloha on a new gold-standard dataset of image hallucinations, hat, and show that aloha improves on clipscore while detecting object hallucinations, and chair while correctly localizing those hallucinations. we conclude by demonstrating that aloha remains reliable and localizable when generalizing to out-of-domain data. ## evaluation & discussion hat: (ap) and localization accuracy (la). the ap of the method measures reliability and is defined as how well the measure identifies captions with hallucinations. for chair, decisions are binary, so @xmath0. for aloha, ap is the weighted mean of precisions across all thresholds. the la, measured on samples containing hallucinations in hat, measures localization and is defined as the accuracy of correctly indicating which of the specific objects were hallucinated. for chair, a hallucination is correctly localized when at least one detected string mismatch is a hallucination, and for aloha when the minimum aloha o score corresponds to a hallucinated object. aloha's performance on hat is shown in ## conclusion this paper introduces aloha, a scalable llmaugmented metric for open-vocabulary object hallucination. aloha correctly identifies 13.6% more hallucinated objects on hat and 31% on nocaps-foil than chair. aloha represents an important modernization of caption hallucination metrics, and detecting complex hallucinations in actions, quantities, and abstract concepts remains an exciting and challenging task for future exploration.