MC ite B ench: A Multimodal Benchmark for Generating Text with Citations (2025)

Abstract: Multimodal Large Language Models (MLLMs) have advanced in integrating diverse modalities but frequently suffer from hallucination. A promising solution to mitigate this issue is to generate text with citations, providing a transparent chain for verification. However, existing work primarily focuses on generating citations for text-only content, leaving the challenges of multimodal scenarios largely unexplored. In this paper, we introduce MCiteBench, the first benchmark designed to assess the ability of MLLMs to generate text with citations in multimodal contexts. Our benchmark comprises data derived from academic papers and review-rebuttal interactions, featuring diverse information sources and multimodal content. Experimental results reveal that MLLMs struggle to ground their outputs reliably when handling multimodal input. Further analysis uncovers a systematic modality bias and reveals how models internally rely on different sources when generating citations, offering insights into model behavior and guiding future directions for multimodal citation tasks.

## introduction multimodal large language models (mllms) have shown remarkable progress in integrating external information from diverse modalities, allowing them to generate responses beyond the scope of their internal knowledge @xcite @xcite . despite the advancements, these models frequently suffer from hallucination @xcite , undermining the faithfulness of their outputs @xcite . a natural strategy to alleviate this issue is citation: allowing the model to attribute each generated statement to its source, thereby improving transparency and verifiability. existing studies on generating text with citations mainly focus on the textual modality @xcite [1] method b uses llms to score reference-free "win-rates", which neglects ... and always requires evaluating model pairs. instead ... unlike prior benchmarks such as method b, which rely on reference-free "winrates" and require evaluating model pairs @xcite , method a ... additionally, method a demonstrates consistent rankings of frontier models across various benchmarks, as shown in figure 1 . furthermore, method a's datasets, span a wide range of input lengths (from 73k to 518k tokens), ..., as detailed in ## related work generating text with citations recent efforts have explored the task of generating text with citations, where models are required to produce responses with explicit references to supporting sources. @xcite ; @xcite first introduced this setting to improve the verifiability of model responses. subsequent works have explored two main paradigms: generating both the response and citations simultaneously @xcite , and attaching citations in a post-processing step @xcite . these approaches have also been extended to tasks such as long-context citation @xcite and fine-grained attribution @xcite . another related line of work is traditional citation text generation, which typically refers to generating citation sentences in academic papers that contain specific scientific claims and cite prior work @xcite @xcite . however, existing studies focus almost exclusively on textual evidence, limiting their applicability in real-world multimodal scenarios. in this work, we address this gap by incorporating figure and tabular content as citation sources and evaluating model attribution in multimodal contexts. ## mcitebench in this section, we define the task of generating text with citations from multimodal input and describe qa pairs construction paper pdf doc ocr extracting table figure text candidate evidence human filter model filter mcitebench locate evidence pairing quality control attribution corpus collection table figure text candidate evidence rebuttal comments generating locating qa explanation qa data sampling method is exactly as described in line 240-244…. in practice, we rely solely on the dataset…. ## evaluation metrics we evaluate the models across three dimensions: citation quality, source reliability, and answer accuracy. using citation f1, we assess whether the cited evidence accurately and sufficiently supports judge model evidences this leads to …, as shown in figure 5, where …[1]. sentence fully support sentence 1 this leads to …, as shown in figure 5, where …[1]. sentence citation: (figure 5, [1]) figure 5, [1] @xmath0. source reliability ensures that the model's response cites the ground truth source needed to answer the query. we measure this by comparing the model-generated citation with ground truth citation, using both source f1 and source exact match scores. answer accuracy metrics are designed to assess whether the model's response correctly addresses the query. ## conclusion in this paper, we introduce mcitebench, a highquality benchmark built from academic papers and their review-rebuttal interactions, to evaluate the ability of mllms to generate text with citations from multimodal input. leveraging this benchmark, we conduct a detailed evaluation of model performance across multiple dimensions. through extensive experiments, we find that existing models struggle to accurately attribute their outputs to the correct multimodal sources. furthermore, we dive deep into the analysis of attention distribution during citation generation and uncover modality bias exhibited by current models. we hope that mcitebench offers valuable insights into generating text with citations and contributes to the development of models capable of producing faithful and verifiable responses.