Where MLLMs Attend and What They Rely On: Explaining Autoregressive
  Token Generation
Authors: name: Ruoyu Chen; name: Xiaoqing Guo; name: Kangwei Liu; name: Siyuan Liang; name: Shiming Liu; name: Qunli Zhang; name: Hua Zhang; name: Xiaochun Cao
Published: 2025-09-26 15:38:42

Abstract: Multimodal large language models (MLLMs) have demonstrated remarkable
capabilities in aligning visual inputs with natural language outputs. Yet, the
extent to which generated tokens depend on visual modalities remains poorly
understood, limiting interpretability and reliability. In this work, we present
EAGLE, a lightweight black-box framework for explaining autoregressive token
generation in MLLMs. EAGLE attributes any selected tokens to compact perceptual
regions while quantifying the relative influence of language priors and
perceptual evidence. The framework introduces an objective function that
unifies sufficiency (insight score) and indispensability (necessity score),
optimized via greedy search over sparsified image regions for faithful and
efficient attribution. Beyond spatial attribution, EAGLE performs
modality-aware analysis that disentangles what tokens rely on, providing
fine-grained interpretability of model decisions. Extensive experiments across
open-source MLLMs show that EAGLE consistently outperforms existing methods in
faithfulness, localization, and hallucination diagnosis, while requiring
substantially less GPU memory. These results highlight its effectiveness and
practicality for advancing the interpretability of MLLMs. The code is available
at https://github.com/RuoyuChen10/EAGLE.