Attention Consistency for LLM s Explanation (2025)

Abstract: Understanding the decision-making processes of large language models (LLMs) is essential for their trustworthy development and deployment, however, current interpretability methods often face challenges such as low resolution and high computational cost. To address these limitations, we propose the Multi-Layer Attention Consistency Score (MACS), a novel, lightweight, and easily deployable heuristic for estimating the importance of input tokens in decoder-based models. MACS measures contributions of input tokens based on the consistency of maximal attention. Empirical evaluations demonstrate that MACS achieves a favorable trade-off between interpretability quality and computational efficiency, showing faithfulness comparable to complex techniques with a 22% decrease in VRAM usage and 30% reduction in latency.

## introduction the rapid advancements in large language models (llms) have revolutionized natural language processing, powering diverse applications @xcite @xcite @xcite . as these models become increasingly integrated into critical systems, understanding their decision-making processes is crucial for ensuring trustworthiness, reliability, and enabling targeted improvements. explainable ai (xai), particularly methods attributing outputs to input tokens, is thus essential. however, existing approaches for interpreting deep decoder-only transformers face notable limitations: full attention aggregation methods like attention rollout can produce noisy or diffuse attributions, potentially due to phenomena like over-squashing, over-mixing, and softmax dispersion which can obscure important signals in deep models and long sequences. concurrently, many other xai techniques, including gradient and perturbation-based metric macs rollout attnlrp atman faithfulness usability comp. eff. vram eff. our main contributions are as follows: we show that macs achieves a favorable tradeoff between computational efficiency and attribution quality, making it a practical component for interpreting llms. ## background understanding the decision-making process of transformer-based models, especially large language models (llms), is a critical research objective in the field of explainable ai (xai) @xcite . existing attribution methods that link model predictions to input tokens can be broadly categorized by their primary mechanisms, including attention aggregation, perturbation, and backpropagation. ## methodology: from full aggregation to attention consistency this section outlines the motivation and formulation of our proposed method, macs. we begin by discussing the limitations inherent in full attention aggregation approaches, which motivates our alternative strategy based on measuring attention consistency. ## ablation study to validate the design of macs and understand the contribution of its distinct components, we conduct a series of ablation studies. we systematically remove or modify key elements of the macs algorithm and evaluate the impact on performance. all ablations are performed on the squad 2.0 subset using auc-pr as the primary metric, as described in section 4.1.1. the full macs method (detailed in section 3.2.2) serves as the baseline for comparison. ## conclusion we have introduced multi-layer attention consistency score (macs), a novel heuristic for quantifying input token importance in decoder-only models. designed to be lightweight, computationally efficient and usable without model modifications. our empirical evaluations reveal that macs frequently matches more sophisticated attribution techniques. these findings also support our hypothesis that measuring the consistent strength of maximal attention links across layers offers a clear and effective way to identify salient input contributions. consequently, macs serves as a practical tool for obtaining rapid insights, thus contributing an efficient way toward enhancing interpretability in llms.