Title: Revisiting LRP: Positional Attribution as the Missing Ingredient for Transformer Explainability
Venue: NeurIPS 2025
Accepted: yes
Keywords: Explainability

Interpretability

Transformers

LRP
Abstract: The development of effective explainability tools for Transformers is a crucial pursuit in deep learning research. One of the most promising approaches in this domain is Layer-wise Relevance Propagation (LRP), which propagates relevance scores backward through the network to the input space by redistributing activation values based on predefined rules. However, existing LRP-based methods for Transformer explainability entirely overlook a critical component of the Transformer architecture: its positional encoding (PE), resulting in violations of conservation, and the loss of an important and unique type of relevance, which is also associated with structural and positional features. To address this limitation, we reformulate the input space for Transformer explainability as a set of position-token pairs, rather than relying solely on the standard vocabulary space. This allows us to propose specialized theoretically-grounded LRP rules designed to propagate attributions across various positional encoding methods, including Rotary, Learned, and Absolute PE. Extensive experiments with both fine-tuned classifiers and zero-shot foundation models, such as LLaMA 3, demonstrate that our method significantly outperforms the SoTA in both vision and NLP explainability tasks. Our code is provided as a supplement.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes Positional-Aware Layer-wise Relevance Propagation (PA-LRP), a novel explainability method for Transformers that addresses a key limitation in existing LRP-based approaches—the omission of positional encodings (PE) in attribution. The proposed method differs from prior work in two main ways: (1) it reformulates the input space to incorporate both token and positional information, and (2) it introduces the first LRP rules specifically designed to propagate relevance through standard PE schemes, including Learnable, Sinusoidal, and Attention-level positional encodings such as Rotary PE.\n\nThe authors provide a thorough theoretical analysis, proving that existing methods violate the conservation property and exhibit low faithfulness in tasks that depend heavily on positional information. In contrast, the proposed PA-LRP satisfies the conservation principle and enables more faithful attribution.\n\nExtensive experiments across vision and NLP tasks -- including evaluations on DeiT and LLaMA models -- demonstrate that PA-LRP outperforms baseline methods in most settings.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The authors identify a critical yet previously overlooked limitation in existing LRP-based explainability methods—the lack of attribution to positional encodings, which are essential to Transformer models.\n\n2. The paper introduces new LRP propagation rules tailored for Learnable, Sinusoidal, and Rotary positional encodings, and interprets positional information as a vector passed through attention blocks via a semi-skip connection mechanism, enabling more faithful propagation.\n\n3. The authors provide formal theoretical justifications showing that prior methods violate the conservation property, and prove that the proposed PA-LRP rules satisfy it, ensuring consistency and faithfulness in attribution.\n\nWeakness:\n1. The qualitative examples in Figure 3 do not clearly demonstrate the superiority of PA-LRP over the AttnLRP baseline. The visualizations appear very similar, which weakens the visual impact of the proposed method.\n\n2. Although Tables 1, 2, and 3 report improved quantitative results, the experimental setup lacks detail. In particular, it is unclear how many samples were used in the evaluation, how they were selected, and whether any variance or statistical significance was assessed. The absence of error bars or multiple runs raises concerns about the robustness of the reported improvements.\n\nMinor:\nThe term \"multi-sink mechanism\" used in Table 4 is not explicitly introduced or defined in the main text. While it appears to refer to the layer-wise aggregation of positional relevance across attention layers (as described in Section 3.1 and 3.3), clarifying this terminology when the concept is first introduced would improve readability and help readers better interpret the ablation results. A brief explanation or subsection referencing this mechanism explicitly would be helpful.\n\nQUESTIONS:\n1. In Figure 3, the explainability heatmaps generated by PA-LRP and AttnLRP appear visually similar. Could you elaborate on the qualitative differences that PA-LRP captures and explain why these differences might not be visually apparent despite the claimed improvements?\n\n2. For the perturbation and segmentation tests (Tables 1–3), how many samples were included in the evaluation? Were the results obtained from the full test set or a selected subset? Did you run multiple trials, and if so, could you report any measures of variance or statistical significance?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper approaches the LRP based explainable methods for transformers and extend this method by assigning relevance for the positional encodings, one of the important and key components of the transformers. This paper proposes theoretically grounded relevance rules to propagate the attributions for the positional embeddings.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. One of the major strength of this paper is reformulation to include the positional embeddings theoretically.\n2. The authors mention that assigning relevance to the positional parts can be decoupled and applied to any LRP based framework.\n3. The empirical results shows the effectiveness of assigning the relevance to PE parts specially for the segmentation tests.\n\nWeaknesses\n1. This work is based on the premise that LRP based explainable methods are most effective model specific XAI method. It would be helpful if authors put references for this claim.\n2. This work seems an incremental work on the AttnLRP with additional considerations of positional encodings. This simply appends the positional relevance to that of Attn LRP. The interactions between the attention layers and tokens needs to be explained.\n3. In quantitative analysis, positive perturbation tests for DeIT variants shows Attn LRP performing better for Base and Tiny models. How does the removal of relevant pixels changes the positional understanding, eg: it is understandable that removing relevant pixels does changes the prediction performance but how does the relevance is distributed in context of the positional encodings and propagation rules. \n4. The evaluation is mostly dependent on the perturbation and segmentation based tests. Human based tests might have helped. The quantitative test are mostly acts as proxies.\n\nQUESTIONS:\n1. One of the questions that arise are the situations where positional embeddings are less informative in the predictive performance for tasks such as classification?\n\nLIMITATIONS:\nYes, the authors did mention the limitations in Section 6.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents and addresses a limitation in the explainability of transformer-based deep learning models: The handling of positional encodings (PEs) in Layer-wise Relevance Propagation (LRP) methods. The authors argue that existing LRP-based XAI ignores the role of positional encodings, violating the conservation property and obscuring important positional and structural attributions. To tackle the problem, they reformulate the input space and propose new propagation rules. The effectiveness of the approach is evaluated on vision and NLP tasks.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n* **New problem suggestion.** The paper lays out a previously unconsidered issue with LRP methods\n* **Theoretical Foundation.** The authors provide formal analysis of their rules in Lemma 1 and 2. The proofs are referenced to the appendix but the statements in the main paper are clear.\n* **Experiments span two domains and feature relevant models** such as DeiT variants and the Llama models in realistic domains with quantitative and qualitative metrics\n* **Open-Source Implementation.** The paper provides a complete codebase, supporting replicability and adoption.\n\n## Weaknesses \n\n* **Limited Scope: Heavy reliance on Transformers and LRP**: The method is tightly coupled to LRP for Transformers. While the extension to other LRP variants is mentioned, its generality outside the AttnLRP framework is assumed. Broader applicability or adaptations to alternative XAI paradigms (e.g., SHAP, counterfactuals, or other propagation-based methods) are not explored.\nTo motivate this, the authors argue that LRP is the most reliable technique for transformers (l.31). I don’t think that the reality is that simple. Some current works, e.g., Leemann et al (2024) investigate surrogate model explanations for transformers such as SHAP and LIME and find their performance often comparable to LRP in common metrics such as insertion or deletion curves. Thus, I think the scope and potentially impact could be extended by generalizing the approach to other XAI techniques as well.\n\n* **Motivation for specific rules** LRP decompositions are not unique for a given function. While the rules are presented, there is no explanation for why exactly the given form was chosen. Even for the simple additions, their can be 0-LRP, epsilon-LRP or even Gamma-LRP Rules (Montavon, 2019). There should be some discussion about alternative rules and why they were discarded.\n\n* **Relevance and Signficance.** I don't fully understand the signficance of the distinction between positional and semantic importance yet. Why don’t we just redefine the input to be $E_k$ + $P_i$ where $E_k$ is the semantic token embedding and $P_i$ is positional embedding $i$, and backpropagate to this input quantity using LRP? This should fulfill the conservation properties and attribute to token k at position I, which is exactly what we want? The paper claims to manage to separate positional relevance from semantic relevance, but doesn't discuss why it matters in the end.\n\n* **Some confusion about notation.** Writing of Equation (4) is a bit confusing. While the input space is a set of vectors in (3), here in (4) it seems to be a set of tuples, however the size of the tuple and which combinations are possible is still unclear to me. Please elaborate.\n\n* **No measures of dispersion.** Unfortunately, the results in Table 1-3 feature no measures of dispersion making it hard to assess their significance.\n\n\n\n------------------\n**References**\n\nGrégoire Montavon, Alexander Binder, Sebastian Lapuschkin, Wojciech Samek & Klaus-Robert Müller. Layer-Wise Relevance Propagation: An Overview, 2019.\n\nLeemann, Tobias, Alina Fastowski, Felix Pfeiffer, and Gjergji Kasneci. Attention Mechanisms Don’t Learn Additive Models: Rethinking Feature Importance for Transformers.\" Transactions on Machine Learning Research. 2024.\n\nQUESTIONS:\nQuestions: Can you elaborate on the baselines used for comparison: What was used as the input for plain LRP? Did you backpropagate to E_i or E_i + P_k for the standard learned or sinusoidal embeddings.?\n\nWhy can't we just use LRP from the sum E_I + P_K (for standard additive embeddings)? \n \nWhat was the motivation between just adding (as far as I understand) semantic and positional importance?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 1, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe work proposes an extension to Layer-wise relevance propagation (a feature\nattribution approach) for Transformer models, which encompasses rules to\nexplicitly assign importance to positional embeddings. Specifically, the work proposes rules for five approaches to positional embedding: learnable, sinusoidal, rotary, and relative positional embeddings, as well as ALiBi.\nThe work provides some theoretic motivation regarding the conservativity and fidelity of the approach compared to AttnLRP.\nThe work compares the proposed approach to AttnLRP in empirical perturbation,\nablation, and segmentation experiments on vision data using DeiT, as well as on\nnatural language data using LLaMa-2-7B, LLaMa-2-7B quantized, Tiny LLaMa, and\nLLaMa-3-8B.\nThe work also provides some qualitative comparison on both vision and language data.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n\n- AttnLRP provides a powerful feature attribution approach. Improving this\n  approach with an explicit handling of the positional embedding is a useful\n  contribution to the field.\n- The work touches a multitude of positional embedding approaches, which\n  significantly aids in its practicality.\n- The work uses established evaluation metrics for feature attribution, namely\n  perturbation and ground-truth segmentation.\n- The work follows a clear structure and is well-written.\n\n### Weaknesses\n\n- The empirical experiments are partly inconclusive, given that for some\n  metrics, the baseline wins, and for others, the improvements seem to be\n  marginal. However, there are no error bars or number of trials reported,\n  which would aid in testing for statistical significance.\n- The work argues that AttnLRP is not conservative unless positional embeddings\n  are attributed. I do not understand why this argument matters for the\n  distinction between considering and not considering positional embeddings.\n  Even if one would deem this argument valuable, both approaches seem to be\n  conservative in their own perspective.\n- The work claims to present some theoretical evidence that the omission of\n  positional embeddings in attribution gives low fidelity. However, this is not\n  proven, but only an extreme-case example is provided. The claim is also not\n  consistent with the empirical results, which indicate that in some cases,\n  fidelity can be higher when omitting the attribution of positional\n  embeddings.\n\nQUESTIONS:\n1. The quantitative experiments do not include any errors over the trials. I\n   could also not find the number of trials conducted for these experiments.\n   Regarding my following two questions, there might seem to be some cases in\n   which there is no clear benefit from explicitly attributing positional\n   embeddings. These points make the results somewhat inconclusive, as the work\n   does not provide any analysis as to when the proposed approach should be\n   used, and when it should not. The qualitative results also do not seem to\n   give a clear difference. The work would be much more convincing if it could\n   provide the number of trials, and conduct a Wilcoxon-Rank-Sum test to\n   provide more conclusive evidence in favor of attributing positional\n   embeddings. It would be immensely helpful for the community to understand\n   when and why we should consider positional embeddings for LRP, to which I\n   was not entirely able to convince myself based on the results.\n\n2. For Table 1, the work states that the proposed approach lags behind for the\n   positive perturbation, but performs better in the negative perturbation. This\n   seems to suggest that for some setups, it is better to not regard the\n   positional embedding. What are the implications of this observation? How\n   significant are these results?\n\n3. In Table 3, PE seems to outperform both AttnLRP as well as AttnLRP+PE on\n   Tiny LLaMa. However, adding AttnLRP+PE does not seem to improve\n   significantly over AttnLRP. Do you have any idea why this could be the case?\n\n4. I am sceptical of how useful Lemma 3.1 is. As far as I understand, AttnLRP\n   assumes the feature attribution only for the sum $R_E + R_P$, where the\n   positional part is simply by design part of the input. I do understand that\n   there are some limitation to not model the positional embedding explicitly,\n   which this work empirically evaluates. However, I do not find this\n   conservativity argument meaningful for understanding the distinction between\n   handling and not handling positional embeddings. I also do not think that\n   Lemma 3.2 is very useful, as Eq. 9 is per definition conservative.\n\n5. Lemma 3.3 claims that in general, current LRP attribution rules (i.e.,\n   AttnLRP) provide low faithfulness for attention-level PE transformers with\n   respect to positional features. This lemma is not proven, but rather only a\n   specific extreme-case example is provided. While the lemma could be\n   reformulated, I do not think it makes a particularly convincing argument as\n   to when it is useful to regard positional embeddings, and when it is not.\n\n\nI would like to emphasize that I think it is helpful to consider positional\nembeddings in LRP. However, I think that there are use-cases for which\npositional embeddings do and do not matter. This works in its current state\nseems to be convinced that positional embeddings should always be regarded,\nalthough empirical experiments do not seem fully conclusive on this idea. The\nwork would be much stronger if it analyzed the idea that the usefulness of\npositional information might be conditional, and explored cases for which it\nmight not. With the current state, I am leaning towards reject, but could be\nconvinced the empirical experiments would be analyzed for statistical\nsignificance, as well as regard cases in which attributing positional\nembeddings might not be useful.\n\nMinor:\n- I think it might be more descriptive to call the \"Lemmas\" propositions\n  instead. Lemmas usually aid in the proof of some Theorem, which is not the\n  case in this work.\n- I think the notation in Appendix E is overly simplified, especially\n  considering the sums without subscripts. I would also recommend to use the `\\text{input}` etc.\\ for descriptive subscripts.\n\nLIMITATIONS:\nThe work states that it is not a limitation that it does not analyze existing LRP rules.\nWhile I agree with this statement, I do not think it provides any insights into\nthe actual limitations of this work.\nIn its conclusion, the work describes a drastic improvement of XAI when\nconsidering positional information. I think a more interesting limitation would be to describe cases where this positional information could be even misleading.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}