The Fragile Truth of Saliency: Improving LLM Input Attribution via Attention Bias Optimization
Authors: Yihua Zhang
Changsheng Wang
Yiwei Chen
Chongyu Fan
Jinghan Jia
Sijia Liu
Venue: NeurIPS 2025 spotlight ()
Keywords: LLM Input Saliency
Primary area: deep_learning
TL;DR: We introduce a long-context stress test revealing that existing saliency methods fail under scale, and propose ABO—a simple, causal, and robust alternative based on attention bias optimization.

Abstract: Input saliency aims to quantify the influence of input tokens on the output of large language models (LLMs), which has been widely used for prompt engineering, model interpretability, and behavior attribution. Despite the proliferation of saliency techniques, the field lacks a standardized and rigorous evaluation protocol. In this work, we introduce a stress-testing framework inspired by the needle-in-a-haystack (NIAH) setting to systematically assess the reliability of seven popular input saliency methods. Our evaluation reveals a surprising and critical flaw: existing methods consistently assign non-trivial importance to irrelevant context, and this attribution error worsens as input length increases. To address this issue, we propose a novel saliency method based on Attention Bias Optimization (ours), which explicitly optimizes the attention bias associated with each input token to quantify its causal impact on target token generation. ABO robustly outperforms existing methods by 10\sim30% in saliency accuracy across diverse NIAH tasks, maintains effectiveness up to 10K-token prompts, and enables practical applications including zero-shot detoxification, sentiment steering, and reasoning-error correction. Our findings highlight the limitations of prevalent attribution methods and establish ABO as a principled alternative for accurate token attribution.

The Fragile Truth of Saliency: Improving LLM Input Attribution via Attention Bias Optimization
Yihua Zhang
Changsheng Wang
Yiwei Chen
Chongyu Fan
Jinghan Jia
Sijia Liu
Michigan State University
Abstract
Input saliency aims to quantify the influence of input tokens on the output of large language models (LLMs), which has been widely used for prompt engineering, model interpretability, and behavior attribution. Despite the proliferation of saliency techniques, the field lacks standardized and rigorous evaluation protocol. In this work, we introduce a stress-testing framework inspired by the needle-in-a-hay-ee (NIAH) setting to systematically assess the reliability of seven popular input saliency methods. Our evaluation reveals a surprising and critical flaw: existing methods consistently assign non-trivial importance irrelevant context, and this attribution error worsens as input length increases. To address this issue, we propose novel saliency method based on Attention Bias Optimization (ABO), which explicitly optimizes the attention bias associated with each input token to quantify its causal impact on target token generation. ABO robustly outperforms existing methods by 10 ~ 30% in saliency accuracy across diverse NIAH tasks, maintains effectiveness up to 10K-token prompts. and enables practical applications including zero-shot detoxification, sentiment steering, and reasoning- error correction. Our findings highlight the limitations of prevalent attribution methods and establish ABO as principled alternative for accurate token attribution.
1 Introduction
What makes particular word in a prompt important a large language model (LLM)) This question lies at the heart of input saliency analysis, a field that seeks to quantify how much each input token contributes to the model's output [1-8]. As L.MMs become increasingly embedded in downstream tasks ranging from coding assistants [9, 10] open-ended reasoning agents [11], understanding their decision- decision-making process is essential for safety, interpretability, and controllability [12, 13]. Input saliency provides foundation for prompt engineering by identifying which tokens most influence the model's behavior [14, 15], and it offers researchers a window into the inner workings of attention mechanisms and token interactions [16, 17J. As such, saliency analysis has emerged cornerstone in the interpretability toolkit for modern language models [18-20].
Diverse input attribution techniques already exists, spanning perturbation-base masking [7, 21]. gradient and interaction methods [8, 12], vector-space decompositions [17, 22], Shapley values [23], and linear approximations [12, 24]. Each line of work claims to reveal the "true" importance of input tokens. Yet the field lacks a rigorous, model-agnostic benchmark for judging whether these scores separate meaningful signal from contextual noise, especially in the long- long-context prompts that define contemporary LLM use. Without such benchmark, practitioners risk acting on attribution maps that noisy. unstable, flat-out wrong.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

To close this gap we design a stress test inspired by the classic "needle-in-ate in-a--yystacc" (NIAH) evaluation [25]. We embed single secret message inside span of distractor text and ask the model to reproduce that message verbatim. Because the distractors have zero causal influence on the output, the ground-truth saliency distribution must concentrate on the secret message (and the final query). This controlled setting lets us audit attribution methods with surgical precision. Our evaluation of six popular techniques [1-6] uncovers a systemic flaw: every method assigns a substantial fraction of saliency to irrelevant tokens, which grows the context length increases. The misattribution reaches over 90% of the total score for 10K-token prompts, implying that the interpretability tools most widely used today can be actively misleading in long- cong-conext scenarios.
Motivated by the observation that reweighting an imput token's 's attention can deterninistically sway the model's 's output when two contradictory messages are present, we recast attribution as an optimization over attention biases. ABO injects a small, learnable bias term into each attention head and solves for the minimal intervention that maximizes the probability of a target output token. The resulting bias magnitudes serve as faithful saliency scores: they are causal by construction, differentiable, and inexpensive to compute. Empirically, ABO withstands our NIAH stress test and remains robust up to 10K-token inputs. Beyond evaluation, ABO proves useful for zero-shot detoxification, sentiment steering, input token pruning, and rectifying errors in the reasoning traces. To summarize, our contributions are as follows:
We introduce a stress-testing framework for LLM saliency based on the needle-in-at in-a-haystack task, enabling ane-ggrined diagnosis under long-context inputs.
We reveal systemic reliability failures in six widely-used attribution methods and show that misattribution to irrelevant tokens exceed over 90 % in 10 K-token prompts, quantifying the brittleness of existing saliency tools.
We propose Attention Bias Optimization (ABO), principled, optimization-basee based technique that delivers token-level saliency scores with superior causal fidelity and scalability to long-contexts.
We demonstrate ABO's broad practical utility across sentiment control, toxic prompt detoxifica- tion, and LLM error correction, underscoring its value for both research and roal-world deployment.
2 Related Work
LLM input saliency methods. Understanding how large language models (LLMs) attribute gen- crated outputs to their preceding input tokens is crucial for model interpretability, debugging. and effective prompt engineering. Several attribution methods have been proposed elucidate these rela- tionships, typically categorized into gradient- based, vector-based, and perturbation-basee approaches. Cradient-based methods measure token importance via gradients obtained through backpropagation. Examples include gradient norm [26-28], gradient-input production [29-31]. Layer-wise Relevance Propagation (LRP-XAD) [32, 33], and Generic Attention Explainability (GAE) [8]. Although these methods have demonstrated their value across diverse NLP models and tasks, they predominantly rely on local linear approximations, potentially accumulating errors in deeply stacked transformer layers. Vector-based methods leverage internal model representations, typically attention weights, to infer token importance, exemplified by Attention Rollout [34], GlobEnc [35], and ALTI [36]. However, these approaches implicitly assume linearity in attention score combinations across layers, limiting their fidelity as model depth and complexity increase. Alternatively, perturbation-bas based techniques [23, 37, 38] measure token significance by assessing changes in model confidence upon token removal or masking. Representative methods include AtMan [6], which combines perturbation strategies with attention mechanisms specifically optimized for transformer models. The evalua- tions of existing methods typically involve short-context scenarios with limited causal interactions. Consequently, existing benchmarks neither sufficiently explore attribution accuracy under realistic long-context scenarios nor rigorously stress-test the methods' robustness against irrelevant contextual noise. Our work fills this critical gap by introducing a systematic long-context stress-test framework. providing insights into the limitations of current attribution methodologies when applied to LLMs.
Controlled generation and behavior steering for LLMs. A large body of work seeks to steer language- model behavior at inference time via prompt design. Manual and automated prompting tech- niques have shown striking gains on downstream tasks [39-41], and toolkits such as PROMPTSOURCE, RLPrompt, and AutoPrompt make cue construction more accessible [42-46]. Yet LLMs remain

---

[System Prompt] You are a helpful Al chat bot that answers questions for a user. Keep your response short and direct. Don't 1 present information outside the document below or repeat your findings ... [ong Irrelevant Context] July 2006... imagine you were going to spend the weekend at a friend's house on a little island of the coast of Maine... (irrelevant text continues)... Quiet is another malter. [Secret Message] The password to get into the zone of Hugazin is "Yin" [Irrelevant Text Resumes] realize it seems a bit eccentric to take earplugs on a trip to an island off the coast of Maine. If anywhere should be quiet, that should. But what the person the next room snored? ... Pen and paper wick ideas. [Query on Secret Message] What is the password to get into the zone of Hagazin? Your answer is:
Figure 1: Color-coded wecdle-in-a-hassteed (NIAH) [25] prompt. System instruction, distractor text, secret message, and final query are visually distinguished. When calibrating an LLM saliency method with this example, only saliency score assigned to the non-gray regions (yellow, green. blue) meaningful: any score allocated to gray tokens reflects incorrect attribution.
brittle: small lexical changes can flip predictions or derail chain-of chain-of-thoughtt reasoning [47, 48]. Com- plementary to prompt engincering, training-time time interventions, c.g. instruction finetuning [49, 50], RLHF and its variants [51-53], or loss terms that privilege mser-marked tokens [54-57], improve overall steerability but require additional data or compute. Parallel work explores lightweight adapta- tion via low-rank or quantized updates (LoRA, AdaLoRA, QLoRA, TOAST) [58-61], and targeted knowledge editing with ROME, MEMIT, MEND, or REMEDI [62-65] Our saliency-base Attention Bias Optimization is orthogonal to these approaches: leaves model weights unchanged, requires no extra supervision, and instead manipulates generation by amplifying or suppressing precisely the tokens that the model itself deems most causal.
3 A Stress Test of LLM Input Saliency Methods
While input saliency methods have become ubiquitous tools for interpreting LLM predictions, it remains uncertain whether these attribution methods accurately capture token importance under realistic, long-context scenarios. Despite numerous techniques proposed in the literature, their evalua- tion often relies on relatively simplistic or short-text scenarios [1-3], leaving their generalizability and reliability largely unchecked. To bridge this gap, this section introduces novel stress-testing framework that systematically probes the robustness and reliability of existing saliency methods under increasingly challenging input conditions. Our aim is twofold: first, reveal critical limitations of current attribution techniques when exposed to long-context inputs; second, to motivate the necessity of fundamentally more robust attribution strategy.
The needle-in---hhysstcck (NIAH) test for saliency assessment. Inspired by the classic needle- in-a-haystack paradigm [25], originally developed to assess the information-rettiivall capabilities of LLMs in long-context settings, we adapt this experimental setup explicitly for the evaluation of saliency methods. Our adapted NIAH scenario is carefully constructed to provide explicit, ground-truth saliency distribution for rigorous evaluation. Specifically, as illustrated in Fig. 1, we craft prompts comprising four core components:
a system prompt to establish the overall context, role, and behavior for the LLM's responses;
a lengthy distractor passage sampled from diverse, public-domain texts (e.g., news articles, Wikipedia entries), explicitly designed to provide irrelevant contextual noise;
short, deliberately constructed embedded secret message, which contain keywords (e.g., "Hagazin" and "Yin" in Fig. 1) are intentionally chosen to be vocabulao words absent from standard LLM pre-training corpora, ensuring the uniqueness and singular salience of this message;
a precise, targeted query placed at the end of the input, asking questions on the secret message hidden in the context.
Before the evaluation, we first make sure the LLM's answer to the query is correct. In the case above, the answer should be "Yin". Then, we use the saliency method to generate saliency scores to all the input tokens with respect to the target output token (i.e., "Yin"). The central evaluation metric employed throughout this analysis is the Hit Ratio:
Hit Ratio = Saliency Score(system) prompt + secret message + query)
Total Saliency
3

---

Intuitively, higher Hit Ratio approaching 1.0 indicates that method effectively isolates truly salient tokens, while lower scores indicate failure in accurately discriminating signal from noise. To ensure fair comparisons across varying methods (each with potentially different saliency scaling conventions), we normalize all saliency vectors to sum to one prior to aggregation.
Why is NIAH naturally a better fit for saliency calibration? Critically, the nature of this task involves pure information retrieval and inferential reasoning required. Consequently, the ground- truth saliency distribution should strietly allocate importance to three well-defined input regions: (1) the initial system prompt (if present), (2) the embedded secret message, and (3) the final query, All other input tokens, by experimental construction, are irrelevant and thus should be assigned negligible saliency scores. Furthermore, following the original NIAH setting [25]. we systematically vary the length of the distractor text from 500 tokens (contexts) up to 4K tokens (contexts), effectively modulating the signal-to-noiss ratio within the input while keeping the informative content fixed. This controlled manipulation of input length provides a straightforward yet rigorous means to evaluate each method's 's robustness to the increasing complexity of realistic long-contex scenarios.
Examination of LLM input saliency via NIAH. To ensure comprehensive coverage, we evaluate broad spectrum of seven representative saliency methods widely adopted in the literature. These methods span diverse methodological paradigms including gradient-based (GRADNORM [4], INPUT GRAD [5]), attention-derived (ATMAN [6]), token distribution-based (TDD [2]), Shapley value- based (TOKENSHAP [I]). and linear modeling (AT2 [3]). As a reference, we also compare these methods with random guess method (NAIVE), which assigns each input token equal saliency. Detailed algorithmic descriptions and hyperparameter selections for each method are provided in Appx. A to facilitate reproducibility and transparency. Our experiments are primarily conducted on frozen LLaMA-2-77 [66] backbone to ensure controlled comparisons.
Fig. presents comprehensive visualization of Hit Ratio perfor- mance as a function of input length. Two critical observations im- mediately emerge: First, there is steep accuracy collapse at long
60
GradNoom
Rollout
40
Token Shopley
contexts. All evaluated methods experience a precipitous drop in attribution fidelity once the input exceeds approximately 1K tokens.
Ratio
30
AtMan
TDD
AT2
20
Alarmingly, at input lengths beyond 2K tokens, several methods approach random performance (represented by the gray dashed base- line), highlighting their severe inability to cope with realistic, noisy, long-text inputs. Second, there non-trivial misattribution even short contexts. Surprisingly, even at the shortest tested context (100 tokens), over half of the attribution mass is consistently misallocated to irrelevant tokens. This persistent misattribution underscores that the root cause is not merely extreme input length, but fundamental vulnerability in existing methods to structural or positional biases.
10
500 1000 I 1500 2000 3000 4000 Context Length
Figure 2: Hit ratio of existing saliency methods stress testing with different input length. Hit Ratio is the fraction of saliency scores assigned to the true causal tokens.
the NIAH
To understand the underlying causes of misattribution, Fig. 3 provides a qualitative heatmap of saliency distributions for representative examples. Strikingly, misattribution does not manifest as
[System Prompt] You _are___________ _a _helpful LA chat bot _that__ ansswrs qqesstions _or _a _user Keep your response .short and direct Don 't present information outside the document _ below _or _rreeat__ _your_ finnd ings ... [Long Irrelevant Context] July 2006... imagine you were going to spend the week end at La friend 's house on little island off the coast of Maine Qu iet another .mttter
[Secret Message] The password to get into the zone of H Ha az is Yin [Irrelevant Text Resumes] I realize it seemss _ bit _ecc ric _to lake eaar pl ugs too an jisland _off _te _coast _of |Maine| [If _anywhere _should _be _quict, |_that _should _But _what jf the _persoo _in _th _next_ _room _sn ored _ ..._Pen _and _paper _w ick _ideas
.tri
[Query on Secret Message] What is the password to get into the zone of H ag az
Your answer is:
Figure 3: Token-level saliency heat-map for a NIAH prompt obtained using AT2 with the context
length of 1K. Each word is background-shaded according to the saliency score (darker=higher). The map reveals substantial saliency weight spilled onto distractor passage, notably fighffrequency function words and punctuation.

---

[System Promptj You are a helpful... or repeat your finaings... [Long Irrelevant Context] July 2006... imagine you... (irrelevant
continues)... Quiet another matter. [Secret Message I] [Irrelevant Text Resumes] I realize it seems bit eccentric... (irrelevant text continues)... ... Pen and paper wick ideas. [Secret Message 2] [Irrelevant Text Resumes] hardly ever go buck and read stuff ... (irrelevant text continues).... ... way of having the next. [Secret Message 3] [Irrelevant Text Resumes] The best notebooks I 've found... ... (irrelevant text continues).. ... out of space, like a Latin inscription. [Query on Secret Message] What is the password to get into the zone Hagazin? Your answer is:
Figure 4: Contradictory NIAH prompt used for the attention-bias study. The long input color-segmented into a system instruction, alternating blocks of irrelevant context, and three mutually conflicting secret messages, message I. message and message 3, followed by the final query. The LLM chooses only one secret message to answer the query.
uniform randomness; instead, existing methods disproportionately latch onto structurally salient yet semantically irrelevant tokens, such as high-frequency function words ("the", "of") or syntactic boundaries (sentence breaks, paragraph separations). This critical observation highlights a fundamental flaw: existing methods systematically confuse tokens that dominate attention structure due to frequent repetition or positional prominence with tokens genuinely contributing causally the model's 's output. This exposes a systemic weakness of current input-salln input-saliency techniques: they conflate attention structure with causal contribution, and the error amplifies with additional distractors.
4 Saliency-Biased Attention for Token Attribution
In Sec. 3, we revealed a critical flaw underlying existing saliency methods: their interpretability severely degrades in long-context scenarios. This limitation naturally raises an important question: Can we not only diagnose these attribution errors but actively intervene to rectify them? To address this, we revisit our NIAH paradigm, this time introducing a novel and provocative modification by embedding multiple contradictory messages into the input.
Contradictory secrets reveal attention bias dynamics. We begin by embedding three conflicting secret messages into the input context: Message 1. The password to get into the zone of Hagazin is "Yin", Message 2. The password to get into the zone of Hagazin is "Vin". Message 3. The
password to get into the zone of Hagazin is "Kin" ". See Fig. 4 as an illustration. At the conclusion of this deliberately confusing passage, we pose the query: "What is the password to get into the zone of Hagazin?" ? Interestingly. we observe a pronounced positional bias in the model's behavior: regardless of their semantic content, LLMs consistently favor the first encountered secret message. This phenomenon suggests that token saliency. model attention toward particular input tokens, may be implicitly influenced by their position rather than their semantic content alone.
Using attention bias to manipulate LLM's behaviors. Inspired by this intriguing observation, we probe deeper by deliberately manipulating the model's attention logits, i.e., the intermediate values computed before the softmax normalization in the attention mechanism. Recall that in a
Oulput Token Tokon Probability (%)
100
P("Yin) "nnn''-st message
80
P("Vin')2nd message (biased)
60
40
standard transformer, the attention score (also called atten- tion logits) between a query qi and a key k, is computed
20
as: Lij q: kj, where de is the key dimensionality, Vdl These logits are then passed through a softmax to yield
0
1.0
1.2
1.4
1.6
1.8
2.0
Applied Bias towards 2nd Message
Figure 5: LLM's prediction can be shifted by biasing attention scores. Curves track the probability that the LLM will output the first-secret token "Yin" vs. the second-secret token "Vin", when positive attention bias is gradually injected toward the second secret message. Increasing the bias drives P(Vin) upward from ~ 14% to 40%, while P(Yin) falls symmetrically, and the two curves cross once the model's ''s final answer flips from the first to the second.
attention weights. In order to investigate how attention scores might influence LLM's outputs, we inject an arti- ficially positive attention bias toward tokens in the sec- ond secret message, gradually increasing its magnitude: Lij + Lij + aj, where [j] represents the input token IDs designed to be enhanced with this bias. Fig, 5 vividly illustrates this experiment: as the bias towards the second secret grows, the probability of outputting the second mes- sage's 's password ("Vin") consistently increases, while the probability associated with the first message's password
5

---

("Yin") correspondingly decreases. Ultimately, the model's final prediction shifts from "Yin" to "Vin". This simple yet profound experiment reveals deeper insight: attention logits directly govern token saliency and thus strongly influence model outputs. Consequently, attention biases can serve as a powerful lever for interpreting and controlling L.LM behavior.
From manipulation to attribution: Attention bias optimization (ABO). Motivated by the above experiment, we propose ABO, novel attribution framework. The central insight is straightforward yet compelling: enhancing a token's attention logits reliably boosts the model's probability of generating a target output, it logically follows that the token is intrinsically salient to that output. Leveraging this principle, ABO reframes token attribution as a differentiable optimization problem over attention logits, thus providing smooth and precise alternative to discrete masking traditional tradient-based attribution approaches.
Formally, given an input sequence x (I), T2............. Fr.) of length L and target output token Vtarget, we introduce learnable attention biases. Specifically, we assign each input token I; a scalar bias parameter og, shared across all attention heads and layers. After injecting these biases, we apply standard softmax normalization to yield final attention weights. This soft re-whighting ensures that token is completely discarded, thus preserving the integrity of the full context. To derive meaningful, interpretable saliency scores, we optimize the parameters {ag} by maximizing the biased model's probability of producing the target token Vtarget
[(a)= = - log Phias (Htarget X, ax) + (as).
(2)
Here, the first term ensures tokens that genuinely contribute generating Vtarget receive greater saliency, The second term is carefully designed regularizer: by applying a sigmoid transformation to the biases, it penalizes excessive saliency across irrelevant tokens and promotes sparsity. This choice ensures numerical stability during optimization and yields a sparse, interpretable saliency distribution that highlights genuinely influential tokens. Unlike discrete token masking, ABO never removes tokens but instead smoothly reallocates attention weights across the entire input sequence. This soft, continuous approach provides clearer interpretability: increased bias for a token directly translates to increased model attention and influence. Furthermore, ABO addresses key drawbacks of previous methods, such as instability gradient-based attribution and arbitrary saliency leakage, by directly coupling token saliency to the model's internal attention dynamics. Consequently, it consistently provides interpretable and robust attributions, even across extremely long contexts.
Isolating the learned bias yields a clearer and more causal attribution signal. An important design choice in ABO lies in measuring attribution through the optimized attention bias rather than the sum of the original attention value and bias. While both components jointly influence the model's output during inference, isolating the learned bias yields cleaner and more causally interpretable signal. Original attention distributions span heterogeneous heads and layers, each capturing structural or positional regularities that may not directly reflect causal relevance. Aggregating them with the optimized bias would inevitably entangle genuine influence with such confounding patterns, thereby weakening attribution clarity. In contrast, the learned attention bias vector forms unified and end-to-end optimized space that quantifies the minimal perturbation required for a token to affect the model's 'ss output. This separation allows ABO to measure causal sensitivity in stable and comparable manner across contexts, avoiding complex normalization or scale mismatches that arise from mixing heterogeneous attention values.
Bias sharing across layers and heads stabilizes optimization and improves practicality. Another design decision concerns sharing the bias parameters across all attention heads and layers. Early variants of ABO explored assigning independent bias parameters each head and layer, but this additional flexibility did not improve attribution fidelity. Instead, it introduced substantial optimization instability, as interactions among layers amplified local gradient noise and slowed convergence. Empirically. the delivered consistently higher saliency precision under equal or lower computational cost. Beyond performance considerations, parameter sharing also enhances the method's 's practicality, reducing memory footprint, simplifying implementation, and enabling ABO to serve as a lightweight plug-in interpretability module applicable to diverse transformer architectures.
formulation achieved more stable optimization dynamics and

---

Table 1: Hit Ratio on the single output token NIAH stress test across different LLMs and context lengths. Each cell reports the percentage of saliency mass correctly assigned to the true causal tokens (system prompt + secret message + query), averaged over multiple secret-message insertion depths (from 10% 90%) within each context and normalized to sum 100. Higher is better.
L.LMMA-7B [66]
Cestral-B-Instrurce
[67]
DeepSeek- - D-Dsttilled Beeetilld--@en--lee WWnn-14B [68]
Method
IK
2K
3K
4K
IK
3K
5K
10K
IK
3K
5K
10K
NAIVE
7.50
2.50
1.88
7.20
2,40
1.44
0.72
0.82
GRADNORM
11.49
8.21
4.44
2.28
10.22
3.23
2.41
1.17
9.35
2.91
2.32
1.43
INPOT X GRAD [5]
12.20
4.44
3.49
1.68
13.58
3.31
2.28
1.81
10.27
2.96
2.56
1.45
TOKENSHAPP
17.73
14.82
5.24
15.54
10.27
6.37
3.73
16.73
11.43
5.59
3.81
TDD [2]
15.57
10.83
7.29
3.37
13.28
9.23
5.51
4.39
12.71
7.68
4.83
3.51
ATMAN [6]
19.94
4.34
20.22
13.33
7.18
3.12
18.88
2.83
AT2 [3]
21.15
18.27
12.28
6.62
20:43
14.41
10.03
5.59
21.76
13.65
9.86
4.99
ABO
41.75
31.72
25.54
18.83
45.34
27.77
18.92
15.43
38.89
25.14
17.37
15.44
5
Experiments
To systematically validate the accuracy, robustness, and utility of our proposed ABO method, conduct systematic and comprchensive experimental evaluations. The first part (Sec. 5.1) focuses on quantitatively comparing the attribution accuracy of ABO against several strong baselines in controlled scenarios, including single-token and multi-token NIAH stress tests, as well as an input token pruning evaluation in the long-context setting. The second part explores broader applications of ABO, illustrating how accurate saliency attribution can facilitate model behavior manipulation (Sec. 5.2 and Sec.5.3), 5.3), and error correction on LLM's input reasoning tasks 5.4).
5.1 Saliency Attribution Accuracy
Single-token NIAH attribution analysis. First, we evaluate methods using the previously described single-token NIAH setup, measuring their Hit Ratio, i.e. the fraction of saliency correctly attributed to truly causal tokens. As summarized in Tab. 1, ABO consistently achieves substantial improvements over all baselines across different models and context lengths. Crucially, ABO maintains robust Hit Ratio of approximately 15- ~ 45%, even as input length expands from 1 to 10K tokens, while the strongest baseline (TOKENŠHAP & AT2) rapidly deteriorates to below 5%. This stark contrast underscores ABO's unique resilience in handling long-context noise environments. Additionally, ABO's superior performance generalizes robustly across architectures, indicating that its advantage stems fundamentally from its principled attention bias optimization mechanism, rather than model- specific artifacts.
Multi-token NIAH attribution analysis. Second, to challenge the saliency methods further, extend the NIAH setting to secret messages containing multiple target tokens, requiring methods to correctly distribute saliency across multiple output tokens. Results presented in Tab. 2 affirm ABO's clear superiority: although moving from single-token to multi-token attribution naturally reduces Hit Ratios slightly for all methods, ABO experiences the mildest performance degradation (around 4 ~ 5%), maintaining approximately 13 ~ 40% accuracy. In contrast, leading baselines suffer more substantial drop towards below 10%, highlighting ABO's superior capability to capture meaningful attribution across multiple output tokens simultancously. The persistent robustness of ABO even under increased complexity strongly advocates for its suitability realistic applications, where multiple tokens frequently contribute jointly to model decisions.
Long-context token pruning evaluation. We further validate saliency accuracy in practical sce- narios by conducting input token pruning experiments on the challenging LongBench-2wiikimge 2wwkkimqa_ dataset [69]. Tokens are pruned in ascending order of estimated saliency, with QA accuracy measured at progressively higher pruning ratios (see Tab. 3). Remarkably. ABO outperforms all baselines, especially under aggressive pruning conditions. For example, at a 95% pruning ratio (retaining only 5% of tokens), ABO achieves QA accuracy 51.2% on QWen3- 144B, significantly higher than the best baseline's 's 36.2%. Furthermore, ABO exhibits flatter accuracy degradation curve across all models tested, demonstrating a stronger ability to reliably identify essential context information. These results confirm ABO's practical value: not only does it accurately reflect token importance,
7

---

Table 2: Hit Ratio on the NIAH stress test across with mutiple output tokens different LLMs and context lengths. Other settings follow Tab. 1.
LLaMA2-7B [66]
Mistral-7B-Iane -B--nssruuctt-O.. [67]
DeepSeck-ER REl--Dstilled-OO Wennlled [68]
Method
1K
2K
3K
4K
IK
3K
5K
10K
1K
3K
5K
10K
NAIVE
7.50
3.75
2.50
1.88
7.20
2.40
1.44
0.72
8.20
2.73
1.64
0.82
GRADNORM [4]
8.39
6.04
3.70
2.12
9.28
3.11
2.20
1.75
8.87
2.69
1.89
1.11
INPUT GRAD [5]
8,73
5.45
3.51
2.03
8.85
2,98
1.92
1.56
9.17
2.82
2.15
1.15
TOKENSHAP [1]
12.13
9.91
4.12
3.98
12.25
10.38
5.73
3.31
11.42
5.51
4.11
2.35
TDD [2]
10.89
8.73
4.31
3.29
9.87
8.82
3.89
2.17
9.91
4.16
3.06
2.05
ATMAN [6]
13.27
10.29
4,98
3.71
15.55
12.27
7.73
5.41
10:40
4.41
3,71
1.51
AT2 [3]
15.37
13.39
5.32
3.19
2.86
ABO
38.32
28.99
21.45
14.49
40.93
24.49
14.37
13.18
34.79
21.19
15.45
13.61
but it also provides actionable insights enabling substantial runtime efficiency by safely pruning non-essential tokens.
Table 3: QA accuracy (%) on the LongBench-2wikke
e multi-choice task after pruning input tokens based saliency scores. Each column indicates the pruning ratio.
QWen3-8B
Mistral-Small --all----l------- 141---44----ssttt-----
QWen3-1411
Method
10%
50%
90%
95%
97%
10%
50%
90%
97%
10%
50%
90%
95%
97%
NAIVE
48.0
39.5
5.3
0.6
0.0
54.4
42.8
4.3
0.0
0.0
59.0
45.5
3.6
0.0
0.0
GRADNORM
58.6
56.9
35.4
8.9
3.2
66.0
65.3
38.1
9.4
3.6
80.5
79.6
41.0
10.1
4.2
INPUT X GRAD
58.4
56.7
36.0
66.9
65.1
3.3
80.2
79.3
41.2
9.8
3.8
TDD
58.5
55.6
33.2
7.2
2.4
67.6
63.9
36.8
8.3
2.9
78.4
77.3
39.1
9.1
3,5
ATMAN
59.1
58.6
44.8
28.2
5.2
67.1
66.5
49.5
31.3
8.3
81.8
80.9
54.1
34.1
9.1
AT2
59.4
58.9
46.3
30.0
7.8
67.4
66.9
32.5
8.7
82.1
81.2
56.0
36.2
9.5
Ours
59.3
59.0
52.6
40.3
9.9
67.5
67.1
58.7
47.0
12.2
82.3
81.7
65.1
51,2
15.6
5.2 LLM Saliency for Zero-Shot Toxicity Suppression.
We further demonstrate the practical utility of accurate saliency attribution through the task of zero- shot toxie language suppression. Despite recent advancements, LLMs remain prone to generating toxic outputs in response to certain inputs [70, 71]. Unlike prior reactive approaches aimed at mitigating toxicity post- cest-eeneration [72-74, 74], our approach proactively identifies and removes toxic triggers directly from the prompt, ensuring safer downstream generation.
Table 4:
Toxicity suppression results REALToXIC [72] using Mistral-7B 7B-Instruct-vO.. [67]. Scores on six toxicity attributes provided by the Perspective API; lower values indicate better suppression.
Method
Toxicity
Severe Toxicity
Sexually Explicit
Threat
Profanity
Identify Attack
Original
0.53
0.24
0.37
0:23
0.42
0.19
TDU
0.36
0.21
0.29
0.18
0.33
0.14
TOKENSHAP
0.41
0.18
0.17
0.14
0.26
0.12
AT2
0.15
0.21
0.11
0.14
0.09
ABO
0.24
0.12
0.14
0.08
0.07
0.05
Experiment setup. Specifically, we utilize the REALTOXICPROMPTS [72] dataset, comprising 1,225 prompts known for eliciting highly toxic outputs, and evaluate method using the Mistral- IB-IInttrut--OO.3 model [67]. Given our rigorous assessment of saliency attribution methods in the previous subsection,
selectively benchmark against the nop-performing saliency approaches,
namely TOKENSHAP [1], AT2 [3], and TDD [2]. During the experiment, we first designate the predefined toxic words from WORDFILTER [72] as target tokens, identifying critical tokens (toxic triggers) via saliency scores, which we subsequently neutralize by replacing them with meaningless space tokens. Following the procedure of prior work [73, 75], we limit generations 20 tokens and evaluate the resulting toxicity using six toxicity attributes provided by Perspective API, ensuring a clear and rigorous comparison of sidencc-bbseed toxicity suppression efficacy among our chosen baseline methods.
Results and analysis. As shown in Tab. 4, our method achieves the lowest toxicity scores across all six attributes, substantially outperforming all baselines. Compared to the original outputs, ABO
8

---

Table 5: Sentiment steering results on OPENWEBTENT [76] using Mistral- -B- I-stttucttttO.3 [67]. Generated outputs classified via Huggingface sentiment analysis [77]: higher indicate stronger steering efficacy.
Neutral
Negative Negative Ratio (%)
Neutral Positive
Method
Positive Ratio (%)
Original
53.6
46.4
75.4
59.9
TOKENSHAP
80.2
68.3
AT2
69.3
66.7
ABO
89.7
81.5
reduces overall toxicity by more than half, and consistently yields the strongest suppression in severe categories such as threat, profamity, and identity attack. While AT2 emerges as the strongest baseline, it still lags behind across every metric. These results confirm that our saliency estimates more accurately pinpoint true toxic triggers, enabling more effective prompt sanitization and offering a robust foundation for safe LLM deployment.
LLM Saliency for Sentiment Steering
We next demonstrate ABO's applicability in zero-shot sentiment steering, an essential task for safely and reliably guiding LLM-generated content towards a desired sentiment polarity.
Experiment setup. To rigorously assess our approach, we use 5,000 neutral prompts sourced from the OPENWEBTERS [76] corpus feed the we designate negative sentiment words from SENTICNET [78] as target tokens and positive words as alternative tokens when steering towards positive sentiment, and reverse this assignment when steering negatively. Subsequently, we utilize saliency attribution methods, focusing exclusively on the bighest-pefforiing
Inntrrrtctttt- model [67]. Specifically,
methods identified previously: TOKENSHAP [1]. AT2 [3], and ATMAN [6] within each prompt. This identified token is then
to pinpoint the single most salient sentiment directly replaced by the explicit keyword (either "positive" or "negative" accordingly) to guide the model's sentiment. Following prior work [79], each prompt undergoes exactly one token replacement due to their relatively short lengths. For evaluation, we exclusively report the sentiment distributions of generated outputs using the Huggingface sentiment analysis classifier [77], clearly reflecting the efficacy of each method in steering sentiment polarity.
Results and analysis. As shown in Tab. 5, single ABO-guided token edit shifts 99.7% of neu- tral prompts negative and 81.5% positive sentiment, outperforming the strongest baseline (TOKENSHAP: 80.2% 91%//68.33 /6..3%). The gap widens in the harder "neutral-ppositvv" scenario, highlight- ing ABO's ability pinpoint the single most causal sentiment cue. Together, these results confirm that bias optimization yields sharper, more actionable saliency than existing heuristics, enabling reliable shot sentiment control with minimal input modification.
[Question] In dance class of 20 students, 20% enrolled in contemporary dance, 25% of the remaining enrolled in jazz dance, and the rest enrolled in hip-hop dance. What percentage of the students enrolled in hip-hop dance?
[Reasoning (Incorrect by Default)] <think To determine the percentage of students en- rolled in hip-hop dance, let's break down the information provided: Students en- rolled in Jazz Dance: 25% of 20 students 0.25 x $ students \] <lthink>
[Reasoning (Correct with Attention Bias)] <think> To determine percentage Students enrolled in Jazz Dance: (2%% ) of the remaining students after contemporary dance 20 4 = 16 students \j Then, 25% of 16: 0.25 16 4 students \... <lthink>
Final Answer: (bboxed5%%A
Final Answer: (\bxxe669%3
Figure 6: Attention bias fixes a reasoning error. Highlighting the token "remaining" (yellow) and adding a positive bias to its attention logits redirects the model to the correct interpretation, changing the wrong 55% answer (left, red) to the correct 60% answer (right, green).

---

5.4 LLM Error Correction via ABO
Beyond interpretability and behavior steering, our ABO (attention bias optimization) approach provides a novel pathway for eπтоr correction during reasoning tasks performed by LLMs. Despite their impressive reasoning capabilities, even advanced reasoning-centnn [68] occasionally commit errors due to misreading critical elements within problem statements. To illustrate this, consider a simple mathematical question from the GSM8K dataset [80] depicted in Fig. 6. Initially, the model erroneously overlooks the keyword "remaining", consequently misunderstanding the problem statement and generating an incorrect answer. However, by explicitly introducing positive attention bias to the token corresponding to the overlooked keyword, we effectively realign the model's attention, enabling the model to correctly interpret the question and subsequently correct its output.
models such as DeepSeek-
R E1-Distille--QWent
Motivated by this observed phenomenon, we systematically examine broader set of examples to quantitatively assess the efficacy of our attention based eπтor correction strategy. Specifically, we identify 50 distinct math reasoning problems from the GSM8K dataset initially answered incorrectly due to misreading overlooking crucial textual cues. For each problem, we manually select and positively bias the attention logits corresponding to the critical tokens identified as frequently misinterpreted. Remarkably, after applying such targeted attention biases, 45 out of these 50 problems (90%) were subsequently answered correctly without any modifications to the input text itself. This result strongly indicates that the errors were indeed consequence of attentional oversight rather than inherent model deficiencies. See Appx. for more examples.
Limitation and Conclusion
We revisited input saliency for LLMs and showed that six popular attribution methods collapse in long-context settings, misplacing most of their mass on distractors. Our NIAH mneedle-in-a-haystack) stress test makes this failure mode transparent and measurable. Building on the empirical insight that tiny shifts in attention logits can deterministically flip a model's 's choice among contradictory messages, we introduced ABO, which frames attribution as a lightweight per-token bias optimization on attention logits. We empirically demonstrated the accuracy of ABO's saliency scores and its ability in zero-shot detoxification, sentiment steering, and correction of reasoning errors.
Despite these promising results, this work also reveals a more fundamental limitation of current input saliency research for LLMs. Our evaluation shows that the gap between how saliency methods perceived to work and how they actually behave practice remains surprisingly large. While ABO substantially narrows this gap relative prior methods, its absolute performance still leaves ample room for improvement. This persistent discrepancy stems from the inherent scale and complexity of modern LLMs, where attribution noise, token interactions, and non-linear dependencies grow rapidly with model size. Consequently, even small estimation errors in saliency can accumulate and distort the interpretation, posing challenge that goes beyond the reach of any single method.
From an algorithmic perspective, ABO's per-example optimization introduces additional variance across runs and examples, especially when the target likelihood landscape is highly non-convex. This sensitivity makes results less stable in certain contexts and increases computational cost for large- scale deployment. Future work could explore amortizing bias inference through meta-learning or distillation techniques, allowing approximate but faster and more consistent saliency estimation. Furthermore, our current stress test mainly targets retrieval-style prompts; extending it multi-step reasoning, dialogue, and multimodal tasks would provide a broader view of attribution fidelity under real-world conditions. Overall, ABO represents an encouraging step toward causally grounded saliency analysis for large language models. It not only improves attribution fidelity under long- context noise but also exposes deeper methodological gaps in how we interpret complex neural systems, gaps that will require sustained community effort to close.
10

---

Acknowledgment
This work supported by the National Science Foundation (NSF) CISE Core Program Award IIS-2207052, the NSF Cyber-Physical Systems (CPS) Award CNS-2235231, the NSF CAREER Award IIS-2338068, the ARO Award 199F3311,,,,,, the Cisco Research Award, the Amazon Research Award, and the IBM PhD Fellowship Award.
References
[1] R. Goldshmidt and M. Horoviez, 'Tookenhap:: Interpreting large language models with monte carlo shapley value estimation," arXiv preprint arXiv:2407.101 10/ 14, 2024.
[2] Z. Feng, Zhou, Zhu, Qian, and K. Mao, "Unveiling and manipulating prompt influence in large language models," arXiv preprini 1oXiv:2405./181, 2024.
[3] B. Cohen- Wang, Y.-S. Chuang, and A. Madry, "Learning to attribute with attention," arXiv preprint arXiv:2504. 13752, 2025.
[4] K. Yin and G. Neubig, "Interpreting language models with contrastive explanations." in Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. Abu Dhabi, United Arab Emirates: Association for Computational Linguistics, Dec. 2022, pp. 184 198. [Online]. Available: https//acclaatbboooooo//2222.2.o
[5] M. Sundararajan, A. Taly, and Q. Yan, "Axiomatic attribution for deep networks," in Interna- tional conference on machine learning. PMLR, 2017, pp. 3319-3328.
[6] B. Deiseroth, M. Deb, S. Weinbach, M. Brack, P. Schramowski, and K. Kersting, "Atman: Understanding transformer predictions through memory efficient attention manipulation," in Thirty-seventh Conference on Neural Information Processing Systems, 2023.
[7] S. Feng, E. Wallace, A. Grissom II, M. Iyyer, P. Rodriguez, and J. Boyd-Graber, "Pathologies neural models make interpretations difficult," Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Brussels, Belgium: Association for Computational Linguisties, Oct.-Nov. 2018, pp. 3719-3728, [Online], Available: http://cccannthoooggor/////L/ 1407/
[8] H. Chefer. S. Gur, and L. Wolf, "Generic attention-mode explainability for interpreting bi-modal and encoder-decoder transformers." " in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 397-406.
[9] D. Nam, Macvean, Hellendoorn, B. Vasileseu, and B. Myers, "Using an 1lm to help with code understanding." in Proceedings of the JEEF/ACM 46th International Conference on Software Engineering, 2024, pp. 1-13.
[10] M. Kazemitabaar, R. Ye, X. Wang, A. Z. Henley, P. Denny, M. Craig, and T. Grossman, "Codeaid: Evaluating a classroom deployment of an Ilm-based programming assistant that balances student and educator needs," in Proceedings of the 2024 chi conference on human factors in computing systems, 2024, pp. 1-20.
[11] X. Huang, W. Liu, X. Chen, X. Wang, Wang, D. Lian, Y. Wang, Tang, and Chen, "Understanding the planning of llm agents: A survey," arXiv preprint arXiv:2402.027//6 2024.
[12] A. Ali, Schnake, O. Eberle, G. Montavon, K.-R. Müller, and L. Wolf, "XAI for transformers: Better explanations through conservative propagation." in Proceedings of the 39th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, K. Chaudhuri, S. Jegelka, I. Song, C. Szepesvari, G. Niu, and S. Sabato, Eds., vol. 162. PMLR, 17-23 Jul 2022, pp. 435-451. [Online]. Available: https://proceedings..u mlr,
wtw.sg/ssss....s.//ssee wrwsssp 112es//116//li222ahhmm
[13] M. Geva, R. Schuster, J. Berant, and O. Levy, "Transformer feed-forwed layers are key-value memories." " in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, M.-F Moens, X. Huang, L. Speeiia, and S. W.-t. Yih, Eds. Online and Punta Cana, Dominican Republic: Association for Computational Linguisties, Nov. 2021, pp. 5484-5495. [Online]. Available: https///cclathoolog.orr/// 1.mmlp--eem----mmmmm.mmmmm main.446/

---

[14] M. Geva, A. Caciularu, K. Wang, and Y. Goldberg. "Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space," in Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, Y. Goldherg, Z. Kozareva, and Y. Zhang, Eds. Abu Dhabi, United Arab Emirates: Association for Computational Linguistics, Dec. 2022, pp. 30-45. [Online]. Available: main.3/
[15] G. Dar, M. Geva, A. Gupta, and J. Berant, "Analyzing transformers in embedding space," in Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), A. Rogers, J. Boyd-Graber, and N. Okazaki, Eds. Toronto, Canada: Association for Computational Linguistics, Jul. 2023, pp. 16 124 16 170. [Online]. Available: https//tclanthollyyoor/// http:///222....loooooo acl-long..99. 893/
[16] S. Abnar and W. Zuidema, "Quantifying attention flow in transformers." in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, D. Jurafsky, J. Chai, N. Schluter, and J. Tetreault, Eds. Online: Association for Computational Linguistics, Jul. 2020, pp. 4190 4197. [Online]. Available: http:://cccattoolegygg220002...2 act.am0....
[17] A. Modarressi, M. Fayyaz, Y. Yaghoobzadeh, and M. T. Pilehvar. "GlobEne: Quantifying global token attribution by incorporating the whole encoder layer in transformers." " in Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, M. Carpuat, M.-C. de Marneffe, and Meza Ruiz, Eds. Seattle, United States: Association for Computational Linguistics, Jul. 2022, pp. 258-271. [Online]. Available: https://ccdanttooog.g.///2222aaa
[18] B. Cohen-Waie Wang, H. Shah, K. Georgiev, and A. Madry, "Contextcite: Attributing model genera- tion to context," Advances in Neural Information Processing Systems, vol. 37, pp. 95 764-95 807, 2024.
[19] T. R.-A. Generation, "Model internals-based based answer attribution for trustworthy retrieval- augmented generation."
[20] J. Vig, S. Gehrmann, Y. Belinkov, S. Qian, D. Nevo, S. Sakenis, J. Huang, Y. Singer, and S. Shieber, "Causal mediation analysis for interpreting neural nlp: the case of gender bias (2020)," arXiv preprint arXiv:2004. 12655, 2004.
[21] V. Prabhakaran, B. Hutchinson, and M. Mitchell, "Perturbation sensitivity analysis to detect unintended model biases," in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNNIP-IJCNIPP) K. Inui, J. Jiang, V. Ng, and X. Wan, Eds. Hong Kong, China: Association for Computational Linguistics, Nov. 2019, pp. 5740-5745. [Online]. Available: hatpr//anhoooooo.ooo//19119111
[22] J. Ferrando, G. I. Gállego, and M. R. Costa-jussà, "Measuring the mixing of contextual information in the transformer," in Proceedings of the 2022 Conference Empirical Methods in Natural Language Processing, Y. Goldberg, Z. Kozareva, and Y. Zhang, Eds. Abu Dhabi, United Arab Emirates: Association for Computational Linguistics, Dec. 2022, pp. 8698 87 14. [Online]. Available: httpss/acccattoooogooog222222222 hww.:/222..mmmmmmmmm..5555.. main.595/
[23] S. M. Lundberg and S.-I. Lee, "A unified approach interpreting model predictions," in Advances in Neural Information Processing Systems, 2017, pp. 4768-4777.
[24] E. Wallace, Tuyls, J. Wang, Subramanian, M. Gardner, and S. Singh, "AllenNLP interpret: A framework for explaining predictions of NLP models," in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLPP) System Demonstrations,
S. Padó and R. Huang, Eds. Hong Kong. China: Association for Computational Linguisties. Nov. 2019, pp- 7-12. [Online]. Available: http:://acannttoogg.oo////11
[25] N. Liu, K. Lin, J. Hewitt, A. Paranjape, M. Bevilacqua, F. Petroni, and Liang, "Lost in the middle: How language models use long contexts," arXiv preprint arXiv:2307.0317. 2023.
12

---

[26] K. Simonyan, A. Vedaldi, and A. Zisserman, "Deep inside convolutional networks: visualising image classification models and saliency maps," in Proceedings of the International Conference on Learning Re Representations (ICIR), 2014, pp. 1-8.
[27] J. Li, X. Chen, E. Hovy, and D. Jurafsky, "Visualizing and understanding neural models in NLP," in Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies. San Diego, California: Association for Computational Linguisties, Jun. 2016, pp. 681-691. [Online]. http://accantboooy.oo//////// Available:
[28] P. Atanasova, J. G. Simonsen, C. Lioma, and I. Augenstein, "A diagnostic study of explainability techniques for text classification," " in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Online: Association for Computational Linguistics, Nov. 2020, pp. 3256-3274. [Online]. Available:
[29] M. Denil, Demiraj, and N. De Freitas, "Extraction of salient sentences from labelled documents," arXiv preprint 1.X2:::412..8111 2014.
[30] A. Shrikumar. P. Greenside, and A. Kundaje, "Learning important features through propagating activation differences." in International conference on machine learning. PMLR, 2017, pp.
3145-3153.
[31] E. Wallace, J. Tuyls, J. Wang, S. Subramanian, M. Gardner, and S. Singh, "AllenNLP interpret: A framework for explaining predictions of NLP models," in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-JCCLPP): System Demonstrations. Hong Kong. China: Association for Computational Linguistics, Nov. 2019, pp. 7-12. [Online]. Available: https://cclantholoy..or/// org11/9330///
[32] S. Bach, Binder, G. Montavon, F. Klauschen, K.-.. Müller, and W. Samek, "On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation." PloS one, vol. 10, no. 7, p. c01 c0130140, 2015.
[33] A. Ali, T. Schnake, O. Eberle, G. Montavon, K.-R. Müller, and L. Wolf, "XAI for transformers: Better explanations through conservative propagation." in Proceedings of the 39th International Conference on Machine Learning. ser. Proceedings of Machine Learning Research, K. Chaudburi, S. Jegelka, L. Song, C. Szepesvari. G. Niu, and Sabato, Eds., vol. 162. PMLR, 17-23 Jul 2022, pp. 435-451, [Online]. Available: www.r/dddiin..iiimrrssss/ii/ea/e
S.
[34] S. Abnar and W. Zuidema, "Quantifying attention flow in transformers." in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Online: Association for Computational Linguistics, Jul. 2020, pp. 4190-4197. [Online]. Available: http://accaathoooggyo/0000...0 watp:/oagoooo///..alaaalaoooo-ee mainn3
http://aclanthooog..orgg
[35] A. Modarressi, M. Fayyaz, Y. Yaghoobzadeh, and M. T. Pilehvar, "GlobEnc: Quantifying global token attribution by incorporating the whole encoder layer in transformers." in Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Seattle, United States: Association for Computational Linguistics, Jul. 2022, pp. 258-271. [Online]. Available: htts://aclaabbboooooo/g/22mmaaa.
[36] J. Ferrando, G. I. Gállego, and M. R. Costa-jussà, "Measuring the mixing of contextual information in the transformer," in Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. Abu Dhabi, United Arab Emirates: Association for Computational Linguistics, Dec. 2022, pp. 8698 8714. [Online]. Available: http://cclnntooooo.o.o/2/2mmmm/m
[37] S. Feng, E. Wallace, A. Grissom II, M. lyyer, P. Rodriguez, and J. Boyd-Graber, "Pathologies of neural models make interpretations difficult," in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Brussels, Belgium: Association for Computational Linguisties, Oct.-Nov. 2018, pp. 3719-3728. [Online]. Available: http::/lclaatblooyyoog///11111
13

---

[38] V. Prabhakaran, B. Hutchinson, and M. Mitchell, "Perturbation sensitivity analysis to detect unintended model biases," " in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-.JCNIPP). Hong Kong. China: Association for Computational Linguistics, Nov. 2019, pp. 5740-5745. [Online]. Available: https//accantboooy..///19111111
[39] T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al., "Language models are few-shot learners," " Advances in neural information processing systems, vol. 33, pp. 1877-1901, 2020.
[40] P. Liu, W. Yuan, J. Fu, Z. Jiang, H. Hayashi, and G. Neubig, "Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing," arXiv preprint (tfi::2007.15880 2021.
[41] Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou al., "Chain-of- thought prompting elicits reasoning in large language models," " Advances in Neural Information Processing Systems, vol. 35, pp. 24 84 837, 2022.
[42] H. Strobelt, A. Webson, V. Sanh, B. Hoover, Beyer, H. Pfister, and A. M. Rush, "Interactive and visual prompt engineering for ad-hoc task adaptation with large language models," 2022.
[43] S. H. Bach, V. Sanh, Z.-X. Yong, A. Webson, C. Raffel, N. V. Nayak, A. Sharma, T. Kim, M. S. Bari, Fevry et al., "Promptsource: An integrated development environment and repository for natural language prompts," arXiv preprint arXiv:2202.0/273 2022.
[44] Shin, Y. Razeghi, R. L.. Logan IV, E. Wallace, and S. Singh, "Autoprompt: Eliciting knowledge from language models with automatically generated prompts," arXiv preprint Ttti::00101198800 2020.
[45] M. Deng, J. Wang, C.-.. Hsieh, Y. Wang, H. Guo, T. Shu, M. Song, E. P. Xing, and Z. Hu, "Riprompt: Optimizing discrete text prompts with reinforcement learning," arXiv preprint arXiv:2205.1 11055/2543 /2548, 2022.
[46] C. Singh, J. X. Morris, J. Aneja, A. M. Rush, and J. Gao, "Explaining patterns in data with language models via interpretable autoprompting." 2023.
[47] A. Webson and E. Pavlick, "Do prompt-based models really understand the meaning of their prompts?" arXiv preprint 0rXiv:2109.01247, 2021.
[48] Y. Lu, M. Bartolo, A. Moore, S. Riedel, and P. Stenetorp, "Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity," arXiv preprint arXiv:2104.088 08786. 2021.
[49] Wei, M. Bosma, V. Zhao, K. Guu, A. W. Yu, B. Lester, N. Du, A. M. Dai, and Q. V. Le, "Finetuned language models are zero-shot learners," arXiv preprint aix: 100.0000,,, 2021.
[50] H. W. Chung. L. Hou, S. Longpre, B. Zoph, Y. Tay, W. Fedus, E. Li, X. Wang, M. De- hghani, S. Brahma et al., "Scaling cntruuctionfinettned language models," arXiv preprins 2rXiv:2210.11416. 2022.
[51] D. M. Ziegler, N. Stiennon, J. Wu, T. B. Brown, A. Radford, D. Amodei, P. Christiano, and G. Irv- ing, "Fine-tuning language models from human preferences," arXiv preprint 1rXiv:1909.08593 2019.
[52] L. Ouyang, J. Wu, X. Jiang. D. Almeida, C. Wainwright, Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray et al., "Training language models to follow instructions with human feedback." Advances in Neural Information Processing Systems, vol. 35, pp. 27 730-27 744, 2022.
[53] H. Lee, S. Phatale, H. Mansoor, K. Lu, T. Mesnard, C. Bishop, V. Carbune, and A. Rastogi, "Rlaif: Scaling reinforcement learning from human feedback with ai feedback," arXiv preprint arXiv:2300. 00009.00077.100 2023.
[54] A. Ross, M. C. Hughes, and F. Doshi-Velez, "Right for the right reasons: Training differen- tiable models by constraining their explanations," arXiv preprint 1703v:1703.03717, 2017.
14

---

[55] L. Rieger, C. Singh, W. J. Murdoch, and B. Yu, "Interpretations are useful: penalizing ex- planations to align neural networks with prior knowledge," " arXiv preprint arXiv:1909.1355 2019.
[56] P. Schramowski, W. Stammer, S. Teso, A. Brugger, F. Herbert, X. Shao, H.-G. Luigs, A.-K. Mahlein, and K. Kersting, "Making deep neural networks right for the right scientific reasons by interacting with their explanations." Nature Machine Intelligence, vol. 2, no. 8, pp. 476-486, 2020.
[57] S. Krishna, J. Ma, D. Slack, A. Ghandeharioun, S. Singh, and H. Lakkaraju, "Post hoc expla- nations of language models can improve language models," arXiv preprint arXiv:2305. 1426, 2023.
[58] E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, "Lora: Low-rank adaptation of large language models," arXiv preprint arXiv:2106. 09680, 2021.
[59] Q Zhang, M. Chen, A. Bukharin, P. He, Y. Cheng, W. Chen, and T. Zhao, "Adaptive budget allocation for parameter-efficient fine-tuning," " in The Eleventh International Conference on Learning Representations, 2023. [Online]. Available: https://ppemeevew.eeeffoonmiiii Iq62uWR.jiY
[60] T. Dettmers, A. Pagnoni, A. Holtzman, and I.. Zettlemoyer, "Qlora: Efficient finetuning of quantized Ilms," 2023.
[61] B. Shi, S. Gai, T. Darrell, and X. Wang, "Refocusing is key to transfer learning," arXiv preprint 27X55:2305.15542, 2023.
[62] K. Meng, D. Bau, A. Andonian, and Y. Belinkov, "Locating and editing factual associations in gpt," Advances in Neural Information Processing Systems, vol. 35, pp. 17 359-17 372, 2022.
[63] K. Meng, A. S. Sharma, A. Andonian, Y. Belinkov, and D. Bau, "Mass-editing memory in transformer," arXiv preprint arXiv:2210007229, 2022.
[64] E. Mitchell, C. Lin, A. Bosselut, C. Finn, and C. D. Manning, "Fast model editing scale," 2022.
[65] E. Hernandez, B. Z. Li, and J. Andreas, "Inspecting and editing knowledge representations in language models," 2023.
[66] H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale el al., "Llama 2: Open foundation and fine-tuned chat models," arXiv preprint wxXiv2230..09288, 2023.
[67] Jiang, "Identifying and mitigating vulnerabilities in llm-integrated applications," Master's thesis, University of Washington, 2024.
(68] D. Guo, D. Yang, H. Zhang, J. Song, R. Zhang, R. Xu, Q. Zhu, S. Ma, P. Wang, X. Bi et al., "Deepseek-rl: Incentivizing reasoning capability in lims via reinforcement learning"" arXiv preprint arXiv:2501.12945 2025.
[69] Y. Bai, X. Lv, J. Zhang, Lyu, Tang, Z. Huang, Z. Du, X. Liu, Zeng, L. Hou, Dong, J. Tang, and J. Li, "Longbench: A bilingual, multitask benchmark for long context understand- ing." " 2023.
[70] B. Wang, W. Chen, H. Pei, C. Xie, M. Kang, C. Zhang, C. Xu, Z. Xiong, R. Dutta, R. Schaeffer el al., "Decodingtrust: A comprehensive assessment of trustworthiness in gpt models," arXiv preprint arXiv:2306. 1698, 2023.
[71] Y. Liu, Y. Yao, J.F.. Ton, X. Zhang, R. G. H. Cheng, Y. Klochkov, M. F. Taufiq, and H. Li, "Trustworthy 1mss: a survey and guideline for evaluating large language models' alignment," arXiv preprint arXi::3388.05374 2023.
15

---

[72] S. Gehman, S. Gururangan, M. Sap, Y. Choi, and N. A. Smith. PrealToxicityProppii Evaluating neural toxic degeneration language models," in Findings of the Association for Computational Linguisties: EMNLP 2020. Online: Association for Computational Linguistics, Nov. 2020, pp- 3356-3369. [Online]. Available: http:///elarthhooy.oo/22220 findings-emnl 301
[73] M. Geva, A. Caciularu, K. Wang, and Y. Goldberg, "Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space," Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. Abu Dhabi, United Arab Emirates: Association for Computational Linguistics, Dec. 2022, pp. 30-45. [Online]. http:://ccaatboooooooo///2222e/d mainmm main.3
Available:
[74] E. Reif, D. Ippolito, A. Yuan, A. Coenen, C. Callison- and J. Wei, "A recipe for arbitrary text style transfer with large language models," in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). S. Muresan, P. Nakov, and A. Villavicencio, Eds. Dublin, Ireland: Association for Computational Linguistics, May 2022. pp. 837 848. [Online]. Available: hutps//cccanttoogg.ggg222......
[75] Schick, Udupa, and H. Schütze, "Self-Diagnosis and Self-Debiasing: Proposal for Reducing Corpus-Based Bias in NLP," Transactions of the Association for Computational Linguistics, vol. 9, pp. 1408-1424, 12 2021. [Online]. Available:
http://ddoi.rg/11.116/6222 L_000/ 00434
[76] A. Gokaslan and V. Cohen, "Openwebtext corpus," 2019.
[77] T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, Cistac, T. Rault, R. Louf, M. Funtowicz, J. Davison, S. Shleifer, P. von Platen, C. Ma, Y. Jernite, J. Plu. C. Xu, T. Le Scao, S. Gugger, M. Drame, Q. Lhoest, and A. Rush, "Transformers: State-of-the--u natural language processing," in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. Online: Association for Computational Linguistics, Oct. 2020, pp. 38-45. [Online]. Available:
http://cccamhology.oo//20000 wwwp://p220eemmmm-mmeee demoo.6
[78] E. Cambria, R. Speer, C. Havasi, and A. Hussain, "Senticnet: A publicly available semantic resource for opinion mining," 2010 AAAI fall symposium series, 2010.
[79] A. Liu, M. Sap, X. Lu, Swayamdipta, C. Bhagavatula, N. A. Smith, and Y. Choi, "DExperts: Decoding-time controlled text generation with experts and anti-experts," in Proceedings of the 59th Anmual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume I: Long Papers). Online: Association for Computational Linguistics, Aug. 2021, pp. 6691-6706. [Online). Available: http://accanthoooy.org//222 htt.:/orrg/22....loooo.....5. lcl-long.522
[80] K. Cobbe, Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek,
J. Hilton, R. Nakano, C. Hesse, and J. Schulman, "Training verifiers to solve math word problems," arXiv preprint arXiv:2110 10. 14168, 2021.
16

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
Yes, the main claims in the abstract and introduction accurately reflect the paper's contributions and scope, including the contributions made in the paper and important assumptions and limitations.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
limitations are discussed in Sec.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-specfiiatiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
17

---

Answer: [NA]
Justification: the paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: Detailed setups and hyper-parmaeters can be found in Appx. A.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm. (b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
18

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: The data used in this paper are fully open datasets. The code and instructions are included in the supplementary material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips cc/
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to to reproduce the results. See the NeurlPS code and data submission guidelines (https: petdeessummsssooooooooooo
/ //nips. ctce/pubiccgggudceo
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: This paper is inference- time saliency analysis, so does not involve data spliting. Detailed experiment setups reported both in Sec. 5 and Appx.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: Although the paper does not explicitly report error bars for each data point, this is understandable given the large volume of quantitative results, where including error bars for every entry could significantly reduce readability. However, the authors state that all reported results are averaged over three independent runs.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emror bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
19

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Detailed computational resources are reported in Appx. A.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https: atttps://neurre hww.rpuu....///e
ines?
Answer: [Yes]
Justification: The authors confirm that in every respect, the research is not against NeurIPS code of Ethies.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. if there special consid- eration due to laws or regulations in their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
The socictal impact statement is included in Appx. C.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
20

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: the paper poses no such risks
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
All the open datasets and models are properly cited.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
21

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: the paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/codeemoodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: the paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: the paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
22

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy ptttp://neeriige http://eerrrss.ccoeeeeereeeeeeee / for what should or should not be described.
23

---

Appendix
A Experiment Setup
All saliency methods were normalized to ensure that their respective attribution scores sum to one.
Gradient Norm (GrADNOrM) computes saliency scores based the Eg norm of the gradient of the model's output with respect to the input token embeddings. Specifically, the score for each token D, calculated by taking the L1 norm of the gradient V Va, g (z),, where q(ye (z) is the model's predicted probability for token y: given input z.
Gradient x Input (INPUTXGRAD) modifies the Gradient Norm approach by caleulating the dot product between each input embedding I; and its gradient Vz, thereby emphasizing tokens highly expressed in the model's predictive process.
Attention Rollout (ROLLOUT) tracks the flow of attention information through transformer layers by recursively multiplying attention matrices across layers. Given raw attention matrices A(I;) at each layer 4, attention from higher layers multiplied downwards layer by layer, creating cumulative attention maps that reflect the propagation of token importance from input layers up to the output layer (https: / /tg::////hhbbbb
woon/hmoebbbi///ccccc mamobebbi/contette context wttpoottiiiiiinnooooliiioo
TokenSHAP (SHAP) estimates Shapley values for tokens via Monte Carlo sampling. The importance of each token determined by evaluating its marginal contribution to the model's output across randomly sampled subsets of tokens, combining essential combinations with randomly sampled ones ensure both computational efficiency and accurate estimation (https // thub. com/ronigold/ TokenSHAP).
Token Distribution Dynamies (TDD) analyzes token saliency using token distribution projections within the embedding space via the language model head. Variants such as forward, backward, and bidirectional TDD measure token saliency through their distributional shifts and causal contributions to model predictions (https:/ // //gittbtttt///////// com/ (1ijia667//TDD)
Attention Manipulation (ATMAN) generates relevance maps through memory-effictie perturba- tions of transformer attention mechanisms. It applies token-level perturbations directly within the attention space, reducing computational overhead and facilitating efficient interpretation of model decisions (https: htt///gtttttut com/ Aleph AlephhAlphal Alpha/ AAtan)).
Attribution with Attention (AT2) treats attention weights from individual heads as learnable features. This method uses ablation-based training to learn attention-heat coefficients, enabling efficient yet accurate token attribution across different contexts and significantly reducing computational requirements (https: //gtthu///tt///// com Httm/MadryLabbAAT
ABO: For ABO, we optimized the bias parameters } using the Adam optimizer with a typical learning rate of 0.01, applying a decay schedule after a fixed number of iterations. The optimization was capped at 50 to 200 steps, depending on the context length, with early stopping triggered by the convergence of the target token probability. To promote sparsity and focus in the saliency distribution, we applied
é regularizer with weight À ranging between 10-1 and 1. All bias parameters were initialized randomly to avoid introducing inductive bias at initialization.
A.1 Compute Environment
All experiments were conducted exclusively for evaluation purposes, with model training or fine- tuning involved. We utilized NVIDIA H100 GPUs for all runs, and the total compute consumption was approximately 1000 GPU hours. This estimate includes forward passes, saliency score extraction, attention bias optimization, and repeated runs for multi-seed robustness analysis across multiple saliency methods and model backbones.
B Additional Experiment Results
In this section, we provide additional qualitative examples (Fig. Al~Fig. A5) to illustrate how BPO can used correct model reasoning errors by redirecting attention to overlooked yet critical tokens. These examples are drawn from the GSM8K dataset and demonstrate instances where the
24

---

[Question] Paul went to shop to buy some groceries. He bought some bread for 2, butter for 3, and juice for two times the price of the bread He had 15 for his shopping. How much money did Paul have left?
[Reasoning (Incorrect by Default)) <think> Let's determine how much money Paul has left after his shopping step by step. The bread costs 2. The juice costs times the price of the butter <ithink> Final Answer: ((boxed4))
[Reasoning (Correct with Attention Bihs) Let's determine how much money Paul has left after his shopping step by step. The bread costs 2. The juice costs two times the price the bread <lthink>
Final Answer: \(boxed66)
Figure A1: Example Initially, the model misidentifies "juice costs twice the butter" instead "twice bread".. With attention bias on the word "bread," the model corrects its interpretation and computes the correct remaining balance.
[Question] There are some jelly beans a jar. Three fourths of red jelly beans are red, and one quarter of the jelly beans are coconut flavored. If 750 jelly heans are coconut flavored, how many jelly beans are there in the jar?
[Reasoning (Incorrect by Default)) <think> Let's solve the problem step by step. Given: Since one quarter of the jelly equals 750, can set the equation:
[Reasoning (Correct with Attention Itasi ... That means if have a total number of jelly beans, say, T, then the num- ber of red jelly beans would be (3/4) T.
<ithink> Final Answer: (Aboxed30001)
<lthink>
Final Answer: ((boxed40000)
Figure A2: Example 2.
The model initially misattributes the fraction to total jelly beans. Highlighting "red" refocuses attention the subject of the fraction, enabling correct inference of the jar size.
model initially produced incorrect answers due misinterpretation. By applying positive bias the appropriate input tokens, we successfully guide the model to revise its understanding and produce the correct
modifying the input text. Each example shows the original faulty reasoning on the left and the corrected reasoning with attention bias on the right. These results showcase the potential of attention-leva interventions for LLM robustness and post hoc debugging.
C Broader Impact
This work advances the field trustworthy AI by proposing Attention Bias Optimization (ABO), a principled method for accurate token attribution in large language models (LLMs). By identifying and correcting systemic flaws in six widely-used saliency methods, our work enables more faithful, scalable, and robust interpretability, especially in long-context settings.
From an ethical standpoint, ABO enhances transparency LLM behavior, reducing the risk of misinterpretation in high-stakes applications such as legal analysis, education, and healthcare. How- ever, accurate attribution also raises dual-use concerns: malicious actors could leverage improved interpretability to reverse-engineer sensitive prompts or fine-tune manipulative behavior. Future research should explore safeguards that balance interpretability with misuse prevention.
At the societal level, our method supports the development of safer AI systems by enabling ap- plications such as zero-shot toxicity suppression, error correction, and sentiment steering. These capabilities can improve the reliability and controllability of AI assistants and mitigate harms caused by misaligned or biased outputs. Still, we caution that attribution not equivalent accountability: robust evaluation and user education remain necessary.
25

---

[Question] A company pays each of its employees 600 in month The company has policy of increasing the salaries of cach of its employees by 10% of the initial salary every year for those who' ve stayed in the company for five years. If Sylvie just clocked 5 years in the company last December, what's annual salary after three more years service?
[Reasoning (Correct with Attention Bihs) First, I need to deter- mine Sylvie's current monthly salary. Since she has been with the Starting salary: 600 per month.
[Reasoning (Incorrect by Default)] <think> Alright, so have this problem about Sylvic's salary ... First, it says that the company pays each employec per ...<thhikk> Final Answer: ((boxed7801)
<lthink>
Final Answer: 544.)
The model mistakes the salary to yearly instead monthly. Biasing word "month" shifts the model toward the correct interpretation and output.
Figure A3: Example 3.
[Question] Brian's friend Bobby has 5 fewer than times many video games as Brian does. Brian video games but right after comparison was made, how many does Bobby have?
[Reasoning (Correct with Attention Biass]<think> Alright, let's try to solve this problem step by step. ... The phrase implies that the comparison occurred, and then Brian lost S. So, it seems like the comparison was done when Brian had 20 games, and after that, he lost 5, <t.nnnhink>> Final Answer: ((boxed555)
[Reasoning (Incorrect by Default)) <think> Okay, let's tackle this problem step by step. Determine the current number of video games Brian has: Originally, Brian has 20 video games. He lost 5 video games. Current number of Brian's games= 20 5 = 15 <lthink> Final Answer: (\boxed401)
Figure A4: Example 4. The model mistakenly adjusts the number of video games after the comparison. Emphasizing the word "after" leads the correct timing of the event and the correct answer.
We encourage continued efforts to integrate interpretability into broader frameworks of Al alignment, governance, and fairness auditing ensure that attribution advances benefit diverse populations responsibly.
26

---

[Question] Jerome had friends who came to visit him certain day. The first friend pressed the doorbell 20 times before Jerome opened, second friend pressed on the doorbell times more than Jerome's 's first friend. The third friend pressed the doorbell 10 times more than the fourth friend. the fourth friend pressed on the doorbell times, how many doorbell rings did the doorbell make?
[Reasoning (Correct with Attention Bias) <think> First, identify the number of times each friend pressed the doorbell. Fourth Friend: The fourth friend pressed the door- hell times. Third Friend: The third friend pressed the doorbell 10 times more than the fourth friend. Calculation: [ 10 60 600 times ... <ithink>
|Reasoning (Incorrect by Default)) <think Alright, let's tackle this problem step by step. First Friend: Pressed the doorbell 20 times. Second Friend: Pressed the doorbell 1/4 times than the first friend. Third Friend: Pressed the doorbell 10 times more than the seeend friend. Third Friend: 250 times ...Atthik>> Final Answer: V(boxed6051)
Final Answer: ((boxed6055)
The model incorrectly attributes the comparison for the third friend to the second friend. Attention bias the word "fourth" enables accurate attribution and total computation.
Figure A5: Example 5.
27


Reviews:
[{"review_id": "G4nxfeVrR4", "reviewer": "Reviewer_YGKp", "summary": "This paper proposes a stress-testing framework based on the Needle-in-a-Haystack (NIAH) paradigm, where a hidden \"secret message\" is embedded in long passages of irrelevant text. Considering seven popular saliency methods, they find: All methods misattribute saliency to irrelevant context, and this error worsens with longer inputs. In prompts over 10K tokens, more than 90% of saliency is incorrectly assigned to distractors. The paper then introduces a method: Attention Bias Optimization (ABO) ABO introduces learnable per-token attention biases into the transformer’s attention logits. It optimizes these biases to maximize the probability of a target output token, with a sparsity inducing regularizer. Saliency is then defined by the magnitude of the learned biases. The paper then shows that effectiveness of this approach across several tasks.", "questions": "See the weakness section. But I list them here. \n\n- Model vs. Method Attribution Failures:  To what extent are the attribution failures observed in your stress tests due to limitations of the attribution methods themselves versus the lack of robustness or interpretability in the pretrained LLMs used (e.g., LLaMA2, Mistral)?\n\n- Relevance of Adversarial and Robust Training: Why doesn’t the paper consider or evaluate attribution methods on models that have been adversarially trained or regularized for off-manifold robustness—given prior work shows this leads to significantly better gradient-based attributions?\n\n- Positioning ABO vs. Robust Training: How does ABO compare with or complement models that have been trained explicitly to yield faithful attributions (e.g., via adversarial robustness)? Would ABO still offer performance gains in that setting?\n\n- Clarity on Problem Setting: Is ABO intended as a general replacement for existing saliency methods, or as a solution specifically for frozen, black-box LLMs that cannot be retrained or regularized? This distinction should be made clearer.\n\n- Interaction Between ABO and Model Smoothness: Does ABO implicitly assume or require smoothness in the model’s input-to-output mapping? If so, how sensitive is it to local irregularities (e.g., sharp gradient transitions) in the model's attention landscape?\n\n- Potential for Training-Time Integration: Could ABO be incorporated during training (e.g., as an auxiliary objective) to produce inherently more interpretable models? If so, how might that compare to existing regularization techniques aimed at interpretability?\n\n- Relation to Existing Literature: Why doesn’t the paper more directly engage with the literature on interpretability improvements via robust training? A brief discussion would help clarify how ABO fits into or diverges from this line of work.", "limitations": "- There are tasks that sole attention, per token, focus of ABO will miss.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths\n\n- Clear Problem Statement and Motivation: The paper identifies a limitation in current input saliency methods: misattribution in long-context settings.\n\n- New Evaluation Framework: The adaptation of the Needle-in-a-Haystack (NIAH) framework for saliency assessment is methodologically sound and cleverly designed. The use of both single-token and multi-token ground truths, as well as contradictory secrets, gives a nice testbed for evaluating attribution fidelity.\n\n- Technical Contribution: Attention Bias Optimization (ABO): ABO is a lightweight, and differentiable method for computing token saliency. It leverages the causal relationship between attention bias and model output, moving beyond proxy-based metrics like attention weights or gradients.\n\n- Comprehensive Evaluation: Includes thorough comparisons across methods (7 baselines), LLMs (LLaMA2, Mistral, QWen), and settings (long-context, pruning, toxicity suppression, sentiment control, reasoning correction). ABO shows consistent and significant improvements (20–40% in hit ratios, better resilience under pruning, etc.).\n\n## Weaknesses\nThis paper proposes a compelling new saliency method, Attention Bias Optimization (ABO), and demonstrates its superiority over several standard attribution techniques. However, in order to contextualize its contributions more rigorously, the paper should directly engage with a body of prior work that challenges the reliability of saliency maps—particularly gradient-based attributions—under specific conditions.\n\nEarlier foundational work (e.g., Hooker et al., 2018 [https://arxiv.org/abs/1806.10758]) showed that standard feature attribution methods can perform no better than random baselines. This result catalyzed a wave of research on more reliable saliency mechanisms. Crucially, subsequent work (e.g., Ilyas et al., 2019 [https://arxiv.org/abs/1905.02175]) argued that these attribution failures are not flaws in the methods per se, but in the models: models learn non-robust features that are incomprehensible to humans but predictive in the training distribution.\n\nMore recent work (e.g., Shah et al., 2021 [https://arxiv.org/abs/2102.12781] and Srinivas et al., 2023 [https://arxiv.org/abs/2305.19101]) has shown that regularizing models for adversarial or off-manifold robustness substantially improves the fidelity of gradient-based attributions. These findings imply that attribution quality is tightly coupled to model smoothness and robustness.\n\nWith this in mind, the present paper should address several key questions:\n\n- Attribution vs. Model Quality: To what extent are the failures of current saliency methods in the NIAH stress test due to flaws in the methods themselves, versus the fact that the underlying models (e.g., LLaMA, Mistral) were not trained with robustness or interpretability constraints?\n\n- Frozen Model Assumption: The paper assumes that the model is frozen and attribution must operate post hoc. This is a valid and important scenario. However, it would be helpful to acknowledge prior work showing that with proper training, even gradient-based methods can yield reliable attributions. This would clarify that ABO is not necessarily replacing flawed methods, but offering a solution when retraining is infeasible.\n\n- Off-Manifold Robustness: Recent findings suggest that off-manifold robustness is a key factor in obtaining faithful attributions. The authors should discuss how their approach relates to or might benefit from this line of work, e.g, could ABO be combined with robust training to further enhance attribution?\n\n- Clarifying the Problem Setting: There is an important distinction between: Attribution for frozen, pre-trained black-box LLMs, and Attribution for models explicitly trained with interpretability in mind. The paper could benefit from explicitly defining this boundary and clarifying where ABO fits in the broader landscape.", "was_revised": true, "final_justification": "The response has satisfactorily addressed all the remaining concerns that I have about this work. In particular, the authors have satisfactorily addressed my concerns around the cause of the improvements by ABO and the engagement with previous work.", "rebuttal": "Thank you for your thoughtful review and for recognizing the key novelties of our work. Below, we provide detailed responses to the main questions and concerns raised.\n\n**Q1. The paper should directly engage with a body of prior work that challenges the reliability of saliency maps—particularly gradient-based attributions—under specific conditions, such as Hooker et al., 2018, Ilyas et al., 2019, Shah et al., 2021, and Srinivas et al., 2023.**\n\n**A1.** We thank the reviewer for pointing out these important references. We have added citations to Hooker et al. (2018), Shah et al. (2021), and Srinivas et al. (2023) in the revised version, and we appreciate your suggestion to connect their findings to our work.\n\nThe reason we did not initially include these papers is that they primarily focus on saliency methods in the pre-LLM era, particularly in the context of discriminative models such as image classifiers. As a result, we felt their scope was somewhat far from  our focus on token attribution in LLMs. That said, we agree that the fundamental concerns they raise about saliency reliability are highly relevant, and we are glad to incorporate these discussions into our related work, problem scope and setup, and future work.\n\n**Q2. Attribution vs. Model Quality?**\n\n**A2.** This is a great question. \nFirst, to isolate the effects of the attribution methods from the capabilities of the tested models, we ensured that the LLM produces the **correct answer to every NIAH query before applying any saliency evaluation.** This guarantees that the attribution scores are tested only on examples where the model behavior is correct, thereby attributing performance differences solely to the attribution methods themselves. This design choice is also reflected in Tables 1 and 2: for instance, we evaluate LLaMA2-7B up to 4K tokens and Mistral-7B up to 10K. The reason for this discrepancy is that **LLaMA2-7B fails to consistently retrieve the correct message beyond 4K tokens,** and thus we exclude longer contexts for that model. In short, the attribution failures observed in our study are **not due to model misbehavior,** but rather stem from **systematic shortcomings in the attribution methods.**\n\nSecond, we agree that evaluating attribution methods on robustness- or interpretability-enhanced models is a promising direction. However, the goal of our work is to improve the fidelity of attribution mechanisms for general-purpose LLMs without relying on specialized training. Assuming the availability of interpretability-optimized models would limit the applicability of attribution tools in real-world settings, where such models are rarely available. While prior works (e.g., Srinivas et al., 2023) have suggested training-time interventions for interpretability in vision models, extending such techniques to LLMs poses substantial challenges. The computational burden of LLMs renders robust training methods such as the simplest randomized smoothing or the min-max optimization-based adversarial training computationally-prohibitive in practice. We will further discuss this in our response to Q4.\n\n\n**Q3. Frozen Model Assumption.**\n\n**A3.** We fully agree with the reviewer. Our goal is not to claim that ABO should replace all existing methods, especially when models are trained with interpretability constraints. Instead, our work aims to highlight that **current evaluation protocols may give overly optimistic or misleading impressions** of attribution quality when using standard general-purpose LLMs, especially in long-context settings. By introducing the **NIAH** stress test, we hope to **encourage more rigorous and ground-truth-based evaluation**. ABO offers a practical and more effective solution in such cases, and we view it as an improved **complement to, rather than a replacement for,** existing attribution methods.\n\n**Q4. Off-Manifold Robustness.**\n\n**A4.** We appreciate the reviewer’s insightful suggestion. While off-manifold robustness is indeed relevant to attribution, most existing work in this area focuses on image classifiers , not LLMs. In contrast, adversarial training for LLMs presents substantial challenges due to the discrete nature of text inputs, the sheer scale of modern models, and the risk of degrading core generation quality. \nYet, motivated by the reviewer’s insightful comment, we conducted a detailed survey of recent advances in adversarial training for LLMs. As a result, we carried out **additional experiments** testing ABO on adversarially trained LLMs from the NeurIPS 2024 paper “Efficient Adversarial Training in LLMs with Continuous Attacks” (e.g., Phi-CAT, Zephyr-CAT). Unfortunately, these models did not satisfy the basic requirement for our attribution evaluation: achieving correct retrieval in the NIAH test. Specifically, they attained <15% accuracy even at 1K-token context length. This failure suggests a potential tradeoff between adversarial robustness and generative accuracy, which makes the studied adversarially trained LLMs improper for saliency evaluation.\nNonetheless, we still view the intersection of attribution and robust training as a promising direction in LLMs. We envision that ABO and robust training approaches can be complementary: future work could explore whether integrating attention bias optimization with robustness-enhancing objectives might yield even more faithful attributions.\n\n\n**Q5. Clarifying the Problem Setting.**\n\n**A5.** Thank you for raising this distinction. We would like to clarify that **ABO does not require the underlying LLM to be explicitly trained with interpretability in mind**, in fact, such LLMS are currently rare in practice. Our goal is to develop a **practical and precise attribution method** that can operate on standard pre-trained (or frozen) LLMs without requiring any model modifications. Like many saliency-based attribution methods, ABO is also **designed for post-hoc, white-box settings**, where we have access to the model’s internal attention mechanism.\n\nAs we responded to Q3, we view ABO as an improved **complement to, rather than a replacement for,** existing attribution methods. We will make this distinction clearer in the revised version.\n\n\n**Q6. Interaction Between ABO and Model Smoothness?**\n\n**A6.** As we responded to Q2, our method does **not explicitly assume or require smoothness** in the model’s input-to-output mapping. While we agree that sharp local irregularities (e.g., abrupt gradient transitions) could, in theory, pose challenges for optimization-based attribution, our empirical results suggest that **ABO remains robust and stable across a wide range of applications**, including NIAH stress test (Sec. 5.1), toxicity suppression (5.2), sentiment steering (5.3), and LLM error correction (5.4). These experiments collectively span various data modalities, context lengths, and attribution goals. The consistently strong performance of ABO across all these settings suggests that it does not suffer from instabilities caused by lack of smoothness in practical use cases.\n\n**Q7. Potential for Training-Time Integration.**\n\n**A7.** We appreciate the reviewer’s thought-provoking question. However, we are conservative to incorporating ABO into LLM training as an auxiliary objective. This is because in the era of large-scale language models, modifying the training process for a specific downstream property, such as interpretability, may **compromise core, general-purpose model capabilities**, as also seen in adversarially trained models (see our response to Q4). While integrating interpretability-aware training objectives with ABO is an interesting direction, it requires careful investigation beyond the scope of this work. We will discuss this in our conclusion for future research plans.\n\n\n**Q8. Relation to Existing Literature.**\n\n**A8.** Thank you for the suggestion. We will enrich our literature review per suggestion. We also agree that robust training is an important line of work in interpretability research. However, as we responded to Q1, Q2, Q4, and Q5, our scope is centered on **LLMs**, where robust training techniques are **not widely adopted for LLMs** and may  **degrade core generation capabilities.**"}, {"review_id": "BChxwNwLD1", "reviewer": "Reviewer_6w9t", "summary": "This paper discusses the LLM input saliency methods and proposes the Attention Bias Optimization (ABO) method.\nIt introduces a systematic evaluation framework based on the needle-in-a-haystack (NIAH) paradigm to stress-test attribution methods, revealing that existing techniques assign substantial importance to irrelevant tokens.\nTo address this issue, they propose ABO, which frames token attribution as an optimization problem over attention biases, demonstrating superior performance across various tasks, including toxicity suppression, sentiment steering, and reasoning error correction.", "questions": "Please refer to the weakness section\n\n- Why share bias parameters across all attention heads and layers?", "limitations": "Yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n\n- The NIAH-based stress testing framework embeds secret messages in distractor text with known ground-truth saliency distributions, which is well-formulated and could become a standard benchmark for future saliency research.\n\n- The proposed method ABO is well-motivated and provides a more direct causal pathway compared to post-hoc gradient-based approaches.\n\n- The paper provides experiments of various models, and the proposed methods show significant improvements in hit ratio. The practical applications (detoxification, sentiment steering, error correction) also indicate applicability in several generation scenarios.\n\n**Weaknesses:**\n\n- The NIAH evaluation primarily tests retrieval-style tasks rather than complex reasoning scenarios where token interactions matter more. The evaluation focuses on relatively simple cases where ground truth is clear-cut, but real-world saliency often involves subtle semantic relationships that may not be captured.\n\n- The paper lacks a theoretical analysis of why attention bias optimization should provide better attribution than existing methods. The connection between attention bias magnitude and true causal contribution is pre-assumed rather than proven. Also, the method requires solving an optimization problem for each input example, which introduces significant computational cost compared to single forward-pass methods.\n\n- The paper should provide more detailed qualitative case studies, e.g. cases when ABO might fail or perform poorly. For instance, how does it handle cases where multiple tokens have legitimate causal relationships?", "was_revised": true, "final_justification": "The authors have addressed my concerns.", "rebuttal": "We sincerely thank you for your constructive feedback and appreciate your positive remarks. Below, we provide detailed responses to address your concerns point by point.\n\n**Q1. The NIAH evaluation primarily tests retrieval-style tasks rather than complex reasoning scenarios where token interactions matter more. The evaluation focuses on relatively simple cases where ground truth is clear-cut, but real-world saliency often involves subtle semantic relationships that may not be captured.**\n\n**A1.** Thanks for this comment. \nFirst, our evaluation is **not limited to retrieval-style scenarios**. As shown in Sections 5.2–5.4, we evaluate ABO across **diverse and semantically rich real-world applications**, including **zero-shot toxicity suppression, sentiment steering**, and **error correction in multi-step reasoning tasks**. These experiments demonstrate that ABO **not only passes basic attribution sanity checks, but also performs robustly** in settings where **token interactions and causal reasoning matter**.\n\nSecond, we would like to stress that the **simplicity of the NIAH setup is a deliberate and important design by choice** – it provides a controlled setting with clear ground truth, which allows us to **rigorously sanity-check attribution methods**. In fact, the NIAH test reveals that **many widely used attribution techniques fail even in these simple settings** with **clear-cut** ground truth (see Tables 1 and 2). We believe this limitation has not been found in the literature, and it represents a minimal requirement that attribution methods should meet before applying to more complex scenarios.\n\n**Q2. The paper lacks a theoretical analysis of why attention bias optimization should provide better attribution than existing methods. The connection between attention bias magnitude and true causal contribution is pre-assumed rather than proven.**\n\n**A2.** We appreciate the reviewer’s point! A theoretical analysis would certainly strengthen the work. However, given that our goal is to attribute behavior in billion-parameter LLMs (and potentially larger models), establishing rigorous theoretical guarantees is challenging, if not infeasible. We view this as a limitation of our work, and more broadly, a limitation shared by much of the interpretability literature in the era of LLMs. We will discuss this limitation in the revision. \n\nWe also acknowledge the importance of “validate the link between attention bias magnitude and causal contribution”, as mentioned by the reviewer, which is why we included the empirical study at  (Lines 178–209, Figure 5) in Sec. 4. While not a formal proof, this experiment demonstrates that in a controlled and interpretable setting, manipulating attention bias alone can deterministically shift the model’s final output. This serves as a strong empirical justification for using optimized bias magnitude as a proxy for causal influence, and it guided the design of our attribution method.\n\n\n**Q3. Also, the method requires solving an optimization problem for each input example, which introduces significant computational cost compared to single forward-pass methods.**\n\n**A3.** We agree that per-instance optimization introduces higher computational cost than  single forward-pass methods. In fact, we have explored multiple alternative approaches that avoid backpropagation, but found that they consistently underperform in attribution accuracy. This is also evidenced by our experimental results, where ABO substantially outperforms other methods across a range of settings.\n\nHowever, the computational overhead introduced by ABO is manageable, as it optimizes input-dimension-aligned parameters of size $L$, same as gradient-based saliency methods. For example, in our experiment on the NIAH stress test with a context of 10K, ABO takes approximately less than 2 minutes per data instance on average.\n\n\n**Q4. The paper should provide more detailed qualitative case studies, e.g. cases when ABO might fail or perform poorly. For instance, how does it handle cases where multiple tokens have legitimate causal relationships?**\n\n**A4.** We appreciate the reviewer’s insightful suggestion. Following that, we peered into the failure cases, especially those involving **multiple tokens with legitimate causal relationships.**\n\nSpecifically, one challenge we observed arises from **how the learned bias scores are converted into final attribution scores.** The raw attention bias values are unbounded and can be both positive and negative, which makes them unsuitable for direct interpretation. We apply a softmax over the learned biases to produce normalized attribution scores. We used this transformation because of  two main benefits: it maps large negative values to near-zero importance and provides a probability-like interpretation across tokens. However, it also introduces sensitivity, especially in scenarios where several tokens have comparable causal effects. In such cases, the softmax can sharply amplify minor differences in the raw bias values, potentially skewing the final attribution toward one token over others due to random initialization or early optimization dynamics.\n\nIn the revision, we will add more detailed qualitative illustrations (e.g., in the style of Figure 3) to demonstrate the failure cases and the lessons that we could learn.  We also see these failure modes not only as limitations, but also as promising directions for future work, for instance, exploring alternative normalization schemes or uncertainty-aware attribution mechanisms.\n\n\n**Q5. Why share bias parameters across all attention heads and layers?**\n\n**A5.** Thank you for this excellent question. We initially experimented with assigning separate learnable attention bias parameters for each layer and head. However, we found that this added complexity did not improve attribution performance. On the contrary, it introduced significant optimization challenges: convergence became slower, and the increased parameter freedom led to instability due to cumulative interactions across layers. In contrast, sharing bias parameters across all heads and layers provided a more stable and lightweight optimization process, while still delivering strong attribution performance. This simplification also reduces the computational and memory overhead, making ABO more practical as a plug-in interpretability tool. We will add this ablation study and discussion in the revision."}, {"review_id": "FqFAbIdOgJ", "reviewer": "Reviewer_iMDk", "summary": "The paper assesses the reliability seven popular input saliency evaluation protocols and reaches the conclusion that existing methods assign importance to non-relevant tokens through the needle-in-a-haystack (NIAH) experiment. In addition, the problem is more severe under long input length. Attention Bias Optimization (ABO) is proposed to resolve this problem. ABO learns the attention bias to be injected in each token that maximizes the probability of generating the target output. Higher attention bias indicates higher saliency of the input token. \nABO robustly outperforms existing methods not only in NIAH but also other practical scenarios.", "questions": "Why do we measure attention bias instead of original attention value plus attention bias? My concern is that if the attention of a relevant token is already high, the learned bias would turn out to be small, as adding additional bias does not make much impact on the final probability of outputting the  target token. Consider the example in Section 4, Line 178, if I set the target token as the first secret message \"Yin\", what would be the learned attention bias of \"Yin\"? What would be the original attention of \"Yin\"?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strength in Quality**: Abundant convincing experiments are conducted to support the claim of the paper.  First, the paper investigates the NIAH setup, in which the relevant tokens are well-defined. Therefore, it is persuasive to show the limitations of existing evaluation methods. Second, the paper runs multiple practical experiments such as zero-shot detoxification, sentiment steering, and reasoning-error correction, in order to demonstrate the real-world strength of the proposed ABO method. \n\n**Strength in Clarity**: The paper is well-structured. First, a simple toy experiment, NIAH,  is studied to show limitations of existing methods. Then, the ABO method is proposed to resolve the limitations. Finally, more practical tests are conducted to demonstrate the strength of ABO.\n\n**Strength in Significance**:  The paper contributes to the interpretability of LLMs by proposing a reliable saliency evaluation protocol. The evaluation is computationally efficient because no model training is involved. \n\n**Strength in Originality**: The paper introduces a new norm of evaluating saliency of input tokens that intervenes on generation only instead of model weights.\n\n**Weakness in Significance**: ABO is motivated specifically by the NIAH setting. As a result, although the performance of ABO is significantly higher than baselines in NIAH, it is not that significant in other practical scenarios. For instance, in Table 3 ABO has similar performance with AT2 in many columns.", "was_revised": true, "final_justification": "I keep my original score of 5. The authors clearly resolve all my concerns. \n\nWhy no higher score: The significance of the paper's approach in practice still needs to be test by time despite the three scenarios tested in the paper.", "rebuttal": "Thank you for your careful review and encouraging remarks. We have carefully considered your suggestions, and we provide detailed responses and clarifications to each of your comments below.\n\n**Q1. Weakness in Significance.**\n\n**A1.** While ABO is indeed motivated by the NIAH setting as a rigorous stress test, our evaluation goes well beyond this controlled setup. As detailed in Sec. 5.2–5.4, we demonstrate ABO’s effectiveness across a range of real-world applications, including zero-shot toxicity suppression, sentiment steering, and multi-step reasoning error correction. \n\nRegarding Table 3, while ABO and AT2 may appear similar under low pruning ratios (e.g., 10% or 50%), the performance gap becomes substantial under more challenging conditions. For instance, at higher pruning ratios (90%, 95%, 97%), ABO consistently retains significantly more task-relevant information, outperforming AT2 by a wide margin (e.g., up to +15% in accuracy at 95% pruning on QWen3-14B), especially as attribution difficulty increases.\n\nWe will make this broader significance more precise in the revised version.\n\n\n**Q2. Why do we measure attention bias instead of original attention value plus attention bias?**\n\n**A2.** Thank you for the insightful question. While both the original attention scores and the optimized bias terms contribute to the model’s output during inference, we deliberately use only the learned attention bias for attribution, motivated by two key reasons below. \nSeparation from confounding attention patterns: Original attention scores are distributed across many heads and layers, each with distinct contextual roles and dynamic ranges. These scores often reflect structural or positional biases rather than causal token influence. Including them directly in attribution would risk entangling meaningful signal with these confounding factors. \nStability and interpretability of the bias space: The learned attention bias vector is optimized end-to-end in a single, unified space. As such, it provides a clean, direct measure of how much “perturbations” (sensitivity or additional emphasis) a token needs to shift the model's output, offering a more faithful and causally grounded saliency signal. Combining it with the original attention would require complex normalization across layers and heads, introducing potential scale mismatches and reducing attribution clarity.\n\n\n**Q3. My concern is that if the attention of a relevant token is already high, the learned bias would turn out to be small, as adding additional bias does not make much impact on the final probability of outputting the target token. Consider the example in Section 4, Line 178, if I set the target token as the first secret message \"Yin\", what would be the learned attention bias of \"Yin\"? What would be the original attention of \"Yin\"?**\n\n**A3.** Thank you for the insightful observation. We agree that a relevant token with already-high attention might require less additional bias to influence the output, and we acknowledge that token position can affect baseline attention. To examine this, we conducted a series of controlled “position rotation” experiments based on the setup in Section 4 (Lines 178–181), where three secret messages (e.g., “Yin,” “Vin,” “Kin”) are inserted into the prompt at fixed positions.\nIn the original Figure 5 setting, “Yin” appears first, “Vin” second, and “Kin” third, and the model naturally selects “Yin” as the answer. We apply a positive attention bias only to “Vin” (not the top-1 token), and observe that its generation probability increases step-by-step across decoding steps, eventually overtaking “Yin” and flipping the final output to “Vin.” This shows that even when the target token already receives moderate attention, a learned bias can still exert causal influence and change the model’s behavior.\n\nTo test whether this effect is consistent and not due to position-specific attention, we rotate the positions of “Yin,” “Vin,” and “Kin” in all possible orders, and repeat the same procedure: always applying positive attention bias to the message that was not selected by the original model. Across all permutations, we consistently observe the same outcome: the biased token gains generation probability over time, and often replaces the original top-1 answer. This holds regardless of whether the target token is at the beginning, middle, or end of the message list.\n\nThese results confirm that the attention bias learned by ABO captures causal influence that goes beyond simply reinforcing tokens with already-high baseline attention. We will include these detailed experimental results and analysis in the revision and Appendix to clarify this key intuition."}, {"review_id": "2YsdYaHE9B", "reviewer": "Reviewer_Rxaz", "summary": "This paper explores the degradation of existing saliency methods in long-context scenarios and proposes a solution: Attention Bias Optimization (ABO). The authors conduct extensive experiments to validate ABO's effectiveness and further demonstrate its utility in controllable text generation. Overall, this is an interesting and in-depth study with meaningful contributions.", "questions": "Please see Strengths And Weaknesses", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1.    The observed relationship between attribution quality and input length is both novel and insightful.\n2.    The proposed ABO method is simple yet effective.\n3.    The paper presents its motivation, methodology, and applications clearly and coherently.\n\nWeakness:\n\nBelow are my questions and concerns. I list more than usual because I believe the paper is promising and could be strengthened with further clarifications. \n\n1.\tRegarding the statement on line 121: does the LLM achieve 100% accuracy on NIAH queries across all context lengths (from 500 to 10k)?\n\n2.\tLines 156–158 reference \"100 tokens,\" but Figure 2 shows results for 500 tokens. Should this be corrected to 500 tokens?\n\n3.\tIn the proposed method, is the learned alpha directly used as the final attribution score for each token?\n\n4.\tIn Figure 5, the output probability appears to be influenced by both the induced bias and the original attention scores (lines 194 and 200). However, the final ABO method uses only the bias scores. Please justify why the original attention scores can be ignored.\n\n5.\tFor the main experiments (Tables 1 and 2), please provide dataset statistics. Also, how many training samples were used to train ABO?\n\n6.\tThe experiments in Tables 1 and 2 may not present a fair comparison. ABO is trained on the same data used for testing, while other methods lack access to this prior knowledge. This is more akin to comparing supervised with unsupervised methods, which disadvantages the baselines.\n\n7.\tIn contrast, the experiment in Table 3 offers a fairer comparison and is more convincing if ABO utilizes the NIAH dataset for training instead of LongBench-2wikimqa_e. May I confirm waht is the training data for this experiment? Also,  I recommend including more experiments of this nature.\n\n8.\tIs ABO only applicable in long-context scenarios? Testing it in classic settings (e.g., TDD experiments) would help validate its generalizability.\n\n9.\tAnother concern is whether ABO truly learns attribution scores or merely copies input repeated tokens. Simple copying can also reduce training loss but may not generalize beyond NIAH scenarios. This distinction needs to be empirically verified. This is also another reason I ask more experiments as questions 7 and 8 above.\n\n10.\tIn Sections 5.2 and 5.3, many settings and operations for controllable text generation seem directly adopted from the TDD framework. The authors should explicitly cite TDD and clearly state, like “We follow the experimental settings and procedures for controllable text generation as described in TDD.”", "was_revised": true, "final_justification": "The authors have addressed all my concerns, so I raised my score accordingly.", "rebuttal": "We greatly appreciate your thoughtful feedback and insightful comments. Your suggestions have been invaluable in helping us improve the clarity and quality of our work. Below, we address each of your points in detail.\n\n**Q1. Regarding the statement on line 121: does the LLM achieve 100% accuracy on NIAH queries across all context lengths (from 500 to 10k)?**\n\n**A1.** Yes, your understanding is correct: for every NIAH example in our evaluation, we ensured that the LLM produced the correct answer to the query before applying any saliency method. This serves as a necessary prerequisite for  evaluating the correctness of saliency under the correct prediction and for conducting additional analysis in the NIAH setup . This is also why in Table 1 and 2, different models may use different max test length, which is the max length that the model makes no mistakes in NIAH prior to further assessing saliency methods. \n\n**Q2. Lines 156–158 reference \"100 tokens,\" but Figure 2 shows results for 500 tokens. Should this be corrected to 500 tokens?**\n\n**A2:** Thanks for your careful reading! You are correct: It should be 500 tokens. We will correct this typo in our revision.\n\n**Q3. In the proposed method, is the learned alpha directly used as the final attribution score for each token?**\n\n**A3.** We apologize for the ambiguity. No, the raw learned alpha values are not directly used as the final attribution scores. Since the alpha values can be both positive and negative, we apply a softmax to convert them. This ensures that strongly negative values are suppressed (near-zero importance), and the attribution scores sum to one, making them interpretable as relative importance percentages across the input tokens. We will clarify this in the revision.\n\n**Q4. In Figure 5, the output probability appears to be influenced by both the induced bias and the original attention scores (lines 194 and 200). However, the final ABO method uses only the bias scores. Please justify why the original attention scores can be ignored.**\n\n**A4.** We appreciate the reviewer’s keen observation, during inference, both the learned attention bias and the original attention scores jointly influence the model’s output. However, when computing attribution scores, we intentionally exclude the original attention scores and rely solely on the optimized bias terms. This design is based on the following practical  considerations.\n\nSeparation from confounding attention patterns: Original attention scores are distributed across many heads and layers, each with distinct contextual roles and dynamic ranges. These scores often reflect structural or positional biases rather than causal token influence. Including them directly in attribution would risk entangling meaningful signal with these confounding factors. \nStability and interpretability of the bias space: The learned attention bias vector is optimized end-to-end in a single, unified space. As such, it provides a clean, direct measure of how much “perturbations” (sensitivity or additional emphasis) a token needs to shift the model's output, offering a more faithful and causally grounded saliency signal. Combining it with the original attention would require complex normalization across layers and heads, introducing potential scale mismatches and reducing attribution clarity.\n\n\n**Q5. For the main experiments (Tables 1 and 2), please provide dataset statistics. Also, how many training samples were used to train ABO?**\n\n**A5.** Thank you. To clarify, ABO does not involve “training”. It is not a method that pretrains or fine-tunes a model using a dataset. Instead, given a frozen model, a specific input prompt, and a target output token, we perform a lightweight optimization over the attention bias vector, which results in the input saliency. This is consistent with other post-hoc input saliency methods, such as gradient-based attribution techniques. While our optimization uses backpropagation, it operates entirely on the given example, i.e., no additional data or training samples are involved. And the dimension of the optimization variables (attention bias vector) is aligned with the input example dimension. Therefore, there is no additional training phase for ABO.\n\nRegarding dataset statistics: As stated in the captions of Tables 1 and 2, our evaluation in the NIAH setup is conducted as follows: for each context length (e.g., 1K–10K tokens), we insert the secret message at 9 different positions, uniformly spanning from 10% to 90% of the prompt. The reported attribution accuracy is the average over these positions, ensuring robustness across varying insertion points.  Apart from this, no additional datasets are involved.\n\nIn short, NIAH is introduced as a new evaluation framework to stress-test attribution methods, not as a training set for ABO. ABO itself is a post-hoc input saliency method that operates on given models and significantly outperforms existing input attribution methods across a range of tasks, including NIAH stress testing (Sec. 5.1), toxicity suppression (Sec. 5.2), sentiment steering (Sec. 5.3), and LLM error correction (Sec. 5.4).  \n\n\n**Q6. The experiments in Tables 1 and 2 may not present a fair comparison. ABO is trained on the same data used for testing, while other methods lack access to this prior knowledge. This is more akin to comparing supervised with unsupervised methods, which disadvantages the baselines.**\n\n**A6.** This might be a misunderstanding of ABO. As clarified in our previous response ABO does not involve any training or external training dataset. The optimization in ABO is entirely post-hoc, i.e., performed for each test instance given an already-trained model, without requiring any external data, supervision, or prior knowledge. Therefore, ABO is not \"trained on the test data,\" and the comparison remains fair. \n\n**Q7. In contrast, the experiment in Table 3 offers a fairer comparison and is more convincing if ABO utilizes the NIAH dataset for training instead of LongBench-2wikimqa_e. May I confirm what the training data for this experiment is? Also, I recommend including more experiments of this nature.**\n\n**A7.** Thank you for the suggestion. As mentioned in our responses to Q5 and Q6, ABO does not involve any training on datasets such as NIAH or LongBench-2wikimqa_e. NIAH is introduced as a new evaluation framework to stress-test attribution methods, not as a training set for ABO.\n\n\n**Q8. Is ABO only applicable in long-context scenarios? Testing it in classic settings (e.g., TDD experiments) would help validate its generalizability.**\n\n**A8.** Thank you for the suggestion. ABO is not limited to long-context scenarios. In fact, we evaluated its performance across a range of context lengths from 1K to 10K tokens in Tables 1-2, demonstrating consistent effectiveness even in shorter inputs. \n\nWe agree that TDD presents several strong examples of attribution evaluation, and we have already incorporated some of its experimental setups, specifically those in Table 4 (toxicity suppression) and Table 5 (sentiment steering). As for the other classic attribution experiments used in TDD, we chose not to adopt them initially due to the lack of clearly defined ground-truth saliency labels, which makes rigorous and objective evaluation challenging.\n\n**Q9. Another concern is whether ABO truly learns attribution scores or merely copies input repeated tokens. Simple copying can also reduce training loss but may not generalize beyond NIAH scenarios. This distinction needs to be empirically verified. This is also another reason I ask more experiments as questions 7 and 8 above.**\n\n**A9.** We appreciate the reviewer’s concern. However, we believe this may stem from a misunderstanding similar to those addressed in Q5–Q7: ABO does not involve training, and therefore the notion of “reducing training loss” does not apply in the NIAH setting. To stress, ABO performs post-hoc optimization for input saliency. There is no learning across examples or fitting to a dataset. \n\nFurthermore, the effectiveness of ABO extends beyond the NIAH setup, as shown in Tables 4 and 5, ABO identifies causal tokens in diverse real-world tasks such as sentiment steering and toxicity suppression, where the key tokens are not duplicated. These results support the claim that ABO meaningfully captures attribution rather than relying on token repetition.\n\n**Q10. In Sections 5.2 and 5.3, many settings and operations for controllable text generation seem directly adopted from the TDD framework. The authors should explicitly cite TDD and clearly state, like “We follow the experimental settings and procedures for controllable text generation as described in TDD.”**\n\n**A10.** Thank you for pointing this out. We appreciate the reminder, and we will make the  citation and clarification more explicit in the revised version."}]

Author rebuttal:

Meta-review: This paper proposes a stress-testing framework based on the Needle-in-a-Haystack (NIAH) paradigm, where a hidden "secret message" is embedded in long passages of irrelevant text.   The authors find existing saliency methods misattribute saliency to irrelevant context, and this error worsens with longer inputs. Based on this observation a new Attribution method is proposed based on  Attention Bias Optimization (ABO), which optimizes these biases to maximize the probability of a target output token, with a sparsity inducing regularizer. The effectiveness of this approach is demonstrated across several realistic tasks  (detoxification, sentiment steering, error correction) . 

This work represents a solid improvement over a family of important and realistic tasks. However, the benchmark and solution are limited to NIAH settings rather than more complex reasoning scenarios. Also the result will be more complete if the relationship to related work can be explained in more detail (e.g., Attribution vs. Model Quality, Frozen Model vd fine tuned, Off-Manifold Robustness).
Decision: Accept (spotlight)