Attribution-Guided Decoding

## Abstract

Abstract The capacity of Large Language Models (LLMs) to follow complex instructions and generate factually accurate text is critical for their real-world application. However, standard decoding methods often fail to robustly satisfy these requirements, while existing control techniques frequently degrade general output quality.
In this work, we introduce Attribution-Guided Decoding (AGD), an interpretability-based decoding strategy.
Instead of directly manipulating model activations, AGD considers a set of high-probability output token candidates and selects the one that exhibits the highest attribution
to a user-defined Region of Interest (ROI). This ROI can be flexibly defined over different parts of the model’s input or internal components, allowing AGD to steer generation towards various desirable behaviors.
We demonstrate AGD’s efficacy across three challenging domains.
For instruction following, we show that AGD significantly boosts adherence ( e.g., improving the overall success rate on Llama 3.1 from 66.0% to 79.1%).
For knowledge-intensive tasks, we show that guiding generation towards usage of internal knowledge components or contextual sources can reduce hallucinations and improve factual accuracy in both closed-book and open-book settings.
Furthermore, we propose an adaptive, entropy-based variant of AGD that mitigates quality degradation and reduces computational overhead by applying guidance only when the model is uncertain.
Our work presents a versatile, more interpretable, and effective method for enhancing the reliability of modern LLMs.

## 1 Introduction

Figure: Figure 1: Illustration of the Attribution-Guided Decoding (AGD) framework.. (a) A Region of Interest (ROI) is defined over a relevant area, such as the user’s instruction. (b) Next, a standard forward pass generates a candidate set of high-probability tokens, both “festival” (most probable) and “royal” are included. (c) An attribution method computes a relevance score for each candidate on the input tokens, quantifying a candidate’s dependence on the ROI. (d) AGD selects the token (“royal”) with the highest aggregated attribution score on the ROI, thereby satisfying the constraint.
Refer to caption: x1.png

Large Language Models (LLMs) have emerged as powerful tools capable of generating fluent, coherent and contextually relevant text across numerous applications (Brown et al., 2020; Ouyang et al., 2022; Achiam et al., 2023).
Despite their success, their reliability is undermined by critical failures, most notably inconsistent adherence to user instructions (Sun et al., 2023; Zhou et al., 2023a; Zeng et al., 2024)
and a tendency to generate non-factual, or *hallucinated* (Wei et al., 2024), information.
A key enabler of the former has been instruction tuning, which teaches models to better follow human commands expressed in natural language (Zhou et al., 2023b). However, despite these advances, models still struggle to follow complex constraints, especially in lengthy contexts (Liu et al., 2024) or multi-turn dialogues (Li et al., 2024; Qin et al., 2025) where constraints can drift.
These shortcomings are not minor flaws but fundamental barriers to deploying LLMs in high-stakes environments that demand precision and trustworthiness.

To address these issues, significant research has focused on developing methods to control and guide the LLM generation process. Standard decoding strategies like top-$k$ (Fan et al., 2018) or nucleus sampling (Holtzman et al., 2020) can modulate the randomness of the output but offer little direct control over semantic properties like factuality or instruction adherence. A more direct line of work involves steering model behavior by directly manipulating the model’s internal activations to guide it towards a desired style or content (Li et al., 2023a; Rimsky et al., 2024). While often effective at enhancing the targeted attribute, these interventions come with a significant drawback: a frequent degradation of general text quality (Arditi et al., 2024; Stolfo et al., 2025). Altering the internal representations can push the model into out-of-distribution states, leading to increased perplexity, repetitive outputs, and a loss of nuance. This creates an undesirable trade-off where users must choose between better control and higher-quality generation.

In this paper, we ask: can we guide generation towards a desired behavior without directly manipulating the model’s internal representations? We propose a new paradigm, Attribution-Guided Decoding (AGD), which reframes decoding as a search for the token that is best justified by a given rationale. The core idea is to leverage post-hoc attribution methods not just for understanding model decisions (Yin & Neubig, 2022), but for guiding them.
The process is illustrated in Figure [1](https://arxiv.org/html/2509.26307v1#S1.F1).
At each generation step, we first identify a small set of plausible next tokens from the model’s output distribution.
Then, using a feature attribution method, we quantify how much each candidate token *relies* on a specified part of the input, e.g. instruction, or model’s internals.
Finally, the token with the highest attribution score is selected for generation.
By restricting its choice to a set of high-probability candidates, AGD maintains fluency and coherence, mitigating degradation in output quality.

AGD is a versatile, fine-tuning-free framework that operates at decoding time, making it broadly applicable to modern LLMs. Although our method incurs additional computational overhead due to the necessity of computing token-level relevance scores, this trade-off results in marked improvements in controllability and interpretability. As contributions, we:

- •
Introduce AGD, a novel, flexible framework for steering LLM generation via post-hoc analysis of candidate tokens to make generation process more grounded.
- •
Apply AGD to instruction following and propose an entropy-based adaptive mechanism that dynamically applies guidance, achieving strong instruction adherence while preserving output quality and reducing computational cost.
- •
Demonstrate the versatility of AGD by targeting parametric knowledge heads to improve factuality and contextual sources to enhance in-context grounding.
- •
Show that AGD provides insights into interpretability, for instance offering an explanation why certain tokens are chosen over others during the generation process.

## 2 Related work

#### Controlled Text Generation

Significant research has focused on steering LLM behavior at inference time without costly retraining. One prominent line of work is activation engineering, where steering vectors are added to the model’s residual stream to guide its internal states towards desired concepts or styles (Subramani et al., 2022; Burns et al., 2023; Tigges et al., 2024; Rimsky et al., 2024). While powerful, these methods directly intervene in the model’s forward pass, fundamentally altering its computation in a way that can harm general output quality (Arditi et al., 2024; Stolfo et al., 2025).
Other techniques modify the output logits, often using contrastive decoding approaches to improve properties like factual accuracy (Li et al., 2023b; Shi et al., 2024; Chuang et al., 2024).
These methods can be broadly categorized as *interventionist* as they actively modify the model’s internal states or output distributions. In contrast, AGD is a *selectionist* method. It does not alter the model’s forward pass or logits. Instead, it utilizes model’s original output distribution and uses attribution methods as a way to select the candidate that best aligns with a specified goal.

#### Instruction-Following

The ability of LLMs to follow commands has been significantly advanced by instruction tuning
(Ouyang et al., 2022; Wei et al., 2022; Gupta et al., 2022; Longpre et al., 2023; Chung et al., 2024), with a corresponding growth in benchmarks for evaluation under varying levels of complexity and context (Zhou et al., 2023a; Zeng et al., 2024; Qin et al., 2024; Jiang et al., 2024).
Current post-training methods aimed at enhancing instruction following often require model- and task-specific preparation, such as profiling (Zhang et al., 2024), training linear probes (Heo et al., 2025), or computing steering vectors (Stolfo et al., 2025). This preparation, combined with the need to tune hyperparameters like steering weights and intervention layers, can limit scalability.
In contrast, our approach operates entirely at inference time.

#### Attribution Methods

Attribution methods aim to explain a model’s prediction by assigning attribution scores to its inputs or internal components.
While attention weights are a natural candidate for analysis, their unreliability as faithful explanations motivates the use of saliency-based methods Bastings & Filippova (2020). These techniques range from simpler gradient-based methods such as Input$\times$Gradient (I$\times$G) (Simonyan et al., 2014; Sundararajan et al., 2017; Smilkov et al., 2017) to more robust techniques like Layer-wise Relevance Propagation (LRP) (Bach et al., 2015; Voita et al., 2019; Achtibat et al., 2024).
Historically, these attribution methods have been used mostly for post-hoc analysis – to understand and debug a model’s behavior after a decision has been made (Lapuschkin et al., 2019; Anders et al., 2022; Pahde et al., 2023; Achtibat et al., 2023). To our knowledge, our work is the first to integrate these analytical tools directly into the decoding loop of LLMs, transforming them from a passive, explanatory role into an active, generative one. By doing so, we not only steer the model’s output but also provide a rationale for each selection.

## 3 Method

An autoregressive language model $\theta$ receives a sequence of input tokens $x=(x_{1},x_{2},\dots,x_{n})$, referred to as the *prompt*, and generates an output sequence $y=(y_{1},y_{2},\dots)$, one token at a time.
Let $\mathcal{V}$ denote the model’s vocabulary - the full set of discrete tokens that the model can emit. At each decoding step $t$, the model predicts a probability distribution over $\mathcal{V}$, denoted as
${\bm{p}}_{\theta}\!\left(y_{t}\mid x,\,y_{<t}\right)$,
conditioned on the input $x$ and the previously generated prefix $y_{<t}=(y_{1},\dots,y_{t-1})$.

### 3.1 Feature Attribution

Attribution methods aim to explain a model’s prediction by quantifying the contribution of its input or internal components
to a specific output.
We define $\Omega$ as the set of all attributable components in the model, such as its input token embeddings or attention heads.
A general attribution function $\mathcal{A}$ maps a token $c$ to a set of relevance scores over these components:

$$ $\mathcal{A}_{\theta}(c\mid x,y_{<t})\rightarrow\{r_{\omega}\mid\omega\in\Omega\},$ (1) $$

where $r_{\omega}$ represents the relevance of component $\omega$ to the model’s logit for token $c$.
In principle, any attribution method could be used, but they involve different trade-offs between faithfulness and computational cost. Perturbation-based methods, while often faithful, are too slow for decoding as they require numerous forward passes (Lundberg & Lee, 2017). Gradient-based methods like I$\times$G (Simonyan et al., 2014) are more efficient, requiring only a single backward pass, but can produce noisy and unreliable attributions due to the non-linearities in network architectures (Ali et al., 2022).

To balance these factors, we adopt Layer-wise Relevance Propagation (LRP) (Bach et al., 2015), which propagates the output logit value backward through the network in a layer-wise manner. Its adaptation for Transformers, AttnLRP (Achtibat et al., 2024), includes specific rules to handle non-linear components like self-attention and layer normalization, resulting in more stable and faithful relevance scores than simpler gradient methods (Arras et al., 2025). Crucially, LRP is as efficient as gradient-based methods, making it suitable for integration into the decoding loop. Therefore, we select LRP as our primary attribution method, additionally reporting results for I$\times$G for comparison.

### 3.2 Attribution-Guided Decoding

#### Candidate Set Selection

To control generation, decoding algorithms often restrict sampling to a subset of likely candidates.
We restrict sampling to a small but plausible subset of the vocabulary, which we term the *candidate set* $\mathcal{C}_{t}\subseteq\mathcal{V}$. At each timestep $t$, this set is formed by first selecting the $k$ tokens with the highest probabilities from the distribution ${\bm{p}}_{\theta}(y_{t}\mid x,y_{<t})$. From this initial set, we then filter out any token whose probability is below a minimum threshold $\pi_{\min}$. This step ensures that we only consider tokens that the model already deems likely, thereby preserving fluency.

#### Attribution Scoring

For each candidate token $c\in\mathcal{C}_{t}$, we compute an attribution score that quantifies its reliance on a specific Region of Interest (ROI) $R$. It can be defined over any part of the model’s input or internal components, such as a subset of input embeddings or specific attention heads, making AGD adaptable to various tasks.
The process starts from the model’s pre-softmax logit for the candidate token $c$.
Using an attribution method $\mathcal{A}$ (e.g., LRP), we backpropagate a signal from this logit to assign relevance scores $r_{\omega}=\mathcal{A}_{\theta}(c\mid x,y_{<t};\omega)$ to the components $\omega$ of the model.
The total attribution score $S(c,R)$ for a candidate $c$ with respect to $R\subseteq\Omega$ is the sum of attributions over all components within that region:

$$ $S(c,R)=\sum_{\omega\in R}r_{\omega}=\sum_{\omega\in R}\mathcal{A}_{\theta}(c\mid x,y_{<t};\omega).$ (2) $$

A higher score $S(c,R)$ indicates that token $c$ was more influenced by the components in $R$.

#### Token Selection

Finally, we select the token $y_{t}$ from the candidate set $C_{t}$ that maximizes the attribution score with respect to the Region of Interest $R$:

$$ $y_{t}=\underset{c\in C_{t}}{\arg\max}\,S(c,R).$ (3) $$

By replacing the standard probability-maximization objective with an attribution-maximization one, we guide the model to generate tokens that are most consistent with the function encapsulated by $R$.

### 3.3 Defining Region of Interest

The flexibility of AGD lies in how the ROI is defined. By selecting different subsets of the model’s attributable components ($R\subseteq\Omega$), we can steer generation towards various desirable behaviors.

#### Instruction Following

For tasks requiring adherence to specific constraints, we partition the input prompt $x$ into an instruction part, $x_{I}$, and a task-specific query, $x_{T}$ (example in Table [2](https://arxiv.org/html/2509.26307v1#S4.T2)). Our objective is to select tokens that are maximally influenced by $x_{I}$.
In this case, the ROI is defined as the set of input token embeddings corresponding to the instruction part of the prompt:

$$ $R_{I}=\{e_{i}\mid x_{i}\in x_{I}\},$ (4) $$

where $\{e_{1},\dots,e_{n}\}$ is the sequence of input embeddings.
The attribution score $S(c,R_{I})$ for each candidate token $c\in\mathcal{C}_{t}$ is then computed by Equation [2](https://arxiv.org/html/2509.26307v1#S3.E2) by summing relevance over these embeddings.
This process selects the token that is most grounded in the instruction part of the prompt.

#### Factuality & In-Context Retrieval

AGD can also be used to improve factual accuracy by defining the ROI over specialized attention heads, leveraging prior work that identifies heads crucial for knowledge processing and retrieval (Jin et al., 2024; Kahardipraja et al., 2025).

- •
Closed-Book Factuality: To reduce hallucinations, the ROI ($R_{P}$), is the set of pre-identified parametric knowledge heads. The attribution score $S(c,R_{P})$ measures how much the prediction of token $c$ relies on these heads. By maximizing this score, we encourage the model to select tokens based on the factual knowledge encoded within its parameters.
- •
Open-Book Retrieval:
To ground the output in provided evidence, we can define the ROI in two ways: (1) as the set of in-context retrieval heads ($R_{IC}$), or (2) as the input embeddings of the context document itself ($R_{C}=\{e_{i}\mid x_{i}\in x_{\text{context}}\}$). Both approaches aim to select tokens that are maximally grounded in the provided evidence.

In Appendix LABEL:app:extraction, we give additional details on the identification process of these specialized heads.

### 3.4 Adaptive Guidance with Entropy-Gating

Applying AGD at every decoding step is computationally expensive due to multiple backward passes of the attribution and can degrade text quality when the model is already confident.
To mitigate this, we introduce an adaptive strategy that applies guidance selectively. Motivated by recent work showing that generation trajectories are largely determined by a few high-entropy *critical forks* (Wang et al., 2025), we use the Shannon entropy of the output distribution as a trigger for intervention.
Let $H({\bm{p}}_{t})$ be the entropy of the probability distribution ${\bm{p}}_{\theta}(y_{t}\mid x,y_{<t})$. AGD is only applied when the model is uncertain, i.e., when its output entropy exceeds threshold $\tau$. Otherwise, we default to standard greedy decoding. The final selection rule is:

$$ $y_{t}=\begin{cases}\underset{c\in\mathcal{V}}{\arg\max}\ {\bm{p}}_{\theta}(c\mid x,y_{<t})&\text{if }H({\bm{p}}_{t})<\tau\\ \underset{c\in\mathcal{C}_{t}}{\arg\max}\ S(c,R)&\text{if }H({\bm{p}}_{t})\geq\tau\end{cases}$ (5) $$

This entropy-gating mechanism can significantly reduce the computational overhead of AGD while preserving its benefits for instruction adherence, as intervention is focused only on critical decision points where the model is most likely to deviate from the desired behavior.

## 4 Instruction following

To comprehensively evaluate the effectiveness of our decoding approach on instruction following task, we conduct experiments across three instruction-tuned language models:
Llama 3.1 (8B) (Grattafiori et al., 2024), Qwen 2.5 (7B) (Yang et al., 2024), and Gemma 3 (4B) (Team et al., 2025).
Below, we detail datasets, specify the ROI and the metrics used for evaluation.

### 4.1 Experimental setup

#### Datasets and metrics

To assess the instruction-following ability under verifiable constraints, we utilized the *IHEval* rule following dataset (Heo et al., 2025), which is based on the *IFEval* dataset (Zhou et al., 2023a), covering 25 types of constraints. Each example contains a clear separation between the instruction (system prompt) and the task (user prompt). We select IHEval to isolate and control the constraint-following evaluation, avoiding the complexity introduced in the original IFEval, where instructions are embedded within less structured input. For AGD, the ROI is the set of input embeddings corresponding to the system prompt. As evaluation metrics, we report loose Prompt Level Accuracy (PLA), the proportion of outputs satisfying all constraints, along with Instruction Level Accuracy (ILA), as each example can consist of more than one constraint.
To measure generation quality, we follow Stolfo et al. (2025) and report a Quality Score (QS), which is a fraction of *yes* answers from an LLM evaluator to yes/no questions about the utility of a response,
given that all constraints are satisfied.
These questions were first generated by the same evaluator based on a task-only input (excluding constraint).
We report details of this procedure in the Appendix [B](https://arxiv.org/html/2509.26307v1#A2).
Finally, we report the combined metric (PLA * QS) to balance adherence and quality.

To examine instruction-following in-the-wild, under more complex, multi-turn conversational settings, we leverage the *SysBench* dataset (Qin et al., 2025), a bilingual Chinese-English benchmark containing 500 examples. Each example includes a system prompt with complex constraints and five subsequent user-model turns. The ROI for AGD is the entire system prompt across whole conversation. In line with Qin et al. (2025), we report three metrics: Constraint Satisfaction Rate (CSR), measuring the average proportion of satisfied constraints; Instruction Satisfaction Rate (ISR), measuring the proportion of individual responses fully satisfying constraints; and Session Stability Rate (SSR), measuring the average number of consecutive turns satisfying all constraints from the conversation’s start. Responses are evaluated exclusively by an LLM with respect to the system prompt constraints, thus blending adherence and utility metrics.

**Table 1: Performance on instruction following benchmarks. Higher is better for all metrics (%). AGD subscripts denote the attribution method (I$\times$G or LRP) and whether it is entropy-gated (e). PLA: Prompt-Level Accuracy, ILA: Instruction-Level Accuracy, QS: Quality Score. CSR, ISR, and SSR are composite metrics for the multi-turn SysBench task.**
| Model | Method | IHEval | SysBench |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
|  |  | PLA (ILA) | QS | PLA*QS | CSR | ISR | SSR |
| Llama 3.1 (8B) | Greedy | 66.0 (75.8) | 81.3 | 53.7 | 67.1 | 48.4 | 26.0 |
|  | Nucleus | 63.6 (73.3) | 73.9 | 47.0 | 58.0 | 40.6 | 20.2 |
|  | CAD | 73.9 (81.3) | 72.6 | 53.7 | 72.2 | 58.8 | 32.3 |
|  | AGDI$\times$Ge | 67.1 (76.9) | 82.1 | 55.1 | 67.8 | 50.1 | 27.2 |
|  | AGDI$\times$G | 70.8 (79.6) | 81.8 | 57.9 | 65.1 | 46.5 | 24.2 |
|  | AGDLRPe | 74.5 (82.6) | 76.4 | 56.9 | 74.3 | 58.2 | 33.9 |
|  | AGDLRP | 79.1 (85.0) | 73.2 | 57.9 | 73.3 | 57.3 | 32.2 |
| Qwen 2.5 (7B) | Greedy | 63.2 (72.7) | 74.1 | 46.8 | 67.6 | 47.9 | 27.1 |
|  | Nucleus | 62.8 (72.7) | 75.2 | 47.2 | 64.8 | 44.8 | 24.7 |
|  | CAD | 67.3 (76.6) | 67.4 | 45.4 | 65.7 | 49.2 | 25.2 |
|  | AGDI$\times$Ge | 62.5 (72.5) | 75.9 | 47.4 | 67.3 | 46.9 | 25.1 |
|  | AGDI$\times$G | 65.6 (74.2) | 74.8 | 49.1 | 66.8 | 46.4 | 25.2 |
|  | AGDLRPe | 70.4 (78.3) | 70.6 | 49.7 | 71.1 | 53.0 | 29.9 |
|  | AGDLRP | 70.1 (78.5) | 67.4 | 47.2 | 73.7 | 56.4 | 32.7 |
| Gemma 3 (4B) | Greedy | 84.7 (89.8) | 82.3 | 69.7 | 69.8 | 52.4 | 33.3 |
|  | Nucleus | 83.3 (88.9) | 85.2 | 71.0 | 69.3 | 52.2 | 33.2 |
|  | CAD | 81.0 (87.1) | 73.2 | 59.3 | 73.0 | 57.9 | 36.0 |
|  | AGDI$\times$Ge | 83.0 (88.7) | 87.3 | 72.5 | 69.0 | 51.6 | 32.2 |
|  | AGDI$\times$G | 80.6 (86.9) | 86.6 | 69.8 | 68.5 | 50.4 | 31.8 |
|  | AGDLRPe | 86.7 (91.0) | 81.4 | 70.6 | 73.0 | 57.9 | 36.0 |
|  | AGDLRP | 86.0 (90.5) | 78.4 | 67.4 | 73.2 | 57.8 | 36.5 |

#### Baselines

We compare AGD against standard decoding methods – greedy and nucleus sampling ($p=0.95$) – and the stronger baseline Context-aware Decoding (CAD) (Shi et al., 2024), a method that modifies output logits via contrastive decoding between a prompt with and without the instruction, adapted to improve adherence. For CAD we set the control hyperparameter $\alpha=1$.

#### Settings

To form the candidate set $C_{t}$ (Section [3.2](https://arxiv.org/html/2509.26307v1#S3.SS2.SSS0.Px1)), we apply a top-$k$ constraint and a minimum probability threshold $\pi_{\min}$.
This design ensures that $C_{t}$ remains small and focused, so that attributions are computed only over semantically plausible candidates.
To ensure fair comparison and demonstrate the generality of the method, we fix the hyperparameters across all experiments, setting $k=5$ and $\pi_{\min}=0.05$.
For our entropy-gated variants, we set the activation threshold to $\tau=1.734$. This value corresponds to the 80th percentile of token-level entropy on IHEval and is motivated by prior work on identifying critical generation steps (Wang et al., 2025) (see Appendix LABEL:appendix:entropy).

### 4.2 Results

As shown in Table [1](https://arxiv.org/html/2509.26307v1#S4.T1), our method significantly improves instruction adherence on both datasets.
On IHEval, AGD with LRP attribution (AGDLRP) consistently achieves the highest Prompt-Level Accuracy (PLA), boosting it by 13.1 points for Llama 3.1 over greedy decoding.
While this strong guidance can lower the Quality Score (QS), the entropy-gated version (AGDLRPe) effectively mitigates this trade-off, preserving higher quality while retaining most of the adherence gains.
Overall, entropy-gated variants consistently improve QS compared to their basic counterparts, with a notable trade-off in instruction adherence observed only for Llama 3.1.
I$\times$G attribution (AGDI$\times$G) preserves or even enhances quality over greedy decoding, but does not consistently improve adherence.
(^1^11Note that QS is measured only on samples where instructions are fully met; methods with lower PLA are thus evaluated on a potentially easier subset of examples, which may inflate their QS.)

On the more complex multi-turn SysBench benchmark, AGD’s advantages persist, particularly in maintaining long-term adherence. For example, with Llama 3.1, AGDLRPe improves the Session Stability Rate (SSR) by 7.9 points, showing a substantial increase in the model’s ability to remember and follow initial instructions over multiple turns. While the CAD baseline is sometimes competitive on the ISR metric, AGD variants consistently show superior performance across others. Overall, LRP proves to be a more effective attribution method than the simpler I$\times$G, providing a more robust mechanism for guiding generation toward instruction adherence.

### 4.3 Analysis & Case studies

Figure: Figure 2: Analysis of attribution signal for word inclusion and exclusion tasks. (a) AGD improves performance on both task types. (b, c) Layer-wise attribution in the residual stream for Llama 3.1 (8B) at decision points where AGD’s choice diverges from the greedy path to satisfy a constraint.
Refer to caption: x2.png

Figure: Figure 3: Heatmap visualizations of attribution scores on input token embeddings. For each task, we compare a token that adheres to the instruction (adherent) with one that does not (non-adherent). Spaces are omitted for clarity. Full examples are in Appendix LABEL:appendix:heatmaps.
Refer to caption: x3.png

To illustrate how AGD operates, we visualize the attribution scores of candidate tokens for different instruction types in Figure [3](https://arxiv.org/html/2509.26307v1#S4.F3). We observe that across various tasks – including word inclusion and exclusion (a, b), length manipulation (c, d), and format adherence (e) – tokens that satisfy the given instruction consistently exhibit higher attribution scores within the relevant parts of the prompt.

#### The Role of the Attribution Sign

Attribution methods often produce both positive and negative scores, which provide distinct and valuable guidance signals.
This is particularly evident when comparing two distinct instruction types from IHEval: keyword existence (e.g., Your response must include the keywords ’forests’ and ’riddle’) and forbidden words (e.g., Do not mention the words ’Taylor’, ’Swift’, or ’Together’). As shown in Figure [2](https://arxiv.org/html/2509.26307v1#S4.F2)a, AGD successfully improves adherence for both positive (inclusion) and negative (exclusion) constraints (except on Gemma 3 (4B), where forbidden words baseline performance is already near-saturated).
On average, a token that satisfies an inclusion rule exhibits a stronger positive attribution signal on instruction inputs throughout the residual stream of the model’s layers (Figure [2](https://arxiv.org/html/2509.26307v1#S4.F2)b), a process exemplified in Figure [3](https://arxiv.org/html/2509.26307v1#S4.F3)a where the candidate token “intern” receives high positive attribution from the same token in the instruction.

In contrast, for negative constraints, candidates must be suppressed. When a forbidden word appears as a candidate, it exhibits a negative attribution signal on the instruction inputs, a penalty observed on average across all layers (Figure [2](https://arxiv.org/html/2509.26307v1#S4.F2)c).
For instance, as shown in Figure [3](https://arxiv.org/html/2509.26307v1#S4.F3)b, the token “night” exhibits negative attribution with respect to itself in the instruction prompt.
This negative score penalizes its selection, allowing an alternative, correct token to be chosen instead, even if its own positive attribution score is small.
This demonstrates that negative attribution scores offer a strong signal for avoidance, making them crucial for steering models away from undesirable outputs. Consequently, only attribution methods that produce negative signals are applicable in this context.

#### Qualitative Impact of Entropy-Gating

As established in our main results, constant intervention during decoding can degrade output quality. Our entropy-gated AGD mitigates this by applying guidance only at decision points where the model is uncertain.
This creates a trade-off, quantified in Figure [4](https://arxiv.org/html/2509.26307v1#S4.F4)a: more frequent intervention boosts instruction adherence (PLA) but can degrade output quality (QS).
Our chosen 80th percentile threshold strikes a balance, capturing most of the adherence gains while preserving quality and reducing computational overhead.

Table [2](https://arxiv.org/html/2509.26307v1#S4.T2) provides a qualitative illustration of this trade-off. In the example, the greedy-decoded riddle is well-formed but fails the instruction by omitting the keyword “distressed”. The standard AGD output, which intervenes at each decoding step, includes the keyword but produces a single, ungrammatical run-on sentence. In contrast, the entropy-gated version successfully adheres to all constraints while maintaining grammatical correctness and a coherent structure.
This highlights that entropy-based intervention is essential for robust instruction adherence without harming quality.

Figure: Figure 4: Ablation analysis for Llama 3.1 (8B)
Refer to caption: x4.png

**Table 2: Qualitative comparison of model outputs for the riddle example. Tokens where AGD was activated (i.e., entropy exceeded $\tau$) are highlighted in blue. For standard AGD (no entropy), all generated tokens are highlighted. More examples in Appendix LABEL:appendix:entropy.**
| Example | Method | Output | Quality Evaluation |
| --- | --- | --- | --- |
| Instruction: Your response must include the keywords ’afternoon’ and ’distressed’. Do not use any commas in your response.<br>Task: Write a riddle about embroidery that has the answer ’needle’. | Greedy | I’m used to stitch and to sew\nIn the afternoon I’m often in view\nA tool for the crafty and skilled\nHelping to create with a delicate will\nI’m small but my work is grand\nA tiny hero in a crafty hand\n\nWhat am I?<br>Constraint failed: ’distressed’ keyword not used. | […] 4. Question: Is the riddle grammatically correct and well-structured? Reason: The riddle is grammatically correct and follows a coherent structure, making it easy to read and understand. Answer: Yes […] |
| AGDLRP<br>(w/o entropy) | In the afternoon I am carefully chosen to weave a delicate design on fabric with threads so fine I pierce through layers one stitch at a time leaving a path of yarn that is both intricate and a work of art often causing the crafter to become distressed when they lose their thread | […] Reason: The response is a single, long sentence that lacks punctuation, making it difficult to read and understand. Answer: No […] |  |
| AGDLRPe<br>(w/ entropy) | I pierce the fabric with gentle care\nAfternoon sunbeams highlight my delicate thread there\nA tool for artisans both skilled and distressed\nI guide the yarn through intricate designs I’ve guessed. | […] Reason: The riddle is grammatically correct and uses a consistent poetic structure, making it well-structured. Answer: Yes […] |  |

#### Ablations

We ablate the minimum probability threshold $\pi_{\min}$ used to form $C_{t}$ (Section [4.1](https://arxiv.org/html/2509.26307v1#S4.SS1.SSS0.Px3)) to analyze its impact on adherence, quality, and efficiency (Figure [4](https://arxiv.org/html/2509.26307v1#S4.F4)b,c). A lower $\pi_{\min}$ expands the candidate set, which can improve adherence but becomes detrimental at extremely low values. We hypothesize this is because the set becomes polluted with noisy, low-probability tokens that may be selected due to spurious high attribution scores, degrading quality and increasing computational cost. Conversely, a high $\pi_{\min}$ improves efficiency but causes adherence to drop sharply as correct tokens are prematurely filtered out. Our experiments show that $\pi_{\min}=0.05$ provides a robust balance, enabling high performance without being computationally prohibitive or susceptible to noise.

## 5 Factuality & in context retrieval

To demonstrate the versatility of AGD beyond instruction following, we evaluate it on knowledge-intensive Question Answering (QA) in two distinct settings. In the closed-book setting, the model must answer questions using only its internal, parametric knowledge. Here, the goal is to mitigate hallucinations by steering generation to rely on the components responsible for storing factual information. In the open-book setting, the model is provided with a context document containing the answer. The goal is to improve its ability to accurately ground its response in the provided evidence.

### 5.1 Experimental setup

For the closed-book setting, we guide generation by maximizing attribution towards a pre-identified set of parametric knowledge heads (AGDLRPh). For the open-book setting, we explore two guidance strategies: maximizing attribution towards the input embeddings of the provided context (AGDLRPc), or towards a pre-identified set of in-context heads responsible for contextual processing (AGDLRPh). We evaluate on three standard QA benchmarks: MRQA version (Fisch et al., 2019) of TriviaQA (TQA) (Joshi et al., 2017) and Natural Questions (NQ) (Kwiatkowski et al., 2019), as well as HotPotQA (HPQA) (Zhilin et al., 2018), all of each can be processed with or without the context.
We report recall (Adlakha et al., 2024) as our performance metric, since instruction-tuned models tend to produce verbose outputs.
Details of the data preprocessing can be found in the Appendix [A](https://arxiv.org/html/2509.26307v1#A1.SS0.SSS0.Px1).

Our baselines include greedy decoding and nucleus sampling, supplemented by strong, task-specific methods. For the closed-book setting, we add DoLA (Chuang et al., 2024), which is designed to reduce hallucinations by contrasting logits from different model layers. For the open-book setting, we use again Context-aware Decoding (CAD) (Shi et al., 2024), as its contrastive mechanism is specifically designed to ground generation in a provided context.

### 5.2 Results

**Table 3: Recall score (%) of Llama 3.1 (8B) in both closed-book (top) and open-book (bottom) settings. Higher scores are better. Full results are in Appendix LABEL:appendix:full_factuality.**
| Method | TQA | NQ | HPQA |
| --- | --- | --- | --- |
| \rowcolorlightred<br>Greedy | 81.4 | 63.6 | 34.6 |
| \rowcolorlightred<br>Nucleus | 79.0 | 59.9 | 31.9 |
| \rowcolorlightred<br>DoLA | 81.2 | 63.8 | 34.3 |
| \rowcolordarkred AGDLRPh | 82.4 | 63.0 | 39.6 |
| \rowcolorlightblue<br>Greedy | 89.4 | 83.5 | 52.4 |
| \rowcolorlightblue<br>Nucleus | 89.7 | 83.3 | 52.9 |
| \rowcolorlightblue<br>CAD | 87.9 | 84.6 | 52.8 |
| \rowcolordarkblue AGDLRPh | 91.0 | 87.0 | 57.4 |
| \rowcolordarkblue AGDLRPc | 91.4 | 87.9 | 59.8 |

The results are presented in Table [3](https://arxiv.org/html/2509.26307v1#S5.T3).
In the closed-book setting, guiding generation towards parametric knowledge heads (AGDLRPh) improves factual recall. For Llama 3.1 (8B), it outperforms standard decoding and the DoLA baseline on TQA and HPQA.
This trend holds for the Qwen 2.5 (7B) model, though improvements are less pronounced for the smaller Gemma 3 (4B).
In the open-book setting, AGD yields more consistent and significant gains across all models. Guiding generation towards either the provided context embeddings (AGDLRPc) or in-context heads (AGDLRPh) consistently outperforms baselines across all datasets and models, with the context-embedding strategy generally proving slightly more effective.
Notably, even for Gemma 3 (4B), where closed-book improvements were limited, AGD provides a clear boost in performance, demonstrating its effectiveness at grounding generation in provided evidence.
Overall, these results show that AGD is a potent method for enhancing the factual accuracy and contextual grounding of LLMs, with particularly strong performance in open-book retrieval scenarios.

## 6 Conclusion

In this work, we introduced Attribution-Guided Decoding (AGD), a fine-tuning-free decoding strategy that enhances LLM reliability by selecting tokens that maximally attribute to a specified Region of Interest (ROI), such as a user instruction or a knowledge-storing component.
Our experiments demonstrate that this approach significantly improves both instruction adherence and factual accuracy in closed-book and open-book settings,
while an entropy-gated variant preserves output quality and reduces computational cost by applying guidance selectively.

AGD’s primary limitation is inherent to its design as a selection mechanism: it cannot generate a desired token if it is not proposed by the model. Other challenges include the computational cost of multiple backward passes and the need to define a relevant ROI for each task. Future work could focus on developing more efficient attribution proxies to mitigate these costs. Moreover, the ROI concept could be extended from input spans or attention heads to more monosemantic structures, such as specific, functionally-identified circuits within the model, enabling more granular control.
The code is available at [https://github.com/piotr-komorowski/attribution-guided-decoding](https://github.com/piotr-komorowski/attribution-guided-decoding).

## Reproducibility statement

The source code for Attribution-Guided Decoding (AGD) and all experimental scripts will be made publicly available upon publication. We provide detailed descriptions of our experimental setup throughout the paper, including the specific models used (Section [4](https://arxiv.org/html/2509.26307v1#S4)), datasets and the fixed hyperparameters for both AGD and all baselines (Sections [4.1](https://arxiv.org/html/2509.26307v1#S4.SS1) and Appendix [A](https://arxiv.org/html/2509.26307v1#A1.SS0.SSS0.Px1)). All experiments involving randomness, such as nucleus sampling, were conducted with a fixed random seed to ensure consistent outcomes. Further implementation details, including the exact prompt templates used for data preprocessing steps (Appendix [A](https://arxiv.org/html/2509.26307v1#A1.SS0.SSS0.Px1)), the quality scoring protocol (Appendix [B](https://arxiv.org/html/2509.26307v1#A2)), and the methodology for extracting specialized attention heads (Appendix LABEL:app:extraction), are documented in the Appendix.

## Acknowledgments

We would like to express our gratitude to Melina Zeeb for her valuable assistance in preparing the main figure of this paper, and to Patrick Kahardipraja for his constructive feedback on this work.

## References

- Achiam et al. (2023)
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al.
Gpt-4 technical report.
*arXiv preprint arXiv:2303.08774*, 2023.
- Achtibat et al. (2023)
Reduan Achtibat, Maximilian Dreyer, Ilona Eisenbraun, Sebastian Bosse, Thomas Wiegand, Wojciech Samek, and Sebastian Lapuschkin.
From attribution maps to human-understandable explanations through concept relevance propagation.
*Nature Machine Intelligence*, 5(9):1006–1019, 2023.
- Achtibat et al. (2024)
Reduan Achtibat, Sayed Mohammad Vakilzadeh Hatefi, Maximilian Dreyer, Aakriti Jain, Thomas Wiegand, Sebastian Lapuschkin, and Wojciech Samek.
AttnLRP: Attention-aware layer-wise relevance propagation for transformers.
In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), *Proceedings of the 41st International Conference on Machine Learning*, volume 235 of *Proceedings of Machine Learning Research*, pp. 135–168. PMLR, 21–27 Jul 2024.
- Adlakha et al. (2024)
Vaibhav Adlakha, Parishad BehnamGhader, Xing Han Lu, Nicholas Meade, and Siva Reddy.
Evaluating correctness and faithfulness of instruction-following models for question answering.
*Transactions of the Association for Computational Linguistics*, 12:681–699, 2024.
doi: 10.1162/tacl˙a˙00667.
URL [https://aclanthology.org/2024.tacl-1.38/](https://aclanthology.org/2024.tacl-1.38/).
- Ali et al. (2022)
Ameen Ali, Thomas Schnake, Oliver Eberle, Grégoire Montavon, Klaus-Robert Müller, and Lior Wolf.
Xai for transformers: Better explanations through conservative propagation.
In *International conference on machine learning*, pp. 435–451. PMLR, 2022.
- Anders et al. (2022)
Christopher J. Anders, Leander Weber, David Neumann, Wojciech Samek, Klaus-Robert Müller, and Sebastian Lapuschkin.
Finding and removing clever hans: Using explanation methods to debug and improve deep models.
*Information Fusion*, 77:261–295, 2022.
ISSN 1566-2535.
doi: https://doi.org/10.1016/j.inffus.2021.07.015.
URL [https://www.sciencedirect.com/science/article/pii/S1566253521001573](https://www.sciencedirect.com/science/article/pii/S1566253521001573).
- Arditi et al. (2024)
Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda.
Refusal in language models is mediated by a single direction.
In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (eds.), *Advances in Neural Information Processing Systems*, volume 37, pp. 136037–136083. Curran Associates, Inc., 2024.
URL [https://proceedings.neurips.cc/paper_files/paper/2024/file/f545448535dfde4f9786555403ab7c49-Paper-Conference.pdf](https://proceedings.neurips.cc/paper_files/paper/2024/file/f545448535dfde4f9786555403ab7c49-Paper-Conference.pdf).
- Arras et al. (2025)
Leila Arras, Bruno Puri, Patrick Kahardipraja, Sebastian Lapuschkin, and Wojciech Samek.
A close look at decomposition-based xai-methods for transformer language models.
*arXiv preprint arXiv:2502.15886*, 2025.
- Bach et al. (2015)
Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, and Wojciech Samek.
On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation.
*PloS one*, 10(7):e0130140, 2015.
- Bastings & Filippova (2020)
Jasmijn Bastings and Katja Filippova.
The elephant in the interpretability room: Why use attention as explanation when we have saliency methods?
In *Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP*, pp. 149–155, 2020.
- Brown et al. (2020)
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al.
Language models are few-shot learners.
*Advances in neural information processing systems*, 33:1877–1901, 2020.
- Burns et al. (2023)
Collin Burns, Haotian Ye, Dan Klein, and Jacob Steinhardt.
Discovering latent knowledge in language models without supervision.
In *The Eleventh International Conference on Learning Representations*, 2023.
URL [https://openreview.net/forum?id=ETKGuby0hcs](https://openreview.net/forum?id=ETKGuby0hcs).
- Chuang et al. (2024)
Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James R. Glass, and Pengcheng He.
Dola: Decoding by contrasting layers improves factuality in large language models.
In *The Twelfth International Conference on Learning Representations*, 2024.
URL [https://openreview.net/forum?id=Th6NyL07na](https://openreview.net/forum?id=Th6NyL07na).
- Chung et al. (2024)
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al.
Scaling instruction-finetuned language models.
*Journal of Machine Learning Research*, 25(70):1–53, 2024.
- Fan et al. (2018)
Angela Fan, Mike Lewis, and Yann Dauphin.
Hierarchical neural story generation.
In Iryna Gurevych and Yusuke Miyao (eds.), *Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)*, pp. 889–898, Melbourne, Australia, July 2018. Association for Computational Linguistics.
doi: 10.18653/v1/P18-1082.
URL [https://aclanthology.org/P18-1082/](https://aclanthology.org/P18-1082/).
- Fisch et al. (2019)
Adam Fisch, Alon Talmor, Robin Jia, Minjoon Seo, Eunsol Choi, and Danqi Chen.
MRQA 2019 shared task: Evaluating generalization in reading comprehension.
In Adam Fisch, Alon Talmor, Robin Jia, Minjoon Seo, Eunsol Choi, and Danqi Chen (eds.), *Proceedings of the 2nd Workshop on Machine Reading for Question Answering*, pp. 1–13, Hong Kong, China, November 2019. Association for Computational Linguistics.
doi: 10.18653/v1/D19-5801.
URL [https://aclanthology.org/D19-5801/](https://aclanthology.org/D19-5801/).
- Grattafiori et al. (2024)
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al.
The llama 3 herd of models.
*arXiv preprint arXiv:2407.21783*, 2024.
- Gupta et al. (2022)
Prakhar Gupta, Cathy Jiao, Yi-Ting Yeh, Shikib Mehri, Maxine Eskenazi, and Jeffrey Bigham.
InstructDial: Improving zero and few-shot generalization in dialogue through instruction tuning.
In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), *Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing*, pp. 505–525, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics.
doi: 10.18653/v1/2022.emnlp-main.33.
URL [https://aclanthology.org/2022.emnlp-main.33/](https://aclanthology.org/2022.emnlp-main.33/).
- Heo et al. (2025)
Juyeon Heo, Christina Heinze-Deml, Oussama Elachqar, Kwan Ho Ryan Chan, Shirley You Ren, Andrew Miller, Udhyakumar Nallasamy, and Jaya Narain.
Do LLMs “know” internally when they follow instructions?
In *The Thirteenth International Conference on Learning Representations*, 2025.
URL [https://openreview.net/forum?id=qIN5VDdEOr](https://openreview.net/forum?id=qIN5VDdEOr).
- Holtzman et al. (2020)
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi.
The curious case of neural text degeneration.
In *International Conference on Learning Representations*, 2020.
URL [https://openreview.net/forum?id=rygGQyrFvH](https://openreview.net/forum?id=rygGQyrFvH).
- Jiang et al. (2024)
Yuxin Jiang, Yufei Wang, Xingshan Zeng, Wanjun Zhong, Liangyou Li, Fei Mi, Lifeng Shang, Xin Jiang, Qun Liu, and Wei Wang.
Followbench: A multi-level fine-grained constraints following benchmark for large language models.
In *Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)*, pp. 4667–4688, 2024.
- Jin et al. (2024)
Zhuoran Jin, Pengfei Cao, Hongbang Yuan, Yubo Chen, Jiexin Xu, Huaijun Li, Xiaojian Jiang, Kang Liu, and Jun Zhao.
Cutting off the head ends the conflict: A mechanism for interpreting and mitigating knowledge conflicts in language models.
In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), *Findings of the Association for Computational Linguistics: ACL 2024*, pp. 1193–1215, Bangkok, Thailand, August 2024. Association for Computational Linguistics.
doi: 10.18653/v1/2024.findings-acl.70.
URL [https://aclanthology.org/2024.findings-acl.70/](https://aclanthology.org/2024.findings-acl.70/).
- Joshi et al. (2017)
Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer.
TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension.
In Regina Barzilay and Min-Yen Kan (eds.), *Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)*, pp. 1601–1611, Vancouver, Canada, July 2017. Association for Computational Linguistics.
doi: 10.18653/v1/P17-1147.
URL [https://aclanthology.org/P17-1147/](https://aclanthology.org/P17-1147/).
- Kahardipraja et al. (2025)
Patrick Kahardipraja, Reduan Achtibat, Thomas Wiegand, Wojciech Samek, and Sebastian Lapuschkin.
The atlas of in-context learning: How attention heads shape in-context retrieval augmentation.
In *Advances in Neural Information Processing Systems*, 2025.
- Kwiatkowski et al. (2019)
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov.
Natural questions: A benchmark for question answering research.
*Transactions of the Association for Computational Linguistics*, 7:452–466, 2019.
doi: 10.1162/tacl˙a˙00276.
URL [https://aclanthology.org/Q19-1026/](https://aclanthology.org/Q19-1026/).
- Lapuschkin et al. (2019)
Sebastian Lapuschkin, Stephan Wäldchen, Alexander Binder, Grégoire Montavon, Wojciech Samek, and Klaus-Robert Müller.
Unmasking clever hans predictors and assessing what machines really learn.
*Nature communications*, 10(1):1096, 2019.
- Li et al. (2023a)
Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg.
Inference-time intervention: Eliciting truthful answers from a language model.
In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), *Advances in Neural Information Processing Systems*, volume 36, pp. 41451–41530. Curran Associates, Inc., 2023a.
URL [https://proceedings.neurips.cc/paper_files/paper/2023/file/81b8390039b7302c909cb769f8b6cd93-Paper-Conference.pdf](https://proceedings.neurips.cc/paper_files/paper/2023/file/81b8390039b7302c909cb769f8b6cd93-Paper-Conference.pdf).
- Li et al. (2024)
Kenneth Li, Tianle Liu, Naomi Bashkansky, David Bau, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg.
Measuring and controlling instruction (in)stability in language model dialogs.
In *First Conference on Language Modeling*, 2024.
URL [https://openreview.net/forum?id=60a1SAtH4e](https://openreview.net/forum?id=60a1SAtH4e).
- Li et al. (2023b)
Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori B Hashimoto, Luke Zettlemoyer, and Mike Lewis.
Contrastive decoding: Open-ended text generation as optimization.
In *Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)*, pp. 12286–12312, 2023b.
- Liu et al. (2024)
Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang.
Lost in the middle: How language models use long contexts.
*Transactions of the Association for Computational Linguistics*, 12:157–173, 2024.
- Longpre et al. (2021)
Shayne Longpre, Kartik Perisetla, Anthony Chen, Nikhil Ramesh, Chris DuBois, and Sameer Singh.
Entity-based knowledge conflicts in question answering.
In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), *Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing*, pp. 7052–7063, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics.
doi: 10.18653/v1/2021.emnlp-main.565.
URL [https://aclanthology.org/2021.emnlp-main.565/](https://aclanthology.org/2021.emnlp-main.565/).
- Longpre et al. (2023)
Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V Le, Barret Zoph, Jason Wei, et al.
The flan collection: Designing data and methods for effective instruction tuning.
In *International Conference on Machine Learning*, pp. 22631–22648. PMLR, 2023.
- Lundberg & Lee (2017)
Scott M Lundberg and Su-In Lee.
A unified approach to interpreting model predictions.
*Advances in neural information processing systems*, 30, 2017.
- Ouyang et al. (2022)
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al.
Training language models to follow instructions with human feedback.
*Advances in neural information processing systems*, 35:27730–27744, 2022.
- Pahde et al. (2023)
Frederik Pahde, Maximilian Dreyer, Wojciech Samek, and Sebastian Lapuschkin.
Reveal to revise: An explainable ai life cycle for iterative bias correction of deep models.
In *International Conference on Medical Image Computing and Computer-Assisted Intervention*, pp. 596–606. Springer, 2023.
- Paszke et al. (2019)
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al.
Pytorch: An imperative style, high-performance deep learning library.
*Advances in neural information processing systems*, 32, 2019.
- Qin et al. (2025)
Yanzhao Qin, Tao Zhang, Tao Zhang, Yanjun Shen, Wenjing Luo, sunhaoze, Yan Zhang, Yujing Qiao, weipeng chen, Zenan Zhou, Wentao Zhang, and Bin CUI.
Sysbench: Can LLMs follow system message?
In *The Thirteenth International Conference on Learning Representations*, 2025.
URL [https://openreview.net/forum?id=KZWaxtzIRx](https://openreview.net/forum?id=KZWaxtzIRx).
- Qin et al. (2024)
Yiwei Qin, Kaiqiang Song, Yebowen Hu, Wenlin Yao, Sangwoo Cho, Xiaoyang Wang, Xuansheng Wu, Fei Liu, Pengfei Liu, and Dong Yu.
InFoBench: Evaluating instruction following ability in large language models.
In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), *Findings of the Association for Computational Linguistics: ACL 2024*, pp. 13025–13048, Bangkok, Thailand, August 2024. Association for Computational Linguistics.
doi: 10.18653/v1/2024.findings-acl.772.
URL [https://aclanthology.org/2024.findings-acl.772/](https://aclanthology.org/2024.findings-acl.772/).
- Rimsky et al. (2024)
Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Turner.
Steering llama 2 via contrastive activation addition.
In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), *Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)*, pp. 15504–15522, Bangkok, Thailand, August 2024. Association for Computational Linguistics.
doi: 10.18653/v1/2024.acl-long.828.
URL [https://aclanthology.org/2024.acl-long.828/](https://aclanthology.org/2024.acl-long.828/).
- Shi et al. (2024)
Weijia Shi, Xiaochuang Han, Mike Lewis, Yulia Tsvetkov, Luke Zettlemoyer, and Wen-tau Yih.
Trusting your evidence: Hallucinate less with context-aware decoding.
In *Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers)*, pp. 783–791, 2024.
- Simonyan et al. (2014)
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman.
Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps.
In *International Conference on Learning Representations - Workshop track (ICLR)*, 2014.
URL [https://arxiv.org/pdf/1312.6034.pdf](https://arxiv.org/pdf/1312.6034.pdf).
- Smilkov et al. (2017)
Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg.
Smoothgrad: removing noise by adding noise.
*arXiv preprint arXiv:1706.03825*, 2017.
- Stolfo et al. (2025)
Alessandro Stolfo, Vidhisha Balachandran, Safoora Yousefi, Eric Horvitz, and Besmira Nushi.
Improving instruction-following in language models through activation steering.
In *The Thirteenth International Conference on Learning Representations*, 2025.
URL [https://openreview.net/forum?id=wozhdnRCtw](https://openreview.net/forum?id=wozhdnRCtw).
- Subramani et al. (2022)
Nishant Subramani, Nivedita Suresh, and Matthew Peters.
Extracting latent steering vectors from pretrained language models.
In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), *Findings of the Association for Computational Linguistics: ACL 2022*, pp. 566–581, Dublin, Ireland, May 2022. Association for Computational Linguistics.
doi: 10.18653/v1/2022.findings-acl.48.
URL [https://aclanthology.org/2022.findings-acl.48/](https://aclanthology.org/2022.findings-acl.48/).
- Sun et al. (2023)
Jiao Sun, Yufei Tian, Wangchunshu Zhou, Nan Xu, Qian Hu, Rahul Gupta, John Frederick Wieting, Nanyun Peng, and Xuezhe Ma.
Evaluating large language models on controlled generation tasks.
In *The 2023 Conference on Empirical Methods in Natural Language Processing*, 2023.
URL [https://openreview.net/forum?id=nuPp6jdCgg](https://openreview.net/forum?id=nuPp6jdCgg).
- Sundararajan et al. (2017)
Mukund Sundararajan, Ankur Taly, and Qiqi Yan.
Axiomatic attribution for deep networks.
In Doina Precup and Yee Whye Teh (eds.), *Proceedings of the 34th International Conference on Machine Learning*, volume 70 of *Proceedings of Machine Learning Research*, pp. 3319–3328. PMLR, 06–11 Aug 2017.
URL [https://proceedings.mlr.press/v70/sundararajan17a.html](https://proceedings.mlr.press/v70/sundararajan17a.html).
- Team et al. (2025)
Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, et al.
Gemma 3 technical report.
*arXiv preprint arXiv:2503.19786*, 2025.
- Tigges et al. (2024)
Curt Tigges, Oskar J. Hollinsworth, Atticus Geiger, and Neel Nanda.
Language models linearly represent sentiment.
In Yonatan Belinkov, Najoung Kim, Jaap Jumelet, Hosein Mohebbi, Aaron Mueller, and Hanjie Chen (eds.), *Proceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP*, pp. 58–87, Miami, Florida, US, November 2024. Association for Computational Linguistics.
doi: 10.18653/v1/2024.blackboxnlp-1.5.
URL [https://aclanthology.org/2024.blackboxnlp-1.5/](https://aclanthology.org/2024.blackboxnlp-1.5/).
- Voita et al. (2019)
Elena Voita, David Talbot, Fedor Moiseev, Rico Sennrich, and Ivan Titov.
Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned.
In Anna Korhonen, David Traum, and Lluís Màrquez (eds.), *Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics*, pp. 5797–5808, Florence, Italy, July 2019. Association for Computational Linguistics.
doi: 10.18653/v1/P19-1580.
URL [https://aclanthology.org/P19-1580/](https://aclanthology.org/P19-1580/).
- Wang et al. (2025)
Shenzhi Wang, Le Yu, Chang Gao, Chujie Zheng, Shixuan Liu, Rui Lu, Kai Dang, Xionghui Chen, Jianxin Yang, Zhenru Zhang, et al.
Beyond the 80/20 rule: High-entropy minority tokens drive effective reinforcement learning for llm reasoning.
*arXiv preprint arXiv:2506.01939*, 2025.
- Wei et al. (2022)
Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V Le.
Finetuned language models are zero-shot learners.
In *International Conference on Learning Representations*, 2022.
URL [https://openreview.net/forum?id=gEZrGCozdqR](https://openreview.net/forum?id=gEZrGCozdqR).
- Wei et al. (2024)
Jason Wei, Nguyen Karina, Hyung Won Chung, Yunxin Joy Jiao, Spencer Papay, Amelia Glaese, John Schulman, and William Fedus.
Measuring short-form factuality in large language models.
*arXiv preprint arXiv:2411.04368*, 2024.
- Yang et al. (2024)
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al.
Qwen2. 5 technical report.
*arXiv preprint arXiv:2412.15115*, 2024.
- Yin & Neubig (2022)
Kayo Yin and Graham Neubig.
Interpreting language models with contrastive explanations.
In *Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing*, pp. 184–198, 2022.
- Yu et al. (2024)
Xiaodong Yu, Hao Cheng, Xiaodong Liu, Dan Roth, and Jianfeng Gao.
ReEval: Automatic hallucination evaluation for retrieval-augmented large language models via transferable adversarial attacks.
In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), *Findings of the Association for Computational Linguistics: NAACL 2024*, pp. 1333–1351, Mexico City, Mexico, June 2024. Association for Computational Linguistics.
doi: 10.18653/v1/2024.findings-naacl.85.
URL [https://aclanthology.org/2024.findings-naacl.85/](https://aclanthology.org/2024.findings-naacl.85/).
- Zeng et al. (2024)
Zhiyuan Zeng, Jiatong Yu, Tianyu Gao, Yu Meng, Tanya Goyal, and Danqi Chen.
Evaluating large language models at evaluating instruction following.
In *International Conference on Learning Representations (ICLR)*, 2024.
- Zhang et al. (2024)
Qingru Zhang, Chandan Singh, Liyuan Liu, Xiaodong Liu, Bin Yu, Jianfeng Gao, and Tuo Zhao.
Tell your model where to attend: Post-hoc attention steering for LLMs.
In *The Twelfth International Conference on Learning Representations*, 2024.
URL [https://openreview.net/forum?id=xZDWO0oejD](https://openreview.net/forum?id=xZDWO0oejD).
- Zhilin et al. (2018)
Yang Zhilin, Qi Peng, Zhang Saizheng, Bengio Yoshua, Cohen William, Salakhutdinov Ruslan, and Christopher D. Manning.
Hotpotqa: A dataset for diverse, explainable multi-hop question answering.
*Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing*, 2018.
doi: 10.18653/v1/d18-1259.
URL [https://cir.nii.ac.jp/crid/1363388846142371200](https://cir.nii.ac.jp/crid/1363388846142371200).
- Zhou et al. (2023a)
Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou.
Instruction-following evaluation for large language models.
*arXiv preprint arXiv:2311.07911*, 2023a.
- Zhou et al. (2023b)
Wangchunshu Zhou, Yuchen Eleanor Jiang, Ethan Wilcox, Ryan Cotterell, and Mrinmaya Sachan.
Controlled text generation with natural language instructions.
In *International Conference on Machine Learning*, pp. 42602–42613. PMLR, 2023b.

## Appendix A General Implementation

Method and experiments were implemented using PyTorch (Paszke et al., 2019).
LLM-based evaluations for Quality Score (QS) for IHEval and SysBench metrics were performed using gpt-4o-2024-08-06.

#### Datasets

For the IHEval dataset, we used the rule-following/single-turn/aligned split from its official repository ([https://github.com/ytyz1307zzh/IHEval](https://github.com/ytyz1307zzh/IHEval)). The SysBench dataset was sourced from its original repository ([https://github.com/PKU-Baichuan-MLSystemLab/SysBench](https://github.com/PKU-Baichuan-MLSystemLab/SysBench)). For TriviaQA (TQA) and Natural Questions (NQ), we used the development splits from the MRQA 2019 Shared Task repository ([https://github.com/mrqa/MRQA-Shared-Task-2019](https://github.com/mrqa/MRQA-Shared-Task-2019)). Lastly, for HotPotQA (HPQA), we used the validation split from Hugging Face ([https://huggingface.co/datasets/hotpotqa](https://huggingface.co/datasets/hotpotqa)).

Factuality datasets underwent a preprocessing procedure. Specifically for NQ, we applied the steps proposed by Yu et al. (2024). Additionally, we filtered out duplicate entries from each dataset, performing this process independently for the closed-book (CB) and open-book (OB) settings. This filtering accounts for instances in the OB setting where the same question may be paired with different evidence passages, but also when context was empty. This process resulted in final evaluation sets of 7,785 samples for TQA (CB and OB); 4,987 for NQ (CB) and 5,450 (OB); and 5,918 for HPQA (CB) and 5,913 (OB).

For all experiments, we used the system prompt: You are a helpful assistant. For closed-book settings, the user prompt consisted solely of the question. For open-book settings, we used the following prompt structure: {{ context }} \n\nBased on this text, answer this question:\nQ: {{ question }}\nA:.

#### Baselines

Proposed by Shi et al. (2024), CAD is a contrastive decoding method designed to improve the faithfulness of generation to a given context. It modifies the output logits at each step by amplifying the difference between the distribution conditioned on the full input and a distribution conditioned on a partial, ”context-free” input. The modified logit is computed as:

$$ $\text{logit}^{\prime}(y_{t})=(1+\alpha)\cdot\text{logit}(y_{t}\mid x_{\text{full}})-\alpha\cdot\text{logit}(y_{t}\mid x_{\text{context-free}})$ (6) $$

We use CAD as a baseline for both instruction following (where $x_{\text{full}}$ includes the instruction and $x_{\text{context-free}}$ omits it) and open-book QA (where $x_{\text{context-free}}$ omits the provided document). Following the original work, as one of the plausible choices, we set the control hyperparameter $\alpha=1.0$.

Proposed by Chuang et al. (2024), DoLA is a method designed to reduce hallucinations in closed-book settings. It is based on the finding that factual knowledge in transformers is often localized in specific layers. The method works by modifying the output logits at each decoding step – it contrasts the logits from the final layer with logits projected from one of earlier layers, exploiting the hierarchical encoding of factual knowledge within LLMs.
While the original implementation suggests contrasting with *higher* layers for QA tasks, we empirically found that contrasting the final layer with *lower* layers consistently yielded better recall scores across all models and datasets in our setup; we therefore report this.

## Appendix B Quality Evaluation

To evaluate the generation quality for the IHEval dataset, we follow the procedure introduced by Stolfo et al. (2025).
First, using only the task portion of the prompt (i.e., without the instruction), we prompt an LLM evaluator to generate up to five simple yes/no questions that break down the core requirements of the task. The examples of these questions can be found in Table [2](https://arxiv.org/html/2509.26307v1#S4.T2) and LABEL:tab:entropy_examples_email. The prompt used for this step is shown in Table LABEL:tab:question_extraction.

Second, the model’s response to the full prompt (task and instruction) along with task itself is evaluated against these generated questions. The evaluator is prompted to answer *Yes*, *No* or *Not Applicable* for each question, providing a brief justification. The prompt for this evaluation step is shown in Table LABEL:tab:qs_evaluation.
he final Quality Score (QS) is defined as fraction of *Yes* responses out of *Yes* and *No*, calculated only for responses that successfully satisfied all instructions.

## Peer Reviews

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: see weakness
soundness: 3
strengths: 1. AGD repurposes passive attribution methods for active generation. Unlike "interventionist" methods that alter activations, this "selectionist" approach chooses among the model's high-probability outputs, preserving text quality while making generation interpretable through token-rationale relationships.

2. The ROI framework applies broadly to diverse problems. AGD improves instruction adherence (using instruction text as ROI), factual recall (targeting "parametric knowledge heads"), and in-context grounding (using context embeddings as ROI).

3. The authors tested across multiple models (Llama 3.1, Qwen 2.5, Gemma 3) and tasks with strong baselines like CAD and DoLA. The analysis includes hyperparameter ablations and case studies visualizing attribution scores to explain why the method works.
summary: This paper presents Attribution-Guided Decoding (AGD), a fine-tuning-free decoding strategy to enhance the reliability of LLMs. AGD is based on the selection of tokens from a set of high-probability candidates. Selection is based on the maximization of the attribution score of a candidate token to a user-defined "Region of Interest" (ROI). This ROI can be flexibly defined over parts of the input prompt or over internal model components. The authors show the effectiveness of AGD in improving instruction following, increasing factual accuracy in closed-book question answering by avoiding hallucinations, and enhancing grounding in open-book (in-context) retrieval tasks. Regarding the computational overhead of calculating attributions at each step, an adaptive variant is also proposed, entropy-gated AGD-e, which applies guidance only when the model output distribution shows high uncertainty, thus preserving the quality of the output while saving cost.
weaknesses: 1. AGD's "selectionist" design limits it to tokens already proposed by the base model with high probability. If the desired token isn't in the candidate set, AGD cannot generate it, making success dependent on the base model's capabilities. 

2.  The method seems to be computationally expensive, requiring backward passes for each candidate token to compute attribution scores. While entropy-gating helps, this overhead could limit its practice. In the meantime, we need to predefine the threshold for the entropy, which can not always be reliable.

3. While ROI flexibility is a strength, it requires users to define relevant ROIs for each task. This is straightforward for instruction following but non-trivial for tasks like factuality improvement, which requires identifying specialized components like "knowledge heads." 

4. Many baselines are missing such as DoLA.

---

confidence: 3
contribution: 2
initial_rating: 6
presentation: 2
questions: 1. **RAG vs AGD:** In open-book, could you compare against a standard RAG baseline (retrieve-then-decode) and a “ROI-prompt only” baseline that emphasises the ROI without attribution? This isolates attribution’s unique contribution.  
2. **Cost & memory:** Please report wall-clock, token cost, and peak memory with and without entropy gating, and versus CAD/RAG on the same hardware. Also provide the average number of backward passes per token under typical thresholds.  
3. **Stronger models:** Do gains hold for newer small models (e.g., Qwen3 or DeepSeek-R1-Qwen3-8B)? Any evidence of diminishing returns as base model adherence improves?  
4. **Head selection robustness:** For “knowledge” or “in-context” heads, how stable are results across identification methods and datasets? Could you provide ablation of head sets and performance sensitivity?
soundness: 2
strengths: - **Simple, model-agnostic control:** Decoding-time, fine-tuning-free; clear ROI abstraction spanning inputs or internals.  
- **Practical quality/cost knob:** Entropy gating reduces intervention and keeps fluency while retaining most adherence gains.  
- **Coverage:** Evaluated on instruction following (IHEval, SysBench) and factual QA (closed/open-book) across three small/mid models.
summary: - **Idea:** At each step, form a small candidate set (top-k with prob floor), compute attribution of each candidate to a user-defined Region of Interest (ROI) (e.g., instruction tokens, context embeddings, selected attention heads), and pick the token with the highest ROI attribution. No logits or activations are modified; it’s selection rather than intervention.  
- **Variants:** Use LRP/AttnLRP (primary) or Input×Gradient for attribution; propose entropy gating to apply guidance only when the model is uncertain to curb quality loss and cost.  
- **Results:** Improves instruction adherence on IHEval/SysBench and boosts factuality in closed-book (via “knowledge heads”) and open-book (via context embeddings or “in-context heads”); entropy gating trades off adherence vs quality and reduces backward passes.
weaknesses: - **Positioning vs RAG/contrastive baselines is incomplete:** Open-book uses an ROI over context/heads, but no retrieval step; comparisons are to greedy/nucleus/CAD, not to RAG-style pipelines or “ROI-prompt only” ablations. This blurs how much gain comes from attribution versus simply highlighting the ROI.  
- **Compute overhead and scalability:** AGD requires backward passes per candidate token when triggered; the paper reports entropy-gated reductions and average backward-pass counts, but lacks end-to-end wall-clock or token-cost comparisons against strong external baselines. Memory implications are not quantified.  
- **Generalisation to stronger LLMs is unclear:** Results are on Llama-3.1-8B, Qwen-2.5-7B, Gemma-3-4B; it’s unknown whether gains persist or shrink on frontier small models (e.g., Qwen3, DeepSeek-R1-8B).

---

confidence: 3
contribution: 3
initial_rating: 4
presentation: 3
questions: 1. Human interpretability:
Have you tested whether human evaluators actually find the attributions helpful for understanding the model’s reasoning? Qualitative examples could strengthen this claim.

2. hyperparameters setting
There are two sensitive hyperparameters: entropy-gated threshold and the minimum probability threshold.  It looks depends on models and tasks. Did you have any related experience?
soundness: 3
strengths: 1. integration of attribution into decoding: introducing attribution-guided decoding (AGD) that dynamically adjusts token-level generation probabilities based on feature attributions. 

2. Experiments span three representative long-form tasks (fact-grounded QA, long summarization, and reasoning). The authors evaluate across multiple model families (Llama-3, Qwen2.5, Gemma2) and both open- and closed-source settings.

3. Interpretability and visualization:
The paper presents clear visualizations (Figures 3–5) of token attributions and their decoding effects. These figures concretely show that AGD improves focus on factually relevant spans and reduces hallucinated continuations
summary: This work propose a method to reduce hallucinations and improve factual accuracy. In order to reduce computational cost, an entropy-based adaptive mechanism is applied. In the experiments, higher factuality is obtained on two benchmarks (IHEval and SysBench).
weaknesses: 1. Limited novelty in algorithmic mechanism: Although the integration of attribution and decoding is original, the underlying technique (re-weighting token probabilities) is relatively straightforward and derivative of prior constrained or guided decoding approaches (e.g., likelihood re-weighting, constrained beam search). Following are some related work on this direction

GeDi: Generative discriminator guided sequence generation. EMNLP 2021
Unlocking Anticipatory Text Generation: A Constrained Approach for Large Language Models Decoding. EMNLP 2024

2. Evaluation lacks human verification.
Improvements in factuality metrics are fully automatic. Human or expert evaluation of factual correctness and coherence would make the results more convincing, especially for interpretability claims.


Decision: Accept (Poster)