NDAD: Negative-Direction Aware Decoding for Large Language Models via Controllable Hallucination Signal Injection
Authors: Panjia Qiu
Mingyuan Fan
Cen Chen
Daixin Wang
Venue: ICLR 2026 Poster (2026)
Keywords: Large Language Models;Contrastive Decoding;Robustness
Primary area: alignment, fairness, safety, privacy, and societal considerations

Abstract: Large language models (LLMs) have recently achieved impressive progress in knowledge-intensive and reasoning tasks. However, their tendency to produce fabricated or factually inconsistent content remains a fundamental challenge to their practical deployment. To address this issue, we propose Negative-Direction Aware Decoding (NDAD), a novel decoding method that identifies and exploits hallucination signals as repulsive directions in the model’s representation space, thereby improving factual adherence without retraining. Specifically, NDAD elicits hallucination-leaning signals by selectively masking critical attention heads, which exposes unstable hypotheses that the model would otherwise amplify during generation. To regulate the influence of these signals, NDAD employs two complementary weights: a global alignment weight measuring how well the induced signal aligns with the layer’s native activations (thus quantifying its referential utility) and a local weight estimating whether low-probability tokens in the masked distribution are likely to evolve toward the final output. Based on the weights, we derive a latent hallucination distribution that serves as the negative direction. A lightweight gradient-descent step then subtracts mass from hallucination-prone regions of the output distribution, adjusting the final logits while preserving the model’s high-confidence predictions. Extensive experiments across multiple LLMs and diverse benchmark datasets demonstrate that NDAD consistently enhances factual reliability without requiring additional training or external knowledge.

Published as a conference paper at ICLR 2026
NDAD: NEGATIVE-DIRECTION AWARE DECODING FOR LARGE LANGUAGE MODELS VIA CONTROLLABLE HALLUCINATION SIGNAL INJECTION
Panjia Qiu Mingyuan Fan'
Cen Chen Daixin Wang
School of Data Science and Engineering, East China Normal University fmy 022266666666mmmee. com daixin. wdx(eantgroup.com
AAttGroup
panjiagiuustta ecnu. edu. en cenchen@dase ecnu.
ABSTRACT
Large language models (LLMs) have recently achieved impressive progress knowledge-intensing and reasoning tasks. However, their tendency to produce fabricated or factually inconsistent content remains a fundamental challenge to their practical deployment. To address this issue, we propose Pegative-Directiin Aware Decoding (NDAD), a novel decoding method that identifies and exploits hallucination signals repulsive directions in the model's representation space, thereby improving factual adherence without retraining. Specifically, NDAD elic- its hallucination-leaniig signals by selectively masking critical attention heads, which exposes unstable hypotheses that the model would otherwise amplify during generation. To regulate the influence of these signals, NDAD employs two com- plementary weights: global alignment weight measuring how well the induced signal aligns with the layer's native activations (thus quantifying its referential utility) and local weight estimating whether probabbibility tokens in the masked distribution are likely to evolve toward the final output. Based on the weights, we derive latent hallucination distribution that serves the negative direction. A lightweight cradient-desceen step then subtracts mass from hallucination- -roone regions of the output distribution, adjusting the final logits while preserving the model's 's high-confidence predictions. Extensive experiments across multiple LLMs and diverse benchmark datasets demonstrate that NDAD consistently enhances factual reliability without requiring additional training or external knowledge.
INTRODUCTION
In recent years, large language models (LLMs)) have achieved remarkable breakthroughs over various
tasks (Achiam et al. 2023) Anil et al., 2023] Touvron al. (023b Team et al.) (023). However,
pervasive challenge is the phenomenon of hallucination, wherein LLMs generate factually incorrect,
fabricated, or nonsensical information with high confidence (Zhang et al. 2023b et al. 2024 Fan
et al. 2025).
Current approaches to mitigate hallucinations mainly fall into two categories: satrivval-augmeneed methods (Li ct al. 2023b Min et al. (023) and training- based methods (Tian et al. 2023 Rafailoy
al. 2023). Retrieval-aue -uggentee methods, while effective, often introduce architectural complexity, latency, and dependency on the availability and integrity of external large-scale databases. Training- based methods, on the other hand, can be computationally intensive and may struggle to generalize across diverse factual domains. A less explored, yet highly promising, avenue is the optimization of the decoding process itself (Welleck et al.) 22024, Shi et al. (024). Importantly, prior studies suggest that LLMs already encode factual signals within their internal representations
byproduct of large- scale pretraining, though conventional decoding techniques often fail to surface this latent knowledge (Wang et al. 2020 Kadavath et al. 2022 Li al. [023a Saunders et al. 2023). Motivated by this
observation, intervention- bassed decoding methods (Chuang et 2003 Li et al.) [023a 2022 Zhang
al. 2023a) have been developed to exploit these factual signals alleviate hallucinations.
"Corresponding author.

---

Published as a conference paper at ICLR 2026
In this study, we propose novel yet quite effective intervention decoding method called Negative- Direction Aware Decoding (NDAD). Unlike prior approaches that focus on extracting latent factual cues from early layers, NDAD instead identifies hallucination signals and then leverages them to calibrate the decoding process. As shown in Figure [ NDAD detects latent distributions that correlate with factually incorrect outputs by masking influential attention heads in the model. We then develop two complementary weights based on the detected distributions to suppress hallucination signals. Specifically, the global weighting component evaluates the alignment between hallucination- logits and earlier-layer logits, estimating whether such trajectories reflect distributions the model is more likely to generate. In parallel, the local weighting component tracks high-risk tail tokens to assess their likelihood of advancing toward the final output. The final token selection is then guided by a single-step gradient-descent adjustment, which penalizes the generation of tokens associated with identified hallucination risks. Our main contributions are as follows:
We propose NDAD, an innovative decoding approach that introduces hallucination signal to expose the model's 's underlying hallucination distribution and applies a negative awareness mechanism for intervention.
We incorporate global weight measuring the directional consistency between hallucination signal and original early-layer logits, and local weight quantifying the likelihood of tail tokens evolving toward the mature distribution.
diverse set of LLMs with different configu- rations and scales. The experimental results indicate that NDAD reliably enhances factual accuracy across multiple tasks and benchmark datasets.
We perform comprehensive experiments
2 RELATED WORK
Hallucination Mitigation. In LLMs, hallucination refers to the generation of content that diverges from factual knowledge, and it has become critical bottleneck for ensuring model reliability. Existing research has proposed mitigation strategies along several directions. Retrieval-based methods introduce external knowledge to calibrate factuality, such as
Generation (RAG)
(Cheng et al.) 2023 Chen et al. 2024a| Fan et al.) 2024 Lewis et al., 2020), or enhance attribution by applying retrieval and editing after generation to improve both factuality and traceability (Gao et al.) 2022; Mishra er al. 2024). Training- and preference-basee
methods rely on additional supervised data or human preference signals for optimization, including Supervised Fine-Tuning (SFT) Tian
cet al. 2023). Reinforcement Learning with Human Feedback (RLHF) (Ouyang et al. 2022), and Direct Preference Optimization (DPO) (Rafailov et al. 2023), thereby reducing hallucination through parameter updates. celf-evaluation-bere based methods do not rely on external data; instead, they improve reliability by leveraging multiple inference-time samples and incorporating techniques such as self-criticism (Saunders et al.) 2023) and diversified reasoning path sampling (Wang et al. (022). To further improve efficiency in enhancing factuality, our goal is to directly optimize the output distribution of language models, thereby strengthening their robustness.
Intervention Decoding. In recent years, a line of research has emerged that enhances the factuality of LLMs by intervening during the decoding time. Inference-Tinee Intervention (TTI) (Li et al. 2023a) identifies attention heads correlated with truthfulness during inference and shifts activations along these "truthful directions". thereby enhancing the truthfulness of generated outputs. Similarly, Activation Decoding (AD) (Chen et al. (024bb leverages the model's internal representations by introducing an entropy-basee metric of contextual activation sharpness as a decoding constraint, thereby biasing outputs toward more reliable generations. Inspired by early work on Contrastive Decoding (CD) (Li et al. 2022). which compared strong expert models against weaker amateur models to improve fluency and coherence without addressing factuality, subsequent studies extended the idea of "contrast" to the logits level. For example, Auto-Contrastive Decoding (ACD) (Gera et al., 2023) requires fine-tuning the prediction heads of earlier layers and is therefore mainly applicable to small-scale models. In contrast, Decoding by Contrasting Layers (DoLA) (Chuang et al. 2023) dynamically selects the early layer that exhibits the largest semantic divergence from the final layer, thereby suppressing erroneous tendencies in lower layers. Building upon this, Self Logits Evolution Decoding (SLED) (Zhang et al. 2024) further integrates multiple early layers through weighted combination and employs a gradient-descent procedure to guide the correction of the final logits, resulting in more robust factuality enhancement.
2

---

Published as a conference paper at ICLR 2026
Where were the Games of the 30th Olympiad held?
Vanilla Forward Propagation
Incorporation Hallucination Signals
8th layer
Togits,
logizs
layer
logtts
16th layer
logits so
logizsts
16th layer
24th layer
logits,
logits
logizega
24th layer
Lth layer
togits
Walobal
logits
Tall tolens distribution
P
Ilcul
togits,
Original: logits,
plul
lngria
False
True
Contrastive Decoding
pintt log loy
Parts
Londoa
Begjop
The tokens indicnte the approximate correctmess direction
pame
logits
xDAD
(Watooou Wyinnal
Party
Dellog
logits, logits, K. MDAD lagins
Figure 1: Overview of NDAD. To enhance factual reliability, we introduce hallucination signals to adjust the final output logits. The weights Wigloua and Wylccal jointly regulate the hallucination signals to form the latent allucination distribution PNDAD, from which the model steers its output away.
METHOD
LLMs are designed to autoregressively predict the next token given a preceding context. Formally, given input prefix represented x {x1, I2 ........... 1r-1). the model first converts these tokens into a sequence of embedding vectors, Ho [ hi ...,. 1/f--1), through an embedding layer. These representations are then updated successively by stack of L transformer blocks. We denote the hidden state of the t-token at the 1-th block as h/") Rdd, To generate a probability distribution over the model's vocabulary V, shared projection head y Rdd Rd is applied to the hidden states. In detail, from the 1-th layer's hidden state, the unnormalized score vector (logits) for the next token and its corresponding probability distribution are defined as:
p(1] softmax(logits((), where logits!" 2 (b( 1= 1=
Typically, the logits from the final layer, logits are used for decoding. However, this can lead to generations that are plausible but factually incorrect or nonsensical. To mitigate this issue, propose NDAD, which adjusts the logits by leveraging hallucination signal, thereby improving the reliability of the generated text.
HALLUCINATION SIGNAL GENERATION
Unlike prior approaches such DoLa (Chuang et al. 2023) and SLED (Zhang al. 2024). which harness early-layer representations proxy for faithful evidence to reshape the final token distribu- tion. we instead attempt explicitly separate the hallucination signal encourage the final output distribution to diverge from it. Intuitively, this shifts calibration from boosting positives to subtracting negatives. In this way, our method can prevent probability mass from accumulating on spurious or speculative trajectories.
Prior studies (Wu et al. 2024) have demonstrated that certain attention heads in LLMs play a critical role in preserving factuality and stabilizing generation. Once the support of these heads is weakened, the model tends to deviate from factual directions, making the decoding process more susceptible hallucinations. Building this insight, exclusively mask influential heads to isolate a hallucination signal, which serves as a negative direction for contrastive decoding. To determine which heads should be masked, adopt head importance scores from prior work (Wu et al. 2024
3

---

Published as a conference paper at ICLR 2026
to evaluate the importance of each head. Furthermore, take into account the entropy of each layer's distribution: lower entropy indicates that the importance is concentrated on a small subset of heads, suggesting that these heads more influential. By integrating both head importance and layer-level entropy, we achieve a more precise selection of heads to be masked. As illustrated in Figure for each block 1 T., following Wu et al. (2024), first obtain score list of n. heads {st,1, .1.,, st,n f in this block. We then normalize the scores into probability distribution and compute the layer entropy as follows:
E1 -
Σ PL. Pr. Pt,
31, 25=1 8L.
(2)
Here, Pt, denotes the normalized importance of head in layer I, and Er measures the uncertainty of head importance within this layer. Then we select the top K layers with the lowest entropy and mask the top x heads within these layers to separate hallucination signal. We represent the hallucination signal corresponding to (-th block logitspusst The complete algorithmic workflow can be found in Appendix Algorithm After extracting these hallucination signals, the remaining question is how to leverage them calibrate the model's final outputs.
Hallucination Signals
Could the members of The Police perform lawful arrests?
Capture
Eg-1
Hu.4
Hu.43
H
Fg
E2
Hg
H
H
H
Ha
H
Hau
Hand
Ha,
Haa
Hy
H
Hus
Sorted by Entropy
E,
E
E,
11
H2
Higd
H
11g
Ha,
H4a
Ha
Hi..
H.,
Figure 2: Hallucination signal generation. Darker colors indicate larger values, and gray cells correspond to masked attention heads.
3.2 DYNAMIC WEIGHTING VIA GLOBAL CONSISTENCY AND LOCAL DIVERGENCE
To exploit the identified negative direction, propose dynamic weighting framework that integrates both global and local perspectives.
Global Consistency. At the global level, we evaluate the directional consistency between the halluci- nation signal and the original early-layer logits from the same layer, which provides quantitative assessment of the correlation between the original signal and the hallucination signal. Specifically, the directional consistency a at layer is measured by computing the cosine similarity between the hallucination signal logitsmast mask and the original logits logits; at the same layer
Welooal = p(c)).
cos_sim (logits,, logits mask
(3)
where (-)) denotes a linear mapping that scales values into the range [0, 1]. By measuring directional consistency, we assess the correlation between the hallucination signal logitssmast and the model's original logits logits,, thereby providing quantitative basis for the referential value of ballucination signal layer A higher consistency indicates that the signal more closely aligned with the model's 's latent hallucination direction. Accordingly, the weighting scheme increases the contribution of more relevant hallucination signals.
Local Divergence. At the local level, we further examine the distribution of low- www-probbillity tokens. Consistent with prior studies (Chuang et al. 2023 Zhang et al. 2024), we approximate the final-layer logits logits,, the ground-truth distribution. We define the evolution trajectory from the premature to the mature state as logits 1 -ogits/makk For the final mature layer, we further obtain the probability distribution Plogites sof fmaa waaataaalltttt.. (logits select the top-/ tokens, and construct I one-hot vectors = (Pe, Pey.... Pe, to serve as approximate distributions of mature, where the index of the selected token is set and all others are set O. In order to derive the hallucination distribution
4

---

Published as a conference paper at ICLR 2026
layer we begin with Plogitsqus softmaz(logita)) Mahaut et al. (2024) suggested that low- probability tokens typically correspond reduced factuality. Based this observation, define our final hallucination distribution by removing the top-/ tokens, which encourages the resulting distribution approximate the negative direction more closely. particular, by assigning a very small probability - 0 to the top-/ tokens in P "oogitspat, we are able to derive cleaner representation of the premature distribution, which defined as piuil logits As illustrated in Figure the vector dj
denotes the evolution trajectory from the premature signal logit.s/ask layer to the mature signal logits,, while d2 represents the trajectory from the hallucination distribution prait logitsqua toward candidate distribution of correctness Pe. Both dj and d2 can be interpreted as representations of the the trajectory of factual evolution, and thus we have:
d1
direction
d2, where
d1 logitss logits) mask d2 V I.
logitsquakk Pec))
logitopua is more likely to evolve toward the mature output, and therefore larger weight should be assigned to suppress its evolution. To quantify this evolution trajectory, we define the local weight as:
Intuitively, if d1 and d2 are more closely aligned, indicates that the token in ptait
Whis max [ossssi (logils)"ms logilST. Piail logitsqu Pa,), 0). iE [1,
(5) After deriving both the global and local weights, we integrate them to obtain the final weight for cach correctness direction within the top-/ tokens. Specifically, for the one-hot vector Pe, corresponding to the i-th distribution in the correctness, the final weight at layer is defined as:
W₁, = y i E [1.
To better capture dominant signals and attenuate weak noisy ones. apply squared trans- formation to the final weight scores. This operation accentuates high-confidence directions while diminishing the influence of marginal ones, thereby producing sharper weighting distribution (Hinton et al. 2015 Müller et al. 2019; Zhang et (002). Formally, the squared weight is:
Wr, (Wh,)² E [1.4]
3.3 NEGATIVE- DIIIIIEIOOIE AWARE DECODING
After introducing the global and local weighting mechanisms, we now integrate them into the overall decoding framework. NDAD leverages these weights to controllably exploit the injected hallucination signal and employs an update in the direction of gradient-descent to guide the model away from hallucination directions during generation. The following describes the specific procedure for adjusting the final-layer logits, we first perform intra-layer normalization on the obtained signals, followed by inter-layer aggregation. The squared weights Wi, are normalized across the correctness directions within each layer, resulting in layer-wise normalized distribution. Formally, the latent distribution of layer is expressed as:
Pi = (Wi,,, Wi..,..,, W..,,,, 2
Σ W
(8)
We further apply inter-layer weighting to obtain the final NDAD distribution:
Zi
PNDAD
N.
where Ni = Σ1_ Zi
(9)
1=1
Here, Mg denotes the relative contribution of layer ensuring that the aggregation respects the proportional importance of each layer while preserving comparability across layers. By incorporating negative-directio awareness, we obtain latent hallucination distribution PNDAD- To suppress the generation of hallucination-prie -rronccon---nnneeeee tokens, penalize the divergence between distribution PNDAD and the original distribution Piogitez using the KL divergence term. The procedure is outlined in Algorithm 2 Here, the parameter a, referred to the Evolution Rate and originally introduced in the (Zhang et al., 2024), controls the magnitude of adjustment applied to the logits along the gradient direction. We then obtain the final adjusted logits as shown below:
logits 2 -logitss a VK VKKL(PNDAD. Paoni Plogitsz)
(10)
5

---

Published as conference paper at ICLR 2026
Table I: Evaluation results of different methods Llama models over varying datasets.
TruthfuQA(MC)
Factor Wiki
CoT
Method
MC1
MC2
MC3
Avg.
StrQA
GSM8K
Llama2-7B-base
26.58
41.88
18.96
29.14
58.42
60.74
13.95
+DoLa-low
33.04
63.73
31.25
42.67
63.36
59.56
14.63
+DoLLa-high
31.77
63.26
30.40
41.81
62.56
60.44
13.19
+AD
32.41
49.89
24.03
35.44
53.14
1.97
2.12
+SLED
34.15
62.57
31.89
42.87
67.00
61.27
14.63
+NDAD
34.39
62.62
31,98
43.00
67.30
61.57
14.86
Llama2-77 -Bhhha
35.62
57.47
32.10
41.73
56,68
63.58
21.23
+DoLa-low
34.18
62.80
31.00
42.66
56.58
64.59
21.46
+ DDLa-hhghh
33.92
61.75
30.40
42.02
56.25
64.19
20.85
+AD
32.15
49.90
23.99
35.35
51.44
+SLED
37.09
63.83
32,96
44.63
64.80
64.50
21.53
+NDAD
36.84
63.42
32.93
44.40
65.06
64.67
21.99
Llama2-13B-base
27.59
43.14
19.53
30.09
63.79
65.98
28.81
+DoLa-low
31.57
62.48
30.41
41.49
65.70
66.46
28.51
+DoLa-high
29.38
63.92
33.62
42.31
52.84
60.83
11.90
+AD
32.15
49.90
23,99
35.35
58.18
2.01
0,00
+SLED
34.76
63.58
31.88
43.41
70.94
29.19
+NDAD
34.88
63.60
31.97
43.48
71.18
66.81
29.26
Llama2-15 Mlamaa-13B-aaa 13B-chtt
36.47
63.06
32.77
44.10
61.96
69.65
36.69
+DeLa-low
34.27
63.27
31.36
42.97
60.69
69.48
35.48
+DoLa-high
31.82
62.55
31.13
41.83
54.81
66.51
33.21
+AD
32.15
49.90
23.99
35.35
56.71
23.14
0.00
+SLED
37.45
63.50
32,90
44.62
67.50
69.74
37.15
+NDAD
37.58
63.63
33.02
44.74
67.74
69.96
37.30
EXPERIMENTS
4.1 EXPERIMENTAL SETUP
Benchmark datasets. We evaluate our approach against strong baselines across both multiple-choice and open-ended generation tasks. For multiple-choice settings, we employ the TruthfulQA (Lin et al. 2021) dataset to measure factuality in short-answer scenarios and the FACTOR (Wiki) (Muhlgay et al. 2023) dataset to assess performance in aangppaagraph contexts. For open-ended generation, consider PopQA (Mallen et al. 2022),, NQ-Open (Lee et al.) (2019), and TriviaQA (Joshi et al.) 2017), as well as reasoning-intensive tasks involving chain-of-thought (CoT), including StrategyQA
(Geva et al. 2021) and GSM8K (Cobbe al. 2021).
Models and Baselines. In our experiments, we adopt a diverse set of representative open-source LLMs, including Llama2-7B (base and chat) (Touvron al. (023b), Llama2-13B (base and chat)
(Tourron et al. 2023b), Qwen2.5-7B 5-B-insttrtt (Team) 2024). Mistral-7B-instruct (Jiang et al. 2023
and -lama3-8BB-nnstuue (Grattafiori et al. 2024p. We compare the following baselines: (1) Greedy
Decoding. (2) DoLA-Low (Chuang et al. (2023) subtracts the logits of the most distributionally
different layer from the first half of the network from the final-layer logits. (3) DoLA-Higgh (Chuang
al. 2023) subtracts the logits of the most distributionally different layer from the second half of the network from the final-layer logits. (4) AD (Chen al.) (024b) uses an entropy- -basd measure of contextual activation sharpness to constrain decoding with the model's internal representations. (5) SLED (Zhang et al. 2024) integrates multiple early layers via weighted combination and applics gradient-descent adjustment to refine the final logits for improved factuality.
Metrics and Parameters. For mmulipple-choice and CoT reasoning tasks, we evaluate factual accuracy following the approach in (Chuang et al., 2023). To assess correctness TriviaQA, HotpotQA, and NQ-Open, we adopt the Exact Match (EM) metric, consistent with the protocol of (Joshi et al.) 2017).
The detailed parameter settings provided in Appendix A.1

---

Published as a conference paper at ICLR 2026
Table Evaluation results on Open-Ended generation tasks.
Llama2-7B-hhe
Llama2-7B-chat
Method
TriviaQA
PopQA
NQ-Open
TriviaQA
PopQA
NQ-Open
Greedy
65.04
13.67
21.02
59.61
18.55
23.41
+DoLa-low
64,96
13.88
20.78
54.65
19.64
23.60
+ +Dolaa-high
63.96
13.41
19.31
54.24
19.48
23.55
+AD
48.78
15.11
22.44
59.64
18.43
23.60
+SLED
65.10
25.86
25.96
59.61
19.98
23,46
+NDAD
65.21
26.00
26.26
59.67
20.13
23.63
Llama2-133 13B base
Llama2- 13B chat
Greedy
68.34
25.04
32.71
66.32
19.82
30.03
+DoLa-low
68.67
28.64
28.78
65.54
17.82
29.14
+ -oLaa-high
62.08
26.12
25.68
61.86
16.32
27.42
+AD
67.67
17.91
30.80
64.50
22.91
34.52
+SLED
71.47
30.53
32.52
66.40
19.84
29.89
+NDAD
71.66
30.64
32.88
66.48
19.85
30.11
4.2 EVALUATION ON DIFFERENT BENCHMARKS
Multiple-Choices Tasks. These tasks are designed to evaluate whether the decoding strategy can more effectively assign higher probabilities to correct answers or reasonable completions, while suppressing its preference for incorrect options. It should be noted that these tasks essentially coabbuutio-fittting problems, and overfitting to specific tasks often undermines the generalization capability of decoding method. Since goal is enhance factuality and robustness while preserving broad applicability, even when the performance deviation is small or only marginal improvements are achieved, the results remain understandable and acceptable. We validated the effectiveness of the NDAD method through short-answer factuality tests on the TruthfulQA dataset and aanaagrgg factuality tests on the FACTOR dataset. The corresponding experimental results are summarized in Table | and more detailed analyses are provided in Appendix B.I B. Our NDAD method demonstrates strong generalization across different models and datasets, and largely achieves improvements over the baseline SL.DD. This suggests that the proposed decoding strategy is generally more effective at calibrating probability assignment between correct and incorrect answers.
Chain-of- Thought Reasoning Tasks. This task primarily focuses on evaluating how different decoding methods can be adapted to the CoT strategy to effectively handle complex reasoning problems. The detailed results can be found in Table Our NDAD method consistently outperforms all baselines in decoding performance. At the same time, the limitations of AD become particularly evident on CoT datasets. AD constrains next-token probabilities incorporating contextual entropy to enhance factuality. However, falls short on reasoning tasks because tokens in CoT datasets exhibit strong logical dependencies, and relying solely token-level activation entropy from the context may deviate from the original semantics. Moreover, some intermediate tokens lack contextual support and are prone to being misclassified hallucinations, thereby impairing reasoning performance.
Open-Ended Generation Tasks. For open-ended tasks, we adopt TriviaQA, PopQA, and NQ-Open datasets. Our NDAD method consistently achieves further improvements wwwlleggeinttenivee models tend to
the baselines. Results
are shown in Ta Since PopQA and NQ-Open are highly rely more on contextual information during generation. The AD method, which is inherently designed to adjust decoding based on contextual attention, therefore shows exceptionally strong reasoning performance on the Llama- 13B-chal model. However, when compared with the results on CoT tasks in Table 11 it becomes evident that AD exhibits substantial variability. Therefore, our NDAD method demonstrates the strongest robustness.
4.3 EVALUATION ON DIFFERENT LLMS
We further conduct experiments on a broader range of model architectures, including models from different families well as different variants within the same family. As reported in Table NDAD consistently delivers state-of-the-a of-thh--rttt results across all tested configurations, surpassing other baselines. This demonstrates that the proposed method is not only effective for specific model class but
7

---

Published as a conference paper at ICLR 2026
Table 3: Evaluation results on varying LLMs.
TMu))flQA(MC))
Factor
CoT
Model
MC1
MC2
MC3
Avg.
Wiki
GSM8K
Qwen2.5-7B-instrruc
41.00
64.59
38.17
47.92
54.54
84.46
+DoLa-low
36.60
66.03
34,21
45.61
56.08
83.02
+DoLLa-high
34.64
2.37
34.51
23.84
40.85
76.95
+SLED
45.04
70.37
39.88
51.76
62.99
84.91
+NDAD
45.17
70.37
39.89
51.81
63.13
85.14
Mistral- -isttal-7Biinstruet instruct
40.27
68.32
37.06
48.55
60,49
53.45
+ +DoLa-low
39,53
68.44
36,16
48.04
64.16
53,22
+DoLa-high
39.53
68.43
36.09
48.02
64.23
53.30
+SLED
45.41
71.17
40.27
52.28
67.53
53,90
+NDAD
45.53
71.31
40.46
52.43
67.70
54.36
Llama3-8B -instructtt
38.92
68.16
36.56
47.88
59,22
75.97
+DoLa-low
35.74
65.27
33.60
44.87
61.32
75.82
LaoLa-highh
35.99
65.04
33.72
44.92
61.29
75.51
+SLED
41.37
68.46
37.61
49.15
67.07
75.82
+NDAD
41.37
69.21
37.89
49.49
67.20
77.18
Table 4: Evaluation results on Llama2-70B.
Table 5: Runtime and memory overhead on
Llama2-7B-bean
Method
Factor
GSM8K
Method Runtime (s)
Memory (MB)
Alama2-70BB
61.92
56.10
+DoLa-Low
74.05
57.01
Greedy
1.11
13503.47
+DoLaa-High
62.53
38.21
DoLa
1.17
15261.99
+SLED
77.32
57.01
SLED
1.17
15452.88
+NDAD
77.52
57.54
NDAD
1.34
17779.01
also generalizes well across diverse architectures. Morcover, the performance gains are particularly pronounced on CoT datasets such GSM8K, where NDAD exhibits substantial improvements over the baselines. This finding highlights the robustness of NDAD handling complex reasoning tasks. Consequently, these results confirm that NDAD achieves both model generality and strong robustness, making it a versatile and effective decoding strategy.
4.4 EVALUATION ON LARGER-SCALE LLM
To assess the viability of the method on substantially larger models, we conducted additional experiments using Llama2-70B on the Factor dataset for multiple-choice tasks and GSM8K for chain-of-thought reasoning. The results, presented Table show that the method continues to deliver strong performance generative tasks such as GSMSK. The second-best baseline improves by 0.91%, whereas our method achieves an improvement of 1.44%, corresponding to a relative gain of 58%. For the Factor dataset, discussed in Section 4.2 this task essentially evaluates distribution fitting, where maintaining a smooth upward trend sufficient. These results demonstrate that the method remains effective when scaled to much larger models and exhibits strong robustness across different model sizes.
4.5 ABLATION STUDY
Incorporation Hallucination Signal. We first demonstrate that our method indeed introduces hallucination signal into the model. To this end, we directly decode the logits obtained after masking the importance attention heads and evaluate their performance. The experimental results are shown in Figure 3 As can be observed, compared with the original decoding, performance consistently drops across different models and datasets, with the most significant decline occurring on the GSM8K dataset. This indicates that complex reasoning tasks heavily rely the aggregation and inference of internal attention heads, and masking these heads introduces stronger hallucination signal. This observation is consistent with the analysis in Section 4.3 where NDAD method achieves better
8

---

Published as a conference paper at ICLR 2026
Original
Mask
Llamu2-70-buue
Llamx2-33b-aa
Llama2-133 13-
$1.00
9068
13.00
11.00
408
48.0
4000
130
8.99
3000
Vacter
S
COMMAK TriataA
Farter
S0rQ4
GSMAK TMMAAQ
Factor
SerQA
CSMBK Trivaga
Pacter
CONMAK TritaAA
Figure 3: Results from Decoding Hallucination Signals.
67.31
$4.80
67.31
14.10
Cormect
Corved
67.22
$4.99
67.33
34.47
67.17-
14.45
Mumber of musted heads
Number of maskeed heeds
Mumberaf masted layers
Number of masked layers
(a) Different number of masked heads
(b) Different number of masked layers
Figure 4: Different head and layer parameters on the Llama-7B-base
results on GSM8K, suggesting that stronger hallucination signal can provide more effective leverage for enhancing the NDAD decoding strategy. Moreover, the ablation experiments in Table 6 based on random head and layer selection further support that hallucination induction guided by head importance and layer-level entropy contributes to the performance gains of NDAD.
Importance of Head and Layer Parameters. To effectively introduce hallucination signal, it is necessary to mask more important attention heads. Using the Llama-7B-base model as an example, present results on the FACTOR and GSM8K datasets under different parameter settings. Figure 4a illltuustrtts the impact on accuracy when varying the number of masked heads while keeping the number of masked layers fixed. Conversely, Figure 4b shows the effect of varying the number of masked layers while fixing the number of masked heads. Overall, the trend generally follows a lisingtthen-falinn pattern. Notably, throughout the experiments, the range of masked heads and layers remained between [6, 13], within which the model consistently achieved relatively strong performance across both datasets. More detailed results are provided in Appendix B.3
Global and Local Weights. We further analyze the effectiveness of the global and local weighting components in our method. The ablation results based on Llama2- -B-baaseeee and Llama2- 1---2-13Bbbss are reported in Table and the more comprehensive results and analyses can be found in Appendix B.2 Specifically, w/o global weight indicates removing the measurement of directional consistency between hallucination signal and the original signals, while w/o local weight corresponds to excluding the measurement of consistency between the tail-token evolution and the transition from the premature to the mature state. From the results, it is clear that both weighting mechanisms play crucial role in enhancing the decoding performance. For example, in the case of Llama2-7B-ban either global local weights leads to drop in performance. A similar trend is observed for Llama2- 13B-base, where the absence of these weights consistently reduces accuracy across all benchmarks. Importantly, the GSM8K dataset again shows the largest degradation, underscoring that complex reasoning tasks are particularly sensitive to the loss of these weighting mechanisms. These results confirm that both global and local weights contribute complementary benefits, and together they enable NDAD to achieve robust and state-of-the- performance.
removing
4.6 COMPUTATIONAL OVERHEAD ANALYSIS
To evaluate the computational overhead of our method, we measured runtime and memory usage on the Llama2-7B- model using a single GSM8K sample, and the results are presented in Table 5 As shown, the additional cost introduced by NDAD is relatively lightweight, with the primary overhead arising from the incorporation of the negative-dires -irrectioo signal. Consistent with existing decoding-bret based approaches, NDAD only modifies the logits of the final layer, requires additional training, and does not depend on high-quality external data, giving it strong soug-and-plat capability. In many real-world applications, safety and factual reliability are often more critical than achieving

---

Published as a conference paper at ICLR 2026
Table 6: Ablation study on the effectiveness of each component in the NDAD method.
(ru))flQA(MCC)
Factor Wiki
CoT
Method
MC1
MC2
MC3
Avg.
StrQA
GSM8K
Llama2-7B- -B-base
26.58
41.88
18.96
29.14
58.42
60.74
13.95
random head
34.15
62.55
31.91
42.87
67.17
61.13
13.95
random layer
34.15
62.61
31.84
42.87
67.10
61.40
14.71
wlo global weight
34.27
62.57
31.93
42.92
67.20
61.09
14.63
w/o local weight
33.90
61.13
31.43
42.15
67.17
61.44
14.10
NDAD
34.39
62.62
31,98
43.00
67.30
61.57
14.86
Llama2- 13B base
27.59
43.14
19.53
30.09
63.79
65.98
28.81
random head
34.88
63.58
31.94
43.47
71.04
66.72
28.13
random layer
34.76
63.56
31.91
43.41
71.01
66.72
28.66
w/o global weight
34.88
63.59
31.93
43.47
70.98
65.41
28.73
w/o local weight
34.76
63.57
31.89
43.41
70.91
66.07
27.98
NDAD
34.88
63.60
31.97
43.48
71.18
66.81
29.26
the absolute lowest decoding latency; thus, moderate amount of runtime and memory overhead is generally acceptable.
5 CONCLUSION
We present an innovative decoding strategy NDAD, which explicitly elicits hallucination signal by masking critical attention heads and leverages them negative directions for contrastive decoding. To controllably leverage these signals, we design a dynamic weighting mechanism: the global weight measures the directional consistency between the hallucination signal and the original early- layer logits, thereby quantifying the referential value of the current hallucination signal; the local weight characterizes the tendency www-prbbbbility tokens evolve toward the mature distribution. tokens through gradient-descen
By suppressing the output probabilities of hallucination-pron adjustments during decoding, NDAD consistently improves factual reliability across diverse models and benchmarks, demonstrating particularly strong robustness in complex reasoning tasks. In conclusion, NDAD provides lightweight yet effective solution for optimizing LLM decoding.
ACKNOWLEDGMENTS
This work was supported by the Guizhou Provincial Program on Commercialization of Scientific and Technological Achievements (Qiankchezhongyindi [2025] No. 006) and Ant Group.
ETHICAL STATEMENT
This paper presents decoding strategy designed to improve the factual reliability of LLMs. Our research does not involve human subjects, sensitive personal data, or potentially harmful datasets. All benchmark datasets employed in experiments are publicly available and widely used within the Natural Language Processing research community.
REPRODUCIBILITTY STATEMENT
To ensure the reproducibility of our experiments, we have provided the source codes in the supple- mentary materials for review. Upon acceptance of this paper, will release the codes open source to enable researchers to replicate and extend our experiments.
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.
10

---

Published as a conference paper at ICLR 2026
arXiv preprint arXhv:2303.08774, 2023.
Rohan Anil, Andrew M Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, et al. Palm technical report. arXiv preprint 1rXxv:2305.004000 2023.
Jiawei Chen, Hongyu Lin, Xianpei Han, and Le Sun. Benchmarking large language models in retrieval-augmented generation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 17754-17762, 2024a.
Shiqi Chen, Miao Xiong, Junteng Liu, Zhengxuan Wu, Teng Xiao, Siyang Gao, and Junxian He. In-context sharpness as alerts: An inner representation perspective for hallucination mitigation. arXiv preprint arXiv:2403.0/55 2024b.
Xin Cheng, Di Luo, Xiuying Chen, Lemao Liu, Dongyan Zhao, and Rui Yan. Lift yourself up: Retrieval-augmenned text generation with self-memory. Advances in Neural Information Processing Systems, 36:43780-433 43799, 2023.
Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James Glass, and Pengcheng He. Dola: Decoding by contrasting layers improves factuality in large language models. arXiv preprint
arXiv:2309.03883, 2023.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jemry Tworek, Jacob Hilton, Reiichiro Nakano, al. Training verifiers to solve math word problems. arXiv preprint arXiv:21 2021.
Mingyuan Fan, Chengyu Wang, Cen Chen, Yang Liu, and Jun Huang. On the trustworthiness landscape of state-of-the-at generative models: A survey and outlook. Int. J. Comput. Vis., 133(7): 4317-4348, 2025.
Wenqi Fan, Yujuan Ding, Liangho Ning, Shijie Wang, Hengyun Li, Dawei Yin, Tat-Seng Chua, and Qing Li. A survey on rag meeting llms: Towards retrieval-augmentt large language models. In Proceedings of the 30th ACM SIGKDD conference on knowledge discovery and data mining, pp. 6491-6501, 2024.
Luyu Gao, Zhuyun Dai, Panupong Pasupat, Anthony Chen, Arun Tejasvi Chaganty, Yicheng Fan, Vincent Y Zhao, Ni Lao, Hongrae Lee, Da-Cheng Juan, et al. Rarr: Researching and revising what language models say, using language models. arXiv preprint arXiv:2210.08726, 2022.
Ariel Gera, Roni Friedman, Ofir Arviv, Chulaka Gunasekara, Benjamin Sznajder, Noam Slonim, and Eyal Shnarch. The benefits of bad advice: Autocontrastive decoding across model layers. arXiv preprint ar 2rXiv:2305.06622 2023.
Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant. Did aristotle laptop? question answering benchmark with implicit reasoning strategies. Transactions of the Association for Computational Linguistics, 9:346-361, 2021.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et The llama herd of models. arXiv preprint arXiv:2407.2178. 2024.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in neural network. arXiv preprint arXiv:1503.02533 2015.
Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b. arXiv preprint arXiv:2310.06825, 2023.
Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. Triviaqa: A large scale distantly supervised challenge dataset for reading comprchension. arXiv preprint arXiv:1705. 03551, 2017.
Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran Jonn-Jonnoon, et al. Language models (mostly) know what they know, arXiv preprint 22t:::2207.00222. 2022.

---

Published as a conference paper at ICLR 2026
Kenton Lee, Ming- Wei Chang, and Kristina Toutanova. Latent retrieval for weakly supervised open domain question answering. arXiv preprint 190v:0990.000300, 2019.
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. Retrieval-augmen genera- tion for nowwedde-intenssive nlp tasks, Advances in neural information processing systems, 33: 9459-9474, 2020.
Junyi Li, Jie Chen, Ruiyang Ren, Xiaoxue Cheng, Wayne Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. The dawn after the dark: An empirical study on factuality hallucination in large language models. arXiv preprint arXiv:2401.0222 00300,, 2024.
Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Plister, and Martin Wattenberg. Inference-time intervention: Eliciting truthful answers from language model. Advances in Neural Information Processing Systems, 36:41451-41530, 2023a.
Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori Hashimoto, Luke Zettlemoyer, and Mike Lewis. Contrastive decoding: Open-ended text generation as optimization. arXiv preprint arXh::2210.50097 2022.
Xiaonan Li, Changtai Zhu, Linyang Li, Zhangyue Yin, Tianxiang Sun, and Xipeng Qiu. Llatrieval: Lim-verified retrieval for verifiable generation. arXiv preprint arXiv:2311.07838, 2023b.
Stephanie Lin, Jacob Hilton, and Owain Evans. Truthfulqa: Measuring how models mimic human falsehoods. arXiv preprint arXiv:2 2021.
Matéo Mahaut, Laura Aina, Paula Czarnowska, Momchil Hardalov, Thomas Müller, and Lluís Mårquez. Factual confidence of lms:: on reliability and robustness of current estimators. arXiv preprint arXiv:2406. 13415, 2024.
Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das. Daniel Khashabi, and Hannaneh Hajishirzi. When not to trust language models: Investigating effectiveness of parametric and non-parametric memories. arXiv preprint arXiv: 10511, 2022.
Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Wei Koh, Mohit lyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. Factscore: Fine-grained atomic evaluation of factual precision in long form text generation. arXiv preprint arXiv:2305. 14251, 2023.
Abhika Mishra, Akari Asai, Vidhisha Balachandran, Yizhong Wang, Graham Neubig, Yulia Tsvetkov, and Hannaneh Hajishirzi. Fine-grained hallucination detection and editing for language models. arXiv preprint urXiv:2401.066: 006555, 2024.
Dor Muhlgay, Ori Ram, Inbal Magar, Yoav Levine, Nir Ratner, Yonatan Belinkov, Omri Abend, Kevin Leyton-Browe Amnon Shashua, and Yoav Shoham. Generating benchmarks for factuality evaluation of language models. arXiv preprint arXiv:2307.06908 2023.
Rafael Müller, Simon Kornblith, and Geoffrey E Hinton. When does label smoothing help? Advances in neural information processing systems, 32, 2019.
Long Ouyang. Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730- 27744, 2022.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher
Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems, 11553228-53441 2023.
William Saunders, Catherine Yeh, Jeff Wu, Steven Bills, Long Ouyang, Jonathan Ward, and Jan Leike. Self-critiqiing models for assisting human evaluators, 2022. URL https://arxiv. org/abs/2206.0500 058802, 2023.
12

---

Published as a conference paper at ICLR 2026
Chufan Shi, Haoran Yang. Deng Cai, Zhisong Zhang. Yifan Wang, Yujuu Yang. and Wai Lam. A thorough examination of decoding methods in the era of llms. arXiv preprint axii:2402..06925, 2024.
Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptinn Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11800, 2023.
Qwen Team. Qwen2 technical report. arXiv preprint arXiv:2407.10671 2, 2024.
Katherine Tian, Eric Mitchell, Huaxiu Yao, Christopher Manning, and Chelsea Finn. Fine-tuning language models for factuality. NeurlPS 2023 Workshop on Instruction Tuning and Instruction Following, 2023.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anme Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302. 13971, 2023,,
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajiwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.0922 2023b.
Chenguang Wang, Xiao Liu, and Dawn Song. Language models are open knowledge graphs. arXiv preprint arXiv:2010. 11967, 2020.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdh- ery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171 2022.
Sean Welleck, Amanda Bertsch, Matthew Finlayson. Hailey Schoelkopf, Alex Xie, Graham Neubig, Ilia Kulikov, and Zaid Harchaoui. From decoding to meta-generation: Inference-time algorithms for large language models. arXiv preprint arXiv:2406. 16838, 2024.
Wenhao Wu, Yizhong Wang, Guangxuan Xiao, Hao Peng, and Yao Fu, Retrieval head mechanistically explains long-context factuality. arXiv preprint WrXiv:2404.15574 2024.
Shang-Bin Zhang, Peng-Tar Tao Jiang, Qibin Hou, Yunchao Wei, Qi Han, Zhen Li, and Ming-Ming Cheng. Delving deep into label smoothing. IEFE Transactions on Image Processing, 30:5984- 5996, 2021.
Jianyi Zhang, Da-Cheng Juan, Cyrus Rashtchian, Chun-Sung Ferng, Heinrich Jiang, and Yiran Chen. Sled: Self logits evolution decoding for improving factuality in large language models. Advances in Neural Information Processing Systems, 37:5188-522 5209, 2024.
Yue Zhang, Leyang Cui, Wei Bi, and Shuming Shi. Alleviating hallucinations of large language models through induced hallucinations. arXiv preprint arXiv:23/2./57/11 2023a.
Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, al. Siren's 's song in the ai ocean: a survey on hallucination in large language models. arXiv preprint arXiv:2309.0/219, 2023b.
A EXPERIMENTAL SETTINGS
A.1 PARAMETER SETTINGS
For the parameters a in Equation 10 and the I correctness distributions in Equation 5 we set the default values to a and tuning may be required in special cases. Following the work of (Zhang et al. (2024), we test from {0.01, 0. 1, 2. 5, 10) and / from (5. 10. 20, 50). During the aforementioned tests, we guarantee that the chosen parameters achieve performance better than greedy decoding. On this basis, we then incorporate our hallucination signal to conduct adaptive negative-directio aware decoding. For the number of masked heads and layers used in introducing hallucination signal, partly explained this in Section 4.5 In experiments, we usually set the range to [6, 13), which generally yields strong performance.
However, due to dataset uncertainty, additional hyperparameter
13

---

Published as a conference paper at ICLR 2026
Pacter
22.00
OSAEK
05.05
Pactor
22.01
CSMEK
Cerrect Bate
$5.07
Cornecd BATA
+2.80
$5.04
Conrect Bate
23.90
66.99
23.80
23.70
64.95
12
22.11
11
12
22.59
53
17
12
(a) Different number of masked heads
(b) Different number of masked layers
Figure Different head and layer parameters on the Llama-7B-chat.
B ADDITIONAL EXPERIMENTAL RESULTS
B.1 EXTENDED ANALYSIS OF MULTIPLE-CHOICE TASKS
As shown in Table D the performance improvements of NDAD multiple-choice tasks are slightly smaller compared to other task types. This consistent with the discussion in Section 4.2 where multiple-choice problems essentially reduce logits-fitting task; long as the model achieves stable performance without large fluctuations and delivers moderate gains, the results remain reason- able. Moreover, since the multiple-choice format inherently constrains the output space with fixed set of candidate answers, the likelihood of hallucination is substantially reduced, leading to weaker hallucination signals and thus smaller benefits from NDAD's decoding adjustments. Nevertheless, our primary focus is on more complex open-ended generation tasks, where hallucinations are far more prevalent and where NDAD demonstrates clear advantages in suppressing hallucination-pron directions and enhancing factual reliability.
B.2 EXTENDED ABLATION ANALYSIS
We further conducted ablation experiments on Llama2-7B-char and LJama2-13B-chan to examine the effect of different components NDAD, with the experimental setup summarized Table 7
Hallucination Signal Induction. During the stage of hallucination signal induction, we observed that the random selection of attention heads or layers occasionally outperformed our guided masking strategy based head importance and layer-level entropy. This can be attributed to the inherently greedy nature of the masking strategy: although generally effective, does not fully explore the extensive search space. Consequently, certain random configurations may fortuitously yield superior outcomes. Nonetheless, such instances are expected and do not diminish the overall effectiveness of a principled importance-guided approach.
Global Weighting in Hultiple-Choice Tasks. For the global weighting component, the performance on Llama2-7B-chal with the TruthfulQA dataset was slightly better when the global weighting was not applied compared to the full NDAD method. As discussed in Section B. these multiple-choice tasks essentially reduce a logits-fitting problem with small set of candidate answers. Since all options are inherently more reliable than open-ended generations, the model is less vulnerable to noisy hallucinations in this setting. Consequently, assessing the reliability of hallucination signals becomes less critical, and the global weighting may even introduce unnecessary adjustments that interfere with straightforward logits alignment. By contrast, in open-ended generation tasks, where hallucination is more prevalent, the global and local weighting strategies play much more important role enhancing factual reliability.
B.3 EXTENDED PARAMETER ANALYSIS
We further conducted hyperparameter experiments on Llama2-7B-chat. As shown in Figure 5 for both the number of masked attention heads and the number of masked layers, performance exhibits general toen-thhe--lllln trend: as the number of masked heads or layers increases, performance initially improves but declines once the masking becomes excessive. The results suggest that the optimal settings typically lie within the range to 13, where better balance is achieved between inducing hallucination signals and preserving the original representations.
14

---

Published as a conference paper at ICLR 2026
Table 7: Additional ablation study on the effectiveness each component the NDAD method.
TruthfluAAMMC)
Factor Wiki
CoT
Method
MC1
MC2
MC3
Avg.
StrQA
GSM8K
Llama2-7B-che
35.62
57.47
32.10
41.73
56.68
63.58
21.23
random head
36.84
63.38
32.65
44.29
64.93
64.72
21.15
random layer
36.47
62.99
32.59
44.02
65.00
64.72
20.62
w/o global weight
36.84
63.71
32.80
44.45
64.93
64.37
21.00
w/o local weight
36.47
60.82
32.40
43.23
64.96
63.58
20.62
NDAD
36.84
63.27
32.76
44.29
65.06
64.67
21.99
Llama2-155 1-B-chatt
36.47
63.06
32.77
44.10
61.96
69.65
36.69
random head
37.45
63.61
32.95
44.67
67.47
69.91
35.63
random layer
37.70
63.58
33.07
44.78
67.57
69.52
35.78
w/o global weight
35.62
63.91
32.49
44.01
67.60
69.43
35.71
w/o local weight
37.21
64.02
32.90
44.71
67.67
69.65
37.00
NDAD
37.58
63.63
33.02
44.74
67.74
69.96
37.30
B.4 EXTENDED LINGUISTIC QUALITY EVALUATION
To assess whether NDAD introduces any degradation in linguistic quality, we conduct additional evaluation focusing on fluency, coherence, and comprehensibility These dimensions reflect whether the generated responses remain natural, logically organized, and easy to understand- quualiiies that are essential for real-wor world deployment but are often overlooked in factuality-oriented methods. We generate model outputs using Llama2-70B GSM8K and obtain linguistic quality scores from the external evaluator Gemini-2.5-Pro. The results are presented Table 8 As shown, the scores across all methods highly consistent, and NDAD performs on par with slightly better than existing decoding strategies, indicating that NDAD does not introduce noticeable negative effects on linguistic quality. This evaluation further demonstrates that NDAD improves factuality while preserving the naturalness and readability of generated text. Table 9 is the full evaluation prompt used for scoring with the Gemini model.
Table 8: Linguistic quality evaluation of different decoding methods using Gemini-2.5-Prr
Method
Fluency
Coherence
Comprehensibility
Greedy
9.37
7.96
8.65
DoLa
9.29
7.91
8.58
SLED
9.32
8.02
8.69
NDAD
9.31
8.04
8.67
C ALGORITHM OF NDAD
The entire algorithmic workflow of the NDAD method is presented in Algorithm 1 and
D CASE STUDY
Table TO reports the results of the Alama-7B-Bass model the GSM8K dataset under different decoding strategies. The examples demonstrate that our NDAD method is more effective in eliciting factual outputs from the model.
15

---

Published as a conference paper at ICLR 2026
Table 9; Prompt for Gemini-2.5-Pro.
You are an advanced artificial intelligence review system specialized in evaluating the quality of model responses. Your task is to rate the quality from three perspectives: fluency, coherence, and comprehensibility. evaluation dimensions below to score each item (range: 0--10, with higher scores indicating better quality).
Please strictly follow the
[Evaluation Criteria]
Fluency: Whether the sentence structure of the answer is clear and natural, with no obvious grammatical errors, inappropriate word usage, or issues affecting the reading experience. Higher scores indicate smooth language that can be read without difficulty.
Coherence: Whether the logical connection ions between parts the answer are tight and information flows smoothly. Check for jumps, breaks, contradictions, or repetition that affect logical coherence. Higher scores indicate clear thinking and reasonable structure.
Comprehensibility: Whether the answer is easy for the target reader to understand. Higher scores indicate clear information delivery easy understanding, and no ambiguity or obscure expressions.
[Output Format]
Please output in the following JSON format:
"Scores for Each Dimension"; {
"Fluency": score, "Coherence": score, "Comprehensibility"" score },
"Reason for Scoring"; Explain the reasons for scoring each dimension, and briefly summarize the overall evaluation
Please validate the question and return the result in JSON format, with other content except the JSON.
16

---

Published as a conference paper at ICLR 2026
Algorithm Hallucination Signal Induction
I: LLM with layers, sequence, following the work of (Wu al., 2024). original score list of n attention head {81,1; 81,2; .. 81,r in layer 1, number of masked attention heads x, number of masked layer K.
2: for I < T. do
,,...:7. 4: Compute attention head scores distribution entropy: Er Di=1 Pi, log Pr.
3: Normalize scores into probability distribution: Pr, _j=1 81,3
81,1
5: end for
6: Obtain the set of distribution entropy {Ei, E,, Er.)
7: Select the set C consisting of the K layers corresponding to the largest entropy values.
8: for I L do
9: Set the weights the top-z scoring attention heads to O.
10: end for
11: The sequence into the LLM to obtain the hallucination signals logits""m*, mask where S L.
12: Return: (logit.qpask logitsmask l..loottssaaak
Algorithm 2 Negative- Direction Aware Decoding
1: Initializatinn: LIM with T. layers, sequence, a Equation 10 number of correctness directions
I, + 0, sp-) maps values into [0, 1]. the one-hot vectors T (Pey Pey Pe.. Pey) of correctness directions.
2: The sequence into the LLM to obtain the original logits logits, and hallucination signal logits mask given by Algorithm the probabilities each layer denoted Plogits (eagttrrloogsss and Piogingu so tmmaa
where < L.
3: Identify the tokens with the top-/ largest probabilities in Ploging and assign the value to their indices and 0 to the remaining positions.
4: Set the indices of top-/ largest probabilities tokens in P Progitapu to e: P Plogisauuk +
Plail logits
5: for < L do
6: Compute Waphobal sp cos (lossii
7: Compute Velccal max cos_sim daiimmlogit (logit.s/mac logitsL, ptail logitsqua - Pa.), 0). Pes ET.
8: Calculate Wi,- (Wygloball W
[1,
9: end for
> Wi. by computing each i E [1,7]
10: Obtain the current latent distribution PNDAD
Et E/ Whs
across different layers.
new 11: Return: logits f, logits a VK VKL.LPNADD Plogitse)
17

---

Published as a conference paper at ICLR 2026
Table 10: Case study of Llama-7B-baa on the GSM8K Dataset.
Input:
Q: There are 15 trees in the grove. Grove workers will plant trees in the grove today. After they are done, there will be 21 trees. How many trees did the grove workers plant today? A: There
15 trees originally. Then there were 21 trees after some more were planted. So there must have been 21 15 = 6. The answer is 6.
Q: there are 3 cars in the parking lot and more cars arrive, how many cars are in the parking lot? A: There are originally 3 cars. 2 more cars arrive. + 2 = 5. The answer is
Q: Leah had 32 chocolates and her sister had 42. If they ate 35, how many pieces do they have left in total? A: Originally, Leah had 32 chocolates. Her sister had 42. So in total they had 32 42 = 74. After eating 35, they had 74 35 39, The answer is 39, Q: Jason had 20 lollipops. He gave Denny some lollipops. Now Jason has 12 lollipops. How many lollipops did Jason give Denny? ? A: Jason started with 20 lollipops. Then he had 12 after giving some to Denny. So he gave Denny 20 - 12 = 8. The answer is 8. Q: Shawn has five toys. For Christmas, he got
toys each from his mom and dad. How many toys does he have now? A: Shawn started with 5 toys. If he got 2 toys each from his mom and dad, then that is 4 more toys. 5 4 =9.
9. The answer is 9.
Q: There were nine computers in the server room. Five more computers were installed each day, from monday to thursday. How many computers are now in the server room? A: There were originally 9 computers. For each of 4 days, 5 more computers were added. So * = 20 computers were added. + 20 is 29. The answer 29.
Q: Michael had 58 golf balls. On tuesday, he lost 23 golf balls. On wednesday, he lost 2 more. How many golf balls did he have the end of wednesday? A: Michael started with 58 golf balls. After losing 23 on tuesday, he had 58 23 35. After losing more, he had 35 2 = 33 golf balls. The answer is 33.
Q: Olivia has 23. 23.Shcbought, fivebagel f for each. How much money does she have left? A: Olivia had 23 dollars. 5 bagels for 3 dollars each will be x = 15 dollars. So she has 23 15 dollars left. 23 15 is 8. The answer is 8.
Q: Marcell and Beatrice are having a contest to see who can eat the most fruit roll-ups, so they unroll many as they can find. Unfortunately, someone makes mistake and Beatrice's 's was two roll-ups wide and 24 rolls up long while Marcell's 's was 3 roll-ups wide and 14 roll-ups long. If they both ate their entire amount, how many did they cat on average?
Dreedy
Marcell's 's was 3 wide and 14 long. Beatrice's was 2 wide and 24 long. So Marcell's 's was 3 14 = 42 rolls. Beatrice's s was 2 24 = 48 rolls. So Marcell ate 42 rolls and Beatrice ate 48 rolls. 42 48 90. The answer 90.
SLED
Marcell's 's was 3 wide and 14 long. Beatrice's was 2 wide and 24 long. So Marcell's 's was 3 14 = 42 rolls. Beatrice's 's was 2 # 24 48 rolls. So Marcell ate 42 rolls and Beatrice ate 48 rolls. 42 48 = 90 rolls. The answer is 90.
NDAD
Marcell's was 3 roll-ups wide and 14 roll-ups long. Beatrice's was 2 roll-ups wide and 24 roll-ups long. So Marcell's was 3 * 14 42 roll-ups long. Beatrice's was 2 * 24 48 roll-ups long. So Marcell's was 42 roll-ups long and Beatrice's 's was 48 roll-ups long. So they both ate 42 48 90 roll-ups. 90 2 45. The answer is 45.
18


Reviews:
[{"review_id": "u6bVQTwImQ", "reviewer": "Reviewer_VbaU", "summary": "The paper proposes Negative-Direction Aware Decoding (NDAD), a training-free decoding strategy to reduce hallucinations in large language models. NDAD first intentionally disrupts a model’s most fact-relevant attention heads to induce a “hallucination-leaning” prediction, which reveals how the model would start to generate unreliable content. It then identifies the most dangerous parts of that signal — the token directions that are likely to grow into the final answer — and treats them as a negative direction. During actual decoding, NDAD pushes the model’s output distribution away from this negative direction via a small KL-based adjustment to the final logits. Across multiple benchmarks (factual QA, open-ended QA, and chain-of-thought math reasoning) and several popular LLMs, NDAD matches or outperforms prior decoding-time methods like SLED and Activation Decoding, without extra training data, external retrieval, or modifying model weights.", "questions": "Please refer to the weaknesses part.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The perspective is new: instead of “pulling the answer toward the correct truth,” NDAD “pushes the model away from being wrong.” It first induces a hallucination signal — the direction the model tends to move toward incorrect content after targeted corruption — and then explicitly repels the final output probabilities away from that direction. This reframes the problem from boosting truthfulness to actively repulsing hallucination, which is conceptually distinct.\n2. The method introduces a weighted mechanism that combines global consistency and local evolution, instead of bluntly suppressing all probabilities. After combining these two views, it applies normalization and squared weighting to emphasize high-confidence risky directions. This fine-grained control avoids the typical failure mode where aggressive down-weighting breaks fluency and makes the sentence fall apart.\n3. The approach shows consistent gains across multiple tasks, and is particularly strong on tasks that require reasoning and multi-step thinking.", "weaknesses": "1. Runtime comparison with baselines. he method requires extra computation at inference (masking heads, computing global/local weights, doing the gradient-style correction), but the paper does not systematically report the runtime or memory overhead versus standard decoding or simpler contrastive methods. \n2. Regarding the reliability and causality of the “critical attention head” selection process: Can the authors provide more fine-grained evidence, for example, whether the heads identified as critical tend to attend to high-confidence knowledge sources (such as factual spans in the context, entity mentions from the question, numerical cues, etc.) rather than merely contributing to fluency or syntax?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XyaTPrRtSc", "reviewer": "Reviewer_vTbm", "summary": "The paper introduces a training-free method to reduce hallucination named NDAD. The method aim to reduce LLM hallucinations, its core ideas is to steer output distributions away from hallucination-prone directions. First, the method masks critical attention heads (selected based on metrics such as importance scores and entropy) to generate a \"hallucination signal,\" then uses dynamic weighting (global weight for signal-logit alignment and local weight for low-probability token evolution) to control this signal's impact. Finally,  the output logits are modified by gradient descent to penalize the KL divergence from the negative distribution. The paper provides tests across multiple LLMs and different benchmarks, the results show that the method has similar results to baselines such as  DoLa, SLED.", "questions": "see weakness", "limitations": "", "rating": 2, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The method presents a unique and compelling perspective by \"subtracting negatives\"—inducing a hallucination signal and then repelling the model from it.\n\n2. The approach demonstrates broad applicability, validated across multiple model families (Llama, Mistral, Qwen) and sizes (7B, 13B, 8B) on a diverse set of tasks (TruthfulQA, GSM8K).\n\n3. Rigorous studies validate the core design choices, including the efficacy of the masking strategy and the necessity of both the global and local weighting components.", "weaknesses": "1. My biggest concern is that the method is not practical at all. The proposed method requires a second, modified forward pass at every decoding step. In the meanwhile, the gradient-descent step is problmetic as well. This will result in inference-time latency and definitely problemetic in many inference settings(may lead to KV-cache related issues). All make the method impractical.\n\n2. The motivation and calculation for the local weight (evolution trajectories, one-hot vectors) lack intuitive, step-by-step clarity\n\n3. The introduction of several new parameters combined with the observed performance curves suggests the method may be highly sensitive to tuning. It can be critical for real-world deployment\n\n4. The choice of using an arbitrary squared transformation for the final weight scores is not sufficiently justified. \n\n5. Another concern is that even with introduction of overhead and hyper-parameters, the performance gain is very small and even worse compared with existing method", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7JoHPBjqgE", "reviewer": "Reviewer_pT8A", "summary": "The following paper proposes a novel approach to hallucination reduction in Large Language Models (LLMs). It decerns a direction responsible for the hallucinations by masking out most influential attention heads responsible for the factuality. This direction is then subtracted from the original output logits of the model as a way to reduce the hallucinations in the final predictions. The paper showcases the effectiveness of the proposed method on a number of LLMs and datasets. It shows that proposed approach increases the accuracy on a number of benchmarks compared to existing hallucination reduction methods and studies the impact of the components driving the hallucinations.", "questions": "See weaknesses\nTable 1 shows that some of the baseline hallucination mitigation approaches worsen the accuracy. Why is that the case ? Isn't the purpose of those methods to increase the accuracy ?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "+ The paper is well written, the method and the experiments  is clearly explained\n+ The authors report results on a number of datasets and models - they show that proposed methods consistently outperform the existing baseline approaches.", "weaknesses": "**Discussion on limitations of the work**\n\nThe paper is well written but there are clearly some insights missing on the limitations of the proposed work. \n\n1) The paper relies heavily on an existing algorithm that identifies heads responsible for factuality but it is unclear if the heads responsible for the factually are consistently the same for all tasks and sample sizes and what's the accuracy in identifying such heads. It is unclear how the number of heads and layers are chosen. It is hard to tell from Figure 4 what number of heads and layers one should choose.\n\n2) It is also unclear how the results are aggregated for single token vs multi-token generation. When we say logits_L is this the logits for the final answer, a single token or aggregation of logits across multiple tokens ?\n\n3) It is also unclear how we set the threshold to select top I tokens ?\n\n\nDescriptions of the figures and tables. \n\nThe descriptions are quite vague and high level. It might be important to mention in the titles what are we exactly measuring e.g. Overall Accuracy or the Factuality Accuracy. Figure 1 is great but the description is very vague and it requires reading the rest of the paper to understand it. \n\nIt would we also interesting to describe the choice of combining global and local weights. Why do we use the product of W_global * W_local vs additive influence W_global + W_local", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KPfWddlsD7", "reviewer": "Reviewer_ehY7", "summary": "This paper introduces Negative-Direction Aware Decoding (NDAD), a raining-free decoding strategy designed to mitigate hallucinations in LLMs. The core contribution lies in its approach of actively identifying and leveraging \"hallucination signals\" as a repulsive force during generation. Rather than solely promoting factual content, NDAD isolates hallucination-prone directions by strategically masking influential attention heads. The method then employs a dynamic weighting mechanism, integrating both global consistency and local divergence measures, to controllably steer the model's output away from these identified negative trajectories via a gradient-descent adjustment. Experiments demonstrate that NDAD consistently enhances factual reliability across diverse LLMs and benchmark datasets, offering a lightweight yet potent solution to a critical challenge in model deployment.", "questions": "1. What is the computational overhead (e.g., latency increase per token) of NDAD compared to greedy decoding and the baseline?\n2. Have you evaluated whether NDAD negatively impacts other text qualities, such as fluency, coherence, or stylistic appropriateness?\n3. How do you expect NDAD's performance and computational cost to scale when applied to much larger models (e.g., 70B+ parameters)?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The method is training-free and operates at inference time, making it a practical and computationally accessible approach for improving factuality compared to methods requiring model fine-tuning\n- The dual-weighting scheme that considers both global consistency (alignment with layer activations) and local divergence (evolution of low-probability tokens) provides a non-trivial method for modulating the hallucination signal.\n- The paper provides ablation experiments that investigate the necessity of its core components, such as the global and local weights and the importance-guided masking strategy, strengthening the justification for its specific design choices.", "weaknesses": "- The performance improvements over the baseline are often marginal, questioning the method's practical significance.\n- The method introduces a number of new hyperparameters (number of masked heads, number of layers, top-I tokens, evolution rate α) that likely require careful tuning for different models and tasks, potentially limiting its out-of-the-box utility.\n- The paper does not analyze the computational overhead or potential impact on generation latency. Furthermore, evaluation is focused on factuality, with no assessment of whether the method harms other text quality aspects like coherence or fluency.\n- All models tested are in the smaller 7B-13B parameter range. The method's viability and computational cost on much larger models (e.g., 70B+) remain unproven.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)