D e LT a: A Decoding Strategy based on Logit Trajectory Prediction Improves Factuality and Reasoning Ability (2025)

Abstract: Large Language Models (LLMs) are increasingly being used in real-world applications. However, concerns about the reliability of the content they generate persist, as it frequently deviates from factual correctness or exhibits deficiencies in logical reasoning. This paper proposes a novel decoding strategy aimed at enhancing both factual accuracy and inferential reasoning without requiring any modifications to the architecture or pre-trained parameters of LLMs. Our approach adjusts next-token probabilities by analyzing the trajectory of logits from lower to higher layers in Transformers and applying linear regression. We find that this Decoding by Logit Trajectory-based approach (DeLTa) effectively reinforces factuality and reasoning while mitigating incorrect generation. Experiments on TruthfulQA demonstrate that DeLTa attains up to a 4.9% improvement over the baseline. Furthermore, it enhances performance by up to 8.1% on StrategyQA and 7.3% on GSM8K, both of which demand strong reasoning capabilities.

## introduction natural language processing has advanced significantly with the rise of large language models (llms) @xcite . however, ensuring the factual accuracy of llmgenerated text remains challenging. a notable issue is hallucination, where models produce factually incorrect content, posing risks in fields like information retrieval, medicine, and law @xcite . calculation errors in the logical reasoning further contribute to inaccuracies, stemming from incorrect token predictions during decoding. mitigation strategies for these issues include the selection of dataset, modifications to loss functions @xcite , and the incorporation layer llama-3.1-8b (32 layer) 30th layer 31st layer 32nd layer 33rd layer logit (olympia) 30 31 32 33 ... ... olympia where is the capital of washington state? ... predicted logit original logit se at tle o lym pi a regression line se at tle o lym pi a se at tle o lym pi a se at tle o lym pi a figure 1: overview of delta. when input tokens are fed into the llm, the logits from each layer (e.g., layers 30, 31, and 32) are computed and shown as bar graphs to illustrate changes between tokens (e.g., "seattle" vs. "olympia"). a linear regression (red line) approximates the logit trajectory (blue dots). using this regression, we extrapolate the logits for a virtual 33rd layer (red dot) and improve prediction beyond the original outputs. to overcome these limitations, we propose a decoding strategy, that boosts generation accuracy without extra training or data. inspired by @xcite , who observed that correct token probabilities tend to rise in higher transformer layers, we introduce decoding by logit trajectory-based approach (delta), which treats each layer's logits as a time-series and use linear regression to predict upper-layer logits (figure 1 ). experiments demonstrate that delta enhances factuality by up to 4.9% on truthfulqa, 5.0% on triviaqa, and 2.4% on natural questions, while also improving reasoning on strategyqa and gsm8k by up to 8.1%. these findings confirm that delta refines token prediction, leading to improved factuality and reasoning capabilities. ## related work previous research on guiding llms to generate factually accurate text can be broadly categorized into training-based @xcite and non-training-based approaches. delta falls into the latter category. among non-training-based methods, @xcite introduced asymptotic probability decoding, which extrapolates output probabilities from llms of different sizes under contrastive decoding (cd) @xcite . @xcite showed that certain capabilities of transformers are concentrated in the latter layers and achieved improvements in sentiment classification by applying linear extrapolation to a text classifier based on cd. these approaches estimate probabilities using linear regression or extrapolation, relying on only two data points (e.g., the outputs of two models or an intermediate layer and the final layer). in contrast, delta predicts logits instead of probabilities and performs regression across the entire sequence of logits from intermediate layers to the final layer, then recalculates the probability values. another non-training-based approach, dola @xcite , uses the difference in log probabilities between a lower and higher layer of the model within cd to encourage factually based text generation. ## method delta aims to improve the probability of generating the correct token during decoding by focusing on changes in the logits across the transformer's layers. specifically, when decoding the next token in an n -layer transformer, we regard the logits produced by each layer as a time series ( § 3.1). based on the observation that higher layers generally assign higher probabilities to the correct token @xcite , we employ a simple linear regression model ( §3.2) to predict the logits of the higher layers. by leveraging the upward trend from lower to higher layers, this approach enhances the final prediction performance. ## analysis in this section, we conduct a series of analyses to empirically validate the core hypotheses underpinning delta. we first verify that deeper layers contribute more to task performance, then investigate the linearity of logit evolution across layers, and finally justify our choice of a linear regression model through an ablation study. ## conclusion this study aimed to enhance the factual accuracy and reasoning of text generated by llms. the proposed method, delta, operates without additional training or data. by leveraging token probability distributions across transformer layers and employing linear regression, we developed a framework that is both computationally efficient and easily integrable. empirical evaluations across multiple benchmarks demonstrate that delta significantly improves factual accuracy and exhibits effectiveness in reasoning tasks.