Monitoring Decoding: Mitigating Hallucination via Evaluating the Factuality of Partial Response during Generation (2025)

Abstract: While large language models have demonstrated exceptional performance across a wide range of tasks, they remain susceptible to hallucinations – generating plausible yet factually incorrect contents. Existing methods to mitigating such risk often rely on sampling multiple full-length generations, which introduces significant response latency and becomes ineffective when the model consistently produces hallucinated outputs with high confidence. To address these limitations, we introduce Monitoring Decoding (MD), a novel framework that dynamically monitors the generation process and selectively applies in-process interventions, focusing on revising crucial tokens responsible for hallucinations. Instead of waiting until completion of multiple full-length generations, we identify hallucination-prone tokens during generation using a monitor function, and further refine these tokens through a tree-based decoding strategy. This approach ensures an enhanced factual accuracy and coherence in the generated output while maintaining efficiency. Experimental results demonstrate that MD consistently outperforms self-consistency-based approaches in both effectiveness and efficiency, achieving higher factual accuracy while significantly reducing computational overhead.

## introduction large language models (llms), such as gpt-4 @xcite and llama @xcite , have achieved extraordinary success across various tasks, including question answering, summarization, and reasoning. despite their impressive capabilities, even state-of-the-art models are known to generate non-factual responses that deviate from verifiable real-world facts @xcite @xcite . these hallucinations pose significant risks, potentially undermining the practical utility of llms and diminishing user trust. consequently, mitigating hallucinations -ensuring that generated responses remain factually accurate and grounded in realworld knowledge -has emerged as an increasingly critical research focus. one common approach to reduce hallucinations is based on the best-of-n (bon) strategy @xcite @xcite , which involves generating multiple responses for a given prompt and carefully selecting or integrating the most reliable output. @xcite have shown that model performance scales nearly log-linearly with the number of samples, highlighting the effectiveness of sampling-based strategies in enhancing output quality. several other methods share this insight by prompting llms to merge or revise generated samples @xcite , or by measuring self-consistency @xcite @xcite to select the optimal response. however, recent studies highlight limitations in these approaches: simple merging or refinement strategies can still introduce factual inaccuracies @xcite , and self-consistency alone does not ensure factual correctness @xcite . to understand why these methods may fail, it is important to note that self-consistent responses can still contain hallucinations if the model exhibits over-confidence in incorrect tokens. as illustrated in figure 1 , the most consistent response among sampled outputs may include a hallucinated token (e.g., "24") with an extremely high probability. such high-confidence errors can be particularly persistent, as generated responses often correlate strongly @xcite , and the marginal benefits from sampling additional responses diminish over time @xcite . this observation leads to a fundamental question: is it necessary to resample multiple full-length and highly similar responses to improve factuality? interestingly, simply intervening the generation by replacing the hallucinated token "24" with "it" can already transform the response into a factually accurate one. this insight reveals that only a small subset of critical tokens during generation contribute to hallucinations, implying that targeted revision could effectively address these issues without the need for resampling entire responses. detecting these hallucination-inducing tokens is challenging due to their variability in length and positions. the model's over-confidence characteristics on these tokens also render self-assessmentbased strategies ineffective to correct these tokens. to solve this, we introduce a monitor function that acts as a supervisory component to assess the factual consistency of intermediate tokens during partial decoding, flagging and rejecting those with hallucination risk. meanwhile, we aim to maintain the benefits of the bon strategy -exploring the generation space to mitigate hallucinations caused by certain tokens with lower generation probabilities -while reducing unnecessary computations for generating redundant tokens. to achieve this, we employ a tree-based decoding strategy that selectively resamples and revises only the critical tokens flagged by our monitor function. this allows us to search for more potentially factual tokens within a smaller context window, effectively reducing hallucinations with improved sampling efficiency. in summary, we propose monitoring decoding (md), a framework that continuously monitors and revises tokens during generation. instead of resampling large portions of text, md specifically focuses on a few tokens that are most likely to be incorrect. by identifying and refining these critical tokens in process of generation, md guides the model toward more factually accurate outputs, and the use of tree search lower computational overhead while maintaining a high accuracy rate. our main contribution can be summarized as follows: ## methodology in this section, we introduce monitoring decoding (md), a novel framework designed to monitor the factuality of partially generated responses, identifying and correcting hallucination-prone tokens during the generation process. specifically, our approach consists of (1) an in-process detection mechanism which assesses the risk of generated tokens via a monitor function and rejects suspicious ones during generation, and (2) a tree-based revision mechanism which intervenes the generation by revising the flagged tokens with resampled ones through a tree-based decoding algorithm. we begin by outlining the general framework of the algorithm, followed by a detailed discussion on the detection and revision mechanisms. ## experiments this section evaluates the effectiveness and efficiency of our framework md in hallucination mitigation. specifically, we conduct the experiments to answer the following questions: we conduct a quantitative analysis on the factuality of the final responses generated by our framework, comparing its performance against other sampling-based and decoding methods. ## conclusions in this study, we propose md, a novel framework for mitigating hallucinations by detecting hallucination-prone tokens which could be hard to recognize by the model itself due to its overconfidence issue. by revising these critical tokens, we can significantly enhance the factual accuracy of the output. to achieve this, we employ a monitor function to identify problematic tokens from the partially generated output, enabling real-time monitoring of the generation process. once these crucial tokens are detected, we implement a treesearch algorithm to efficiently and effectively explore the token space for optimal replacements. experimental results on both question-answering and reasoning tasks demonstrate that our targeted intervention approach is both efficient and effective in mitigating hallucinations, outperforming traditional methods that rely on repeatedly sampling multiple full-length responses.