SH 2: Self-Highlighted Hesitation Helps You Decode More Truthfully (2024)

Abstract: Large language models (LLMs) demonstrate great performance in text generation. However, LLMs are still suffering from hallucinations. In this work, we propose an inference-time method, Self-Highlighted Hesitation (SH2), to help LLMs decode more truthfully. SH2 is based on a simple fact rooted in information theory that for an LLM, the tokens predicted with lower probabilities are prone to be more informative than others. Our analysis shows that these low-confidence tokens are more likely to be closely related to factual information, such as nouns, proper nouns, and adjectives. Therefore, we propose to ”highlight” the factual information by selecting key tokens with the lowest probabilities and concatenating them to the original context, thus forcing the model to repeatedly read and hesitate on these tokens before generation. During decoding, we also adopt contrastive decoding to emphasize the difference in output probabilities brought by the hesitation. Experimental results demonstrate that our SH2, requiring no additional data or models, can effectively help LLMs elicit factual knowledge and distinguish hallucinated contexts by themselves. Significant and consistent improvements are achieved by SH2 for LLaMA-7b, LLaMA2-7b and Mistral-7b on various hallucination tasks.

## introduction depending on massive training corpora, large language models (llms) have made tremendous progress in natural language understanding and text generation @xcite @xcite . however, during reasoning and generation, llms could suffer from hallucinations and generate non-factual answers @xcite . to clear these falsehoods, some researchers construct datasets with higher quality and train llms to respond in the correct form @xcite . but in the domain not included in the training, llms assign similar probabilities to correct and wrong choices since they do not have enough relevant knowledge to distinguish them. to fill the gap of knowledge, retrieval augmentation methods @xcite @xcite leverage external knowledge bases and tools to correct the output of llms. although they provide additional information for llms, it still seems impossible to guarantee zero error in the external knowledge. other researchers propose decoding reformulation methods @xcite @xcite to address hallucinations. however, most of them rely on external data with human labor or larger models to rescale probability distribution during decoding. recent works @xcite have demonstrated that a few more computation steps for llms can make a difference. @xcite propose chain-of-thought (cot) prompting to elicit intermediate reasoning steps of llms. these intermediate steps can lead to better answers. however, elaborate prompts need to be prepared in advance. @xcite append learnable pause tokens to the input prefix and train models from scratch. these delays introduced by pause tokens provide the model with more computation steps to generate better answers. in this paper, we seek to leverage the llm itself to highlight informative tokens and digest them as input during the hesitation steps to elicit truthful knowledge inside llms. we propose a simple yet effective method, self-highlighted hesitation (sh2), to help llms decode more truthfully. sh2 introduces hesitations to give llms more time to understand contexts and answer questions. for llms, the tokens assigned with lower probabili-ties are harder to predict, while more likely to be informative. llms can select these key tokens by themselves from the input and hesitate on these highlighted tokens. we calculate the difference brought by highlighted tokens through contrastive decoding @xcite and integrate it into the output probability. experiments on multiple tasks demonstrate that such a difference could elicit factual knowledge inside the model and successfully mitigate hallucinations. unlike other methods, our method does not leverage any other external tools or data. additionally, it can be directly deployed during inference with no more training. ## self-highlighted hesitation the illustration of our self-highlighted hesitation is shown in figure 1 . for the original inference procedure, the instruction, document, and summary are directly fed into the llm to generate the judgment. the llm could be easily confused by the hallucinated context. for our sh2, we underline the key tokens of the input document by the prediction likelihood. these tokens are listed as a hesitation and appended to the document. the prediction probability is scaled with the difference of confidence between the hesitated input x' and the original input x. our method can effectively help llms identify the hallucinated context. we will elaborate our method in the following subsections. ## experiments we conduct experiments on five tracks of three hallucination evaluation benchmarks. ## analysis we conduct further studies regarding choices of highlighted tokens, manners of hesitations, and the effect of contrastive decoding in this section. ## conclusion in this paper, we delve into the challenge of large language models when capturing important factual information. to address this, we introduce a novel inference-time method, sh2. our method proposes to highlight the key tokens that are hard for llms to predict and construct hesitations with these informative tokens. by reformulating the decoding procedure with the probability differences brought by hesitations, we enable llms to discern factual content more effectively. through extensive ex-periments and analysis across multiple tasks, sh2 demonstrates a significant enhancement in the truthfulness of llms, all achieved without reliance on external data or models.