Coarse-to-Fine Highlighting: Reducing Knowledge Hallucination in Large Language Models
ICML, 2024

Generation of plausible but incorrect factual information, often termed hallucination, has attracted significant research interest. Retrieval-augmented language model (RALM)—which enhances models with up-to-date knowledge—emerges as a promising method to reduce hallucination. However, existing RALMs may instead exacerbate hallucination when retrieving lengthy contexts. To address this challenge, we propose COFT, a novel <b>CO</b>arse-to-<b>F</b>ine highligh<b>T</b>ing method to focus on different granularity-level key texts, thereby avoiding getting lost in lengthy contexts. Specifically, COFT consists of three components: <em>recaller</em>, <em>scorer</em>, and <em>selector</em>. First, <em>recaller</em> applies a knowledge graph to extract potential key entities in a given context. Second, <em>scorer</em> measures the importance of each entity by calculating its contextual weight. Finally, <em>selector</em> selects high contextual weight entities with a dynamic threshold algorithm and highlights the corresponding paragraphs, sentences, or words in a coarse-to-fine manner. Extensive experiments on knowledge hallucination benchmark demonstrate the effectiveness of COFT, leading to a superior performance over 30% in F1 score metric. Moreover, COFT also exhibits remarkable versatility across various long-form tasks, such as reading comprehension and question answering.

Included in review: No