The Law of Knowledge Overshadowing: Towards Understanding, Predicting and Preventing LLM Hallucination (2025)

Abstract: Hallucination is a persistent challenge in large language models (LLMs), where even with rigorous quality control, models often generate distorted facts. This paradox, in which error generation continues despite high-quality training data, calls for a deeper understanding of the underlying LLM mechanisms. To address it, we propose a novel concept: knowledge overshadowing, where model’s dominant knowledge can obscure less prominent knowledge during text generation, causing the model to fabricate inaccurate details. Building on this idea, we introduce a novel framework to quantify factual hallucinations by modeling knowledge overshadowing. Central to our approach is the log-linear law, which predicts that the rate of factual hallucination increases linearly with the logarithmic scale of (1) Knowledge Popularity, (2) Knowledge Length, and (3) Model Size. The law provides a means to preemptively quantify hallucinations, offering foresight into their occurrence even before model training or inference. Built on overshadowing effect, we propose a new decoding strategy CoDa, to mitigate hallucinations, which notably enhance model factuality on Overshadow (27.9%), MemoTrap (13.1%) and NQ-Swap (18.3%). Our findings not only deepen understandings of the underlying mechanisms behind hallucinations but also provide actionable insights for developing more predictable and controllable language models.

## introduction large language models (llms) have revolutionized artificial intelligence, but their success is accompanied by a critical issue known as hallucination @xcite . hallucination refers to models generating unfaithful or nonfactual statements. in many applications, this issue undermines performance and reliability, posing substantial challenges to their practical deployment @xcite . some studies attribute hallucination to lowquality pretraining corpora @xcite . however, we find it persists even when the pretraining corpus is strictly controlled to contain only factual statements. specifically, when extracting knowledge using queries, we observe a tendency for certain knowledge to overshadow other relevant information. this causes the model to reason without adequately considering overshadowed knowledge, leading to hallucinations. as shown in figure 1 , when queried for "famous singer in north korea", the model incorrectly nominate "kim jong un", who is in fact a politician, as a result of "north korea" overshadowing "singer". this observation highlights how knowledge of varying forms interacts, distorting the reasoning process and causing the model to misassemble facts, thereby generating hallucinations. to investigate this phenomenon, we raise the following questions: through extensive experiments, we find that knowledge overshadowing broadly induces factual hallucinations in both pretrained and fine-tuned models, across diverse model families and sizes. despite its importance, the factors influencing this phenomenon remain unexplored. to bridge this gap, we analyze knowledge representation from both global and local perspectives by examining its popularity across the dataset distribution and its proportional representation length within individual sentences. additionally, since increasing model size has been shown to improve language model performance @xcite , we further explore its impact on factual hallucinations. to examine the impact of these factors, we pretrain llms from scratch on a synthetic dataset with strictly controlled quality. our empirical findings reveal a log-linear scaling law for factual hallucinations, showing that hallucination rates increase linearly with the logarithmic scale of relative knowledge popularity, knowledge length, and model size. finetuning on diverse tasks further confirms this law applies to finetuned llms, enabling the preemptive quantification of hallucinations before model training or inference. this not only bridges the gap in understanding hallucinations emerging from factual training data but also introduces a principled approach for evaluating training data and predicting model behavior in advance. the empirical discovery of this law leads us to investigate its underlying cause. we hypothesize that knowledge overshadowing stems from the overgeneralization of popular knowledge, suppressing less popular counterparts. theoretically, we derive a generalization bound for auto-regressive language modeling, linking the model's behavior to key properties of its training data. our analysis shows that generalization improves with increasing relative knowledge popularity and length, mirroring the trend observed in hallucination rates. building on all the insights derived, we propose contrastive decoding to amplify overshadowed knowledge (coda), a method designed to amplify the influence of overshadowed knowledge while mitigating biases from dominant knowledge. first, we identify overshadowed knowledge by computing the mutual information between the next-token probability distributions of the original and modified prompts, where specific tokens are masked. this approach reveals knowledge encoded in the masked tokens, which is often overlooked and prone to hallucination. we then employ contrastive decoding to reduce the bias introduced by dominant knowledge. without requiring additional training, coda significantly improves factuality, achieving gains of 13.1%, 18.3%, and 27.9% on the memo-trap, nq-swap, and overshadowing datasets, respectively. our contributions are three-fold: • we are the first to identify knowledge overshadowing as a key driver of hallucinations and demonstrate its prevalence across llms. ## when to expect factual hallucination? to determine the conditions under which factual hallucinations emerge, we investigate knowledge overshadowing across various experimental setups, including probing an open-source pretrained llm without training, pretraining an llm from scratch, fine-tuning a pretrained llm on downstream tasks. ## why knowledge overshadows? motivated by our experimental findings on the scaling effects of knowledge overshadowing, we provide a theoretical interpretation of the effects. ## discussion for broader social impact our work contributes to building more predictable and reliable ai systems by interpreting hallucinations through knowledge overshadowing and introducing the coda method to rebalance information during decoding. this improves the factuality of ai-generated content and enhances transparency in llms. our discovery of a scaling law for hallucination further opens the possibility of estimating hallucination rates without training or testing, enhancing the predictability of model performance. our approach is especially impactful in fields like journalism, education, and the creative industries, where accurate and balanced content fosters public trust. moreover, by mitigating the dominance of popular narratives, our work helps amplify underrepresented voices, promoting cultural diversity, inclusivity, and responsible ai deployment. ## conclusion our work identify knowledge overshadowing as a contributional cause of llms hallucination, where dominant knowledge suppresses less frequent facts, leading to fact distortions. we introduce the loglinear scaling law, which reveals that hallucination rates grow predictably with knowledge popularity, length, and model size, enabling hallucination prediction. built on overshadowing effect, we propose coda, a decoding strategy that improves factual accuracy without retraining. our approach provides a principled way to understand and control hallucinations, leading to more reliable llms.