A da CAD : Adaptively Decoding to Balance Conflicts between Contextual and Parametric Knowledge (2025)

Abstract: Knowledge conflict arises from discrepancies between information in the context of a large language model (LLM) and the knowledge stored in its parameters. This can hurt performance when using standard decoding techniques, which tend to ignore the context. Existing test-time contrastive methods seek to address this by comparing the LLM’s output distribution with and without the context and adjust the model according to the contrast between them. However, we find that these methods frequently misjudge the degree of conflict and struggle to handle instances that vary in their amount of conflict, with static methods over-adjusting when conflict is absent. We propose a fine-grained, instance-level approach called AdaCAD, which dynamically infers the weight of adjustment based on the degree of conflict, as measured by the Jensen-Shannon divergence between distributions representing contextual and parametric knowledge. Across four LLMs, six question-answering (QA) and three summarization datasets, we demonstrate that AdaCAD consistently outperforms other decoding baselines with average QA accuracy gains of 14.21% (absolute) over a static contrastive baseline, and improves the factuality of summaries by 6.19 (AlignScore). Lastly, we show that while contrastive baselines hurt performance when conflict is absent, AdaCAD mitigates these losses, making it more applicable to real-world datasets in which some examples have conflict and others do not.

## introduction large language models (llms) encode vast amounts of information from pretraining in their parameters @xcite , giving them remarkable capabilities in knowledgeintensive nlp tasks. however, llms also hallucinate plausible but factually incorrect responses 1 our code is publicly available at: https://github.com/ hannight/adacad decoding fails to attend to the context, resulting in incorrect answers. contrastive decoding allows the context to be incorporated, but must be done with care: in lowconflict cases, excessive contrast can over-correct (e.g., by cad with α = 1), resulting in incorrect outputs. adacad dynamically adjusts the degree of contrast, allowing it to handle both high and low-conflict cases. in this work, we present a simple and effective dynamic decoding method, adaptive context aware decoding (adacad) , aimed at automatically modeling the degree of conflict between the context and parametric knowledge and dynamically inferring the degree of adjustment needed for every token. we use the jensen-shannon divergence (jsd) between output distributions with and without the context to measure the degree of knowledge conflict, using the resulting value to reweight the combination of distributions. a higher jsd indicates a greater degree of conflict and signals the need for higher adjustment (more weight on the contextual knowledge) while a lower jsd reflects a smaller degree of conflict requiring a smaller adjustment (more weight on the parametric knowledge). as illustrated in fig. 1 , this leads to correct answers both for high and low-conflict examples by helping the model adaptively decide how to weigh contextual vs. parametric knowledge. we demonstrate adacad's effectiveness on a diverse range of tasks, covering questionanswering (qa) and summarization, with six qa datasets (natural question (nq; @xcite , nq-swap @xcite , triviaqa @xcite , popqa @xcite , hotpotqa @xcite , tabmwp @xcite ) and three summarization datasets (cnn-dm @xcite , xsum @xcite , and tofueval @xcite ). we test a range of base llms, examining llama-2 @xcite , llama-3 (ai@meta, 2024), and mistral @xcite . we consider different sizes of these models and also test both the base and instruction-tuned variants. our results and analyses show that decoding with a uniform level of contrast benefits high-conflict scenarios but generally hurts performance, while the adaptive contrast of adacad results in improvements across the board. overall, adacad generally achieves superior performance compared to the baselines, with an absolute gain of 14.21% over cad (a static baseline), 4.82% over coiecd @xcite baseline that classifies instances as conflicting or not), 5.86% over confcd @xcite a method that makes dynamic token-level adjustments based on llm confidence), and 2.41% over greedy decoding when averaged across models and qa datasets. on summarization, adacad improves summary quality and factuality, with an average alignscore @xcite gain of 4.16 over greedy decoding, 2.19 over cad, 10.44 over coiecd, and 7.96 over confcd. furthermore, in our analyses, we explore why adacad improves over the baselines. we first validate the hypothesis that adacad is able to balance contextual and parametric knowledge by assigning lower weights to lower-conflict instances, testing each method on datasets designed to have high and low conflict, finding that adacad's inferred weight is much lower when there is no conflict. we also compare the amount by which cad and adacad adjust the base model's distribution on examples with and without conflict, finding that while adacad changes the distribution less when there is no conflict (i.e., when the base model's distribution is already sufficient), cad adjusts by roughly the same amount whether there is conflict or not, explaining its lower qa performance. additionally, for summarization tasks, adacad generates more faithful summaries whereas other methods tend to hallucinate details. contrast in text generation contrastive approaches for text generation have been widely studied and used to enhance response diversity in conversations @xcite , steering model generations towards desired attributes while maintaining fluency and diversity @xcite , and contrasting between larger and relatively smaller language models to generate high-quality text @xcite , and improve visually-grounded generation tasks @xcite . context-aware decoding (cad; @xcite ) leverages a contrastive output distribution that amplifies the differences between the output probabilities predicted by a model with and without the context, promoting greater attention to the input context for more faithful and reliable text generation. unlike adacad, these past contrastive approaches do not adapt the weight on distributions to varying degrees of knowledge conflict. to address this, @xcite introduce coiecd, a decoding-time method that categorizes instances into two discrete bins -high and low conflict -based on a complex informationentropy constraint governed by tuned hyperparameters, and employs different decoding strategies (by altering cad) for each. @xcite uses llm confidence to adjust the output probabilities dynamically (denoted as confcd) as well as relies on additional noisy and irrelevant contexts. in contrast, adacad employs a single dynamic instance-level strategy that automatically models (based on jensen-shannon divergence) a continuous degree of conflict without imposing rigid cate-gories or requiring additional noisy and irrelevant contexts, accommodating more general knowledge conflict settings. in addition to these conceptual differences, in section 4.2, we show that adacad outperforms cad, coiecd, and confcd on qa and summarization. ## methodology task and notation given an input query x with a relevant context c, a language model parameterized by θ is tasked with generating a correct response @xmath0, . . . , y n of length n that respects the context. at each decoding step t, a token y t can be sampled autoregressively from a probability distribution conditioned on query x and context c as y ∼ p θ (y | c, x, @xmath1. however, when there is conflict between knowledge in the context c and parametric knowledge encoded in llm, the model can struggle to pay enough attention to c and overly rely on the parametric knowledge @xcite , i.e., sample from a distribution more akin to p θ (y|x, @xmath2. background: context-aware decoding to mitigate knowledge conflicts, @xcite introduce context-aware decoding (cad), which samples from a contrastive output distribution that amplifies the difference between output probabilities with and without context. cad measures the parametric knowledge via p θ (y|x, y t ) and prioritizes relevant contextual knowledge over the model's parametric knowledge by using the pointwise mutual information (pmi) between the context c and the generation y, conditioned on x, @xmath3. ## discussion and conclusion in naturalistic scenarios with mixed datasets containing examples with and without knowledge conflicts, existing decoding methods, including cad, fail to adapt to changing amounts of conflict and in fact can lead to reduced performance. although larger and more performant models can store more information in their parametric knowledge -thus leading to less and less conflict as models improve -there will still always be gaps between the model and the actual state of the world (e.g., because of time cutoffs). this means that models will encounter both low-and high-conflict scenarios, no matter their strength. to this end, we introduce adacad, a simple yet effective dynamic decoding method that uses jensen-shannon divergence to dynamically model the degree of conflict for a given example (and timestep) and automatically balance the con-trast between contextual and parametric knowledge. on diverse qa datasets, we show that adacad combines the best of greedy decoding and contextaware decoding, improving performance. additionally, experiments on summarization demonstrate that adacad enhances both the quality and factuality of generated text, while other methods tend to hallucinate details. lastly, adacad consistently outperforms coiecd, another hybrid decoding strategies that detects conflict. our analysis reveals that adacad mitigates the overcorrection seen in cad by dynamically adjusting the weight of contextual knowledge based on the degree of conflict.