Trusting Your Evidence: Hallucinate Less with Context-aware Decoding (2024)

Abstract: Language models (LMs) often struggle to pay enough attention to the input context, and generate texts that are unfaithful or contain hallucinations. To mitigate this issue, we present context-aware decoding (CAD), which follows a contrastive output distribution that amplifies the difference between the output probabilities when a model is used with and without context. Our experiments show that CAD, without additional training, significantly improves the faithfulness of different LM families, including OPT, GPT, LLaMA, and FLAN-T5 for summarization tasks (e.g., 14.3% gain for LLaMA in factuality metrics). Furthermore, CAD is particularly effective in overriding a model’s prior knowledge when it contradicts the provided context, leading to substantial improvements in tasks where resolving the knowledge conflict is essential. Our code is publicly released at https://github.com/xhan77/context-aware-decoding.

## introduction language models (lms) are effective in generating fluent continuations of a prompt or document prefix. during generation, they rely on two sources of knowledge: (1) prior knowledge, which is learned during pretraining and stored implicitly within the model parameters; (2) context knowledge, which is passed as inputs in the prefix context @xcite . however, it remains an open question how a pretrained lm, particularly a vanilla lm without task-specific finetuning, balances these two knowledge sources during generation. previous research shows that lms can fail to pay enough attention to new information introduced in the context knowledge. this can lead to hallucination in summarization @xcite , where the generated summaries include facts not present in the input doc- * equal contribution. order randomly determined. ument. insufficient attention to context is especially problematic when the context knowledge contradicts with the prior knowledge @xcite . for instance, when llama @xcite is presented with a latest document "argentina won the fifa world cups in 1978, 1986 and 2022 ..." in its context (figure 1 ), it still predicts "two" in response to the question "how many world cups have argentina won?", due in part to the outdated training data. in this work, we present a simple context-aware decoding (cad) method to encourage the lm to attend to its context during generation. as shown in figure 1 , cad samples from a new output distribution, which amplifies the difference between output probabilities with and without the context document. this provides a new form of contrastive decoding @xcite , which effectively downweights the prior knowledge when more relevant contextual information is provided. cad can be used with off-the-shelf pretrained language models without any additional training. experimental results from summarization tasks show that context-aware decoding significantly enhances the generation faithfulness of vanilla lms including opt @xcite , gpt-neo @xcite , llama @xcite and instruction-finetuned lms such as flan @xcite . for instance, when applied to llama-30b in cnn-dm, cad leads to substantial improvement in both rouge-l (21%) and factuality evaluation metrics (14.3%). more notably, cad is especially beneficial for knowledge conflicting tasks, where the context contains information contradictory to the model's prior knowledge. cad brings a 2.9x improvement to llama-30b on a knowledge conflicts qa dataset @xcite . furthermore, we observe that this gain brought by cad increases as the model size grows in knowledge conflicts tasks. these results demonstrate the potential of cad in mitigating hallucinations in text generation and overriding prior knowledge with reliable and trusted information. ## experimental setup we perform evaluation on tasks that require lms to read and reason over contexts and produce outputs that are faithful to the contexts. following prior work @xcite , we evaluate the models using prompting. ## results summarization ## conclusion language models suffer from an insufficient attention to the given context compared to its prior knowledge, leading to an unfaithful generation to the input context. we present cad, a simple inference-time method that downweights an output probability associated with the model's prior knowledge to promote models' attention to the context. we experiment on two families of tasks that require a strong attention to the context and show that cad provides more faithful outputs across different language models of various sizes.