Venue: NAACL 2024

# Trusting Your Evidence:

Hallucinate Less with Context-aware Decoding

Weijia Shi\({}^{1,2}\) +
Footnote †: Equal contribution. Order randomly determined.

Xiaochuang Han\({}^{1,2}\)

**Mike Lewis\({}^{2}\)****Yulia Tsvetkov\({}^{1}\)****Luke Zettlemoyer\({}^{1,2}\)****Wen-tau Yih\({}^{2}\)**

###### Abstract

Language models (LMs) often struggle to pay enough attention to the input context, and generate texts that are unfaithful or contain hallucinations. To mitigate this issue, we present context-aware decoding (CAD), which follows a contrastive output distribution that amplifies the difference between the output probabilities when a model is used with and without context. Our experiments show that CAD, without additional training, significantly improves the faithfulness of different LM families, including OPT, GPT, LLaMA and FLANT5 for summarization tasks (e.g., 14.3% gain for LLaMA in factuality metrics). Furthermore, CAD is particularly effective in overriding a model's prior knowledge when it contradicts the provided context, leading to substantial improvements in tasks where resolving the knowledge conflict is essential. Our code is publicly released at [https://github.com/xhan77/context-aware-decoding](https://github.com/xhan77/context-aware-decoding).

## 1 Introduction

Language models (LMs) are effective in generating fluent continuations of a prompt or document prefix. During generation, they rely on two sources of knowledge: (1) _prior knowledge_, which is learned during pretraining and stored implicitly within the model parameters; (2) _context knowledge_, which is passed as inputs in the prefix context Chan et al. (2022). However, it remains an open question how a pretrained LM, particularly a vanilla LM without task-specific finetuning, balances these two knowledge sources during generation.

Previous research shows that LMs can fail to pay enough attention to new information introduced in the context knowledge. This can lead to hallucination in summarization Maynez et al. (2020); Pagnoni et al. (2021), where the generated summaries include facts not present in the input document. Insufficient attention to context is especially problematic when the context knowledge contradicts with the prior knowledge Longpre et al. (2021); Zhou et al. (2023). For instance, when LLaMA Touvron et al. (2023) is presented with a latest document "Argentina won the FIFA World Cups in 1978, 1986 and 2022 ..." in its context (Figure 1), it still predicts "Two" in response to the question "How many World Cups have Argentina won?", due in part to the outdated training data.

In this work, we present a simple context-aware dccoding (CAD) method to encourage the LM to attend to its context during generation. As shown in Figure 1, CAD samples from a new output distribution, which amplifies the difference between output probabilities with and without the context document. This provides a new form of contrastive decoding Li et al. (2023), which effectively downweights the prior knowledge when more relevant contextual information is provided. CAD can be used with off-the-shelf pretrained language models without any additional training.

Experimental results from summarization tasks show that context-aware decoding significantly enhances the generation faithfulness of vanilla LMs including OPT Zhang et al. (2022), GPTNeo Black et al. (2021), LLaMA Touvron et al. (2023) and instruction-finetuned LMs such as FLAN Chung et al. (2022). For instance, when applied to LLaMA-3DB in CNN-DM, CAD leads to

Figure 1: An illustration of context-aware decoding.



substantial improvement in both ROUGE-L (21%) and factuality evaluation metrics (14.3%). More notably, CAD is especially beneficial for knowledge conflicting tasks, where the context contains information contradictory to the model's prior knowledge. CAD brings a 2.9x improvement to LLaMA-30B on a knowledge conflicts QA dataset Longpre et al. (2021). Furthermore, we observe that this gain brought by CAD increases as the model size grows in knowledge conflicts tasks. These results demonstrate the potential of CAD in mitigating hallucinations in text generation and overriding prior knowledge with reliable and trusted information.

## 2 Method

### Background

Given a LM \(\theta\), an input query \({\mathbf{x}}\), and a context \({\mathbf{c}}\) that contains some external knowledge _unfamiliar_ or _in conflict_ to the model's prior knowledge, we ask our model \(\theta\) to generate a response \({\mathbf{y}}\) given the the query and context. The response can be directly sampled (autoregressively) from the probability distribution conditioned on query \({\mathbf{x}}\) and context \({\mathbf{c}}\):

\[y_{t}\sim p_{\theta}(y_{t}~{}|~{}{\mathbf{c}},{\mathbf{x}},{\mathbf{y}}_{ct})\] \[\propto\exp\mathrm{logit}_{\theta}(y_{t}~{}|~{}{\mathbf{c}},{\mathbf{x}} ,{\mathbf{y}}_{<t})\]

However, in cases where the context \({\mathbf{c}}\) contains knowledge that is out-of-distribution with respect to \(\theta\), we hypothesize that the model can struggle to effectively attend to \({\mathbf{c}}\) and overly rely on the prior knowledge encoded in \(\theta\). For instance, as illustrated in Figure 1, when the context \({\mathbf{c}}\) states "Argentina won the FIFA World Cups in 1978, 1986 and 2022 ...", it contradicts the LM's outdated prior knowledge that Argentina has won the World Cup twice. The language model may still incorrectly predict "Two" even when presented with the context \({\mathbf{c}}\) and the query \({\mathbf{x}}\).

### Context-aware Decoding

To mitigate such issues, we factor out the prior knowledge from the model's original output distribution contrastively. Here, we model the prior knowledge as \(p_{\theta}(y_{t}~{}|~{}{\mathbf{x}},{\mathbf{y}}_{<t})\) and adjust the model's original output probability distribution using the pointwise mutual information (PMI) between the context \({\mathbf{c}}\) and the generation \(y_{t}\), conditioned on \({\mathbf{x}},{\mathbf{y}}_{<t}\). Formally, we have:

\[y_{t}\sim\tilde{p}_{\theta}(y_{t}~{}|~{}{\mathbf{c}},{\mathbf{x}},{\mathbf{ y}}_{<t})\\ \propto p_{\theta}(y_{t}~{}|~{}{\mathbf{c}},{\mathbf{x}},{\mathbf{y}}_{<t}) \bigg{(}\frac{p_{\theta}(y_{t}~{}|~{}{\mathbf{c}},{\mathbf{x}},{\mathbf{y}}_{<t})}{p_{ \theta}(y_{t}~{}|~{}{\mathbf{x}},{\mathbf{y}}_{<t})}\bigg{)}^{\alpha}\]

where the output probability is a product-of-experts of the original output probability and PMI weighted by \(\alpha\). Essentially, outputs that become much more likely when the context is included are preferred (Figure 1).

This expression is not a valid probability distribution and needs to be normalized across all possible values of \(y_{t}\). By rearranging the terms, we obtain the final form:

\[y_{t}\sim\mathrm{softmax}[(1+\alpha)\,\mathrm{logit}_{\theta}(y_{ t}~{}|~{}{\mathbf{c}},{\mathbf{x}},{\mathbf{y}}_{<t})\\ -\alpha\,\mathrm{logit}_{\theta}(y_{t}~{}|~{}{\mathbf{x}},{\mathbf{y}}_{<t })]\]

Larger \(\alpha\) means more weight on our adjustment (\(\alpha=0\) reduces to regular decoding).1 We refer to this simple method as context-aware decoding. From the adjusted output distribution \(\tilde{p}\), we can apply various sampling strategies, such as nucleus sampling Holtzman et al. (2020).

Footnote 1: If we identify an external knowledge \({\mathbf{c}}\) conditionally independent to the generation, \(p_{\theta}(y_{t}~{}|~{}{\mathbf{c}},{\mathbf{x}},{\mathbf{y}}_{<t})=p_{\theta}(y_{t}~{}|~ {}{\mathbf{x}},{\mathbf{y}}_{<t})\), even a non-zero \(\alpha\) would not have an impact to the original output distribution.

Essentially, context-aware decoding is just a contrastive ensemble between the logits of \(p_{\theta}(y_{t}~{}|~{}{\mathbf{c}},{\mathbf{x}},{\mathbf{y}}_{<t})\) and \(p_{\theta}(y_{t}~{}|~{}{\mathbf{x}},{\mathbf{y}}_{<t})\). A similar contrastive objective is universal in image generation, where classifier-free diffusion models Ho and Salimans (2022) predict diffusion noise with \((1+\alpha){\mathbf{e}}_{\theta}({\mathbf{x}},{\mathbf{c}})-\alpha{\mathbf{e}}_{\theta}({\mathbf{x}})\), with \({\mathbf{c}}\) being a control to the image. In text generation, Malkin et al. (2022) propose coherence boosting with the same intuition, with a focus on contrasting the full input and a short premise-free input, promoting coherence w.r.t. the long context. Instead of using a single model \(\theta\) in this work, different models can also be used in the distribution adjustments to demote unwanted model behaviors or distill expert model's capability Liu et al. (2021); Li et al. (2023). We further discuss related works in SS6 and SSA.2.

## 3 Experimental Setup

We perform evaluation on tasks that require LMs to read and reason over contexts and produce outputs that are faithful to the contexts. Following prior work Zhang et al. (2024); Zhou et al. (2023), we evaluate the models using prompting.

### Datasets and Metrics

SummarizationWe conduct summarization experiments on CNN-DM See et al. (2017) and XSUM Narayan et al. (2018). We use ROUGE-L Lin (2004) to evaluate summarization quality.

 

To measure the factual consistency of summaries, we adopt state-of-the-art factuality evaluation metrics: BERT-Precision (Pagnoni et al., 2021) and FactKB (Feng et al., 2023), which has been demonstrated to achieve high correlations with human judgment on the summarization datasets, outperforming other metrics such as FACTCC (Kryscinski et al., 2020) and SUMMAC (Laban et al., 2022).

Knowledge ConflictsWe evaluate performance on two knowledge conflict datasets: MemoTrap (Liu and Liu, 2023) and NQ-Swap (Longpre et al., 2021). MemoTrap is created to investigate whether language models fall into memorization traps. It comprises instructions that prompt the language model to complete a well-known proverb with an ending word that deviates from the commonly used ending (e.g., _Write a quote that ends in the word "early": Better late than___). NQ-Swap is based on a QA dataset, natural questions (NQ) (Kwiatkowski et al., 2019), where the objective is to answer questions based on a gold document. To generate NQ-Swap, Longpre et al. (2021) identify questions in NQ with named entity answers, find the supportive document for each question and replace the gold answer entity in the document with a random entity. A faithful LM should generate the replaced entity as the answer when given the question and modified document. We also include the original NQ dataset with the question and original document for evaluation. We use Exact Match (EM) as the evaluation metric for NQ-Swap, NQ and MemoTrap.

In Table 1, we show illustrative examples of the contexts we aim to upweight for the model and the queries across different datasets. We hope LMs pay more attention to the source document in XSUM and NQ-Swap. On the other hand, we hope LMs focus more on the instruction in MemoTrap.

### Models and Baselines

We apply CAD to pretrained language models including OPT (Zhang et al., 2022), GPT-Neo (Black et al., 2021), LLaMA (Touvron et al., 2023) and instruction-finetuned LMs such as FLAN-T5 (Chung et al., 2022).

CAD introduces a hyperparameter \(\alpha\) to control the adjustment level. We set \(\alpha\) = \(0.5\) for all models evaluated on the summarization datasets and \(\alpha\) = \(1\) for all models evaluated on the knowledge conflict datasets. We observed that \(\alpha\) = \(0.5\) generally yielded good results across all settings and all datasets, but a slightly higher \(\alpha\) is more effective in the knowledge conflict setting, where the prior knowledge needs to be factored out more. We investigate the effect of \(\alpha\) in Section 5.

For the baselines, we use the regular decoding methods following prior work (Longpre et al., 2021; Kwiatkowski et al., 2019): greedy decoding for knowledge conflict tasks and top-\(p\) sampling with \(p\)=0.9 for summarization tasks (Holtzman et al., 2020). For CAD, we use the same sampling strategies on top of the adjusted output probability distribution.

## 4 Results

SummarizationTable 2 reports the results on CNN-DM and XSUM. We observe that CAD outperforms the standard decoding algorithm by a large margin in all eight models across both datasets. Specifically, when applied to LLaMAM-30B in CNN-DM, CAD leads to 21% increase in ROUGE-L, 14.3% increase in factKB and 7.8% increase in BERT-P. This result demonstrates that CAD could effectively improve the quality and factuality of the generated summaries from a diverse set of language models.

Knowledge ConflictsOur results for the knowledge conflict datasets, NQ-SWAP and MemoTrap, as well as the original NQ are detailed in Table 3. CAD is significantly better than the regular decoding in all settings, with the exception of a minor decrease observed for FLAN-T5 on the non-conflict NQ dataset.2 Despite this, CAD achieves better per

\begin{table}
\begin{tabular}{l l} \hline \multicolumn{2}{c}{**XSUM**} \\ \hline \(\mathbf{c}\) & Article: Prison Link Cymru had 1,099 referrals in 2015-16 and said some ex-offenders were living rough for up to a year before finding suitable accommodation ... \\ \(\mathbf{x}\) & Summarize the article in one sentence. Summary: \\ \hline \multicolumn{2}{c}{**NO-SWAP**} \\ \hline \(\mathbf{c}\) & Tesla CEO Elon Musk is now in charge of Twitter , CNBC has learned ... \\ \(\mathbf{x}\) & Who is Twitter CEO now? \\ \hline \multicolumn{2}{c}{**MemoTrap**} \\ \hline \(\mathbf{c}\) & Write a quote that ends in the word “early”: \\ \(\mathbf{x}\) & Better late than \\ \hline \end{tabular}
\end{table}
Table 1: An illustration of the inputs to CAD applied to each dataset. CAD upweights the context \(\mathbf{c}\) (in red) by sampling each token from \(\mathrm{softmax}[(1+\alpha)\log\!t_{\theta}(y_{t}\mid c,\mathbf{x},\mathbf{y}_{<t})- \alpha\log\!i_{\theta}(y_{t}\mid\mathbf{x},\mathbf{y}_{<t})]\).

 

formance on the knowledge conflict datasets, e.g., CAD improve GPT-Neo 20B by 54.4% on Memotrap and by 128% on NQ-SWAP. This substantial improvement suggests that context-aware decoding is particularly beneficial for LMs to adhere to the given context, in scenarios where the model's prior knowledge contradicts with the context knowledge.

## 5 Analysis

CAD brings consistent improvement to LMs with different sizes.In Tables 2 and 3, we show that CAD could be used to enhance a diverse set of LM families, including OPT, GPT-Neo, LLaMA, and FLAN-T5. We further investigate whether CAD is effective in improving language models of different sizes. Specifically, we focus on OPT models across a range of sizes: 125M, 350M, 1.3B, 2.7B, 6.7B, 13B, 30B. We observe that the performance gain brought by CAD stays consistent with different model sizes in CNN-DM. In Memotrap and NQ-SWAP, this gain increases as the model size grows, indicating that larger LMs can have a greater tendency to rely on their prior knowledge instead of reading the contexts, thereby benefiting more from CAD. In Figure 2, we observe that the performance gain brought by CAD stays consistent with different OPT model sizes in CNN-DM. In Memotrap and NQ-SWAP, this gain increases as the model size grows, indicating that larger LMs can have a greater tendency to rely on their prior knowledge instead of reading the contexts, thereby benefiting more from CAD.

Effect of adjustment level\(\alpha\)We then investigate the effect of different adjustment level \(\alpha\) (a small \(\alpha\) makes the distribution closer to the original next token distribution). We conduct experiments with various values of \(\alpha\) and present the results in Figure 3. Across all three datasets, we find \(\alpha=0.5\) consistently provide robust improvements over regular decoding.

## 6 Related Work

Summarization factualitySummarization models have shown a tendency to generate hallucinated texts Maynez et al. (2020); Pagnoni et al. (2021). This has led to growing efforts to improve the factual consistency, including applying attentions to

\begin{table}
\begin{tabular}{c c l|r r r|r r r} \hline \hline  & & & \multicolumn{3}{c|}{**CNN-DM**} & \multicolumn{3}{c}{**XSUM**} \\ \cline{4-9}  & **Model** & **Decoding** & ROUGE-L & factKB & BERT-P & ROUGE-L & factKB & BERT-P \\ \hline \multirow{4}{*}{OPT} & \multirow{2}{*}{13B} & Regular & 22.0 & 77.8 & 86.5 & 16.4 & 47.2 & 85.2 \\  & & CAD & **27.4** & **84.1** & **90.8** & **18.2** & **64.9** & **87.5** \\  & & Regular & 22.2 & 81.7 & 87.0 & 17.4 & 38.2 & 86.1 \\  & & CAD & **28.4** & **87.0** & **90.2** & **19.5** & **45.6** & **89.3** \\ \hline \multirow{4}{*}{GPT-Neo} & \multirow{2}{*}{3B} & Regular & 24.3 & 80.5 & 87.5 & 17.6 & 54.0 & 86.6 \\  & & CAD & **27.7** & **87.5** & **90.6** & **18.1** & **65.1** & **89.1** \\  & & Regular & 18.7 & 68.3 & 85.2 & 14.9 & 42.2 & 85.7 \\  & & CAD & **24.5** & **77.5** & **89.4** & **19.0** & **63.3** & **90.6** \\ \hline \multirow{4}{*}{LLaMA} & \multirow{2}{*}{13B} & Regular & 27.1 & 80.2 & 89.5 & 19.0 & 53.5 & 87.8 \\  & & CAD & **32.6** & **90.8** & **93.0** & **21.1** & **73.4** & **91.7** \\ \cline{1-1}  & & Regular & 25.8 & 76.8 & 88.5 & 18.7 & 47.7 & 87.1 \\ \cline{1-1}  & & CAD & **31.8** & **87.8** & **92.2** & **22.0** & **66.4** & **90.3** \\ \hline \multirow{4}{*}{FLAN} & \multirow{2}{*}{3B} & Regular & 25.5 & 90.2 & 91.6 & 18.8 & 31.9 & 88.2 \\  & & CAD & **26.1** & **93.9** & **92.1** & **19.5** & **35.9** & **88.8** \\ \cline{1-1}  & & Regular & 25.4 & 90.4 & 91.4 & 19.4 & 29.8 & 88.3 \\ \cline{1-1}  & & CAD & **27.1** & **93.1** & **92.2** & **20.0** & **35.0** & **88.8** \\ \hline \hline \end{tabular}
\end{table}
Table 2: **CAD consistently outperform the regular decoding method in terms of both summary quality metric (ROUGE-L) and summary factuality (factKB and BERT-P). The best scores for each setting are boldfaced. FLAN 3B and 11B refer to FLAN-T5 XL and FLAN-T5 XXL respectively.**

\begin{table}
\begin{tabular}{l l l|c c c} \hline \hline
**Model** & **Decoding** & **Memo.** & **NQ** & **NQ-SWAP** \\ \hline \multirow{4}{*}{OPT} & \multirow{2}{*}{13B} & Reg. & 32.5 & 29.2 & 18.8 \\  & & CAD & **44.5** & **32.2** & **36.9** \\  & & Reg. & 28.4 & 29.4 & 14.7 \\  & & CAD & **41.0** & **35.5** & **29.0** \\ \hline \multirow{4}{*}{GPT.} & \multirow{2}{*}{3B} & Reg. & 22.5 & 31.9 & 19.1 \\  & & CAD & **47.3** & **39.9** & **41.2** \\  & & Reg. & 37.1 & 22.8 & 16.1 \\  & & CAD & **57.3** & **32.1** & **36.8** \\ \hline \multirow{4}{*}{ILaMA} & \multirow{2}{*}{13B} & Reg. & 23.8 & 22.3 & 11.7 \\  & & CAD & **57.1** & **33.6** & **36.7** \\  & & Reg. & 25.8 & 23.8 & 9.6 \\  & & CAD & **50.6** & **34.0** & **37.7** \\ \hline \multirow{4}{*}{FLAN} & \multirow{2}{*}{3B} & Reg. & 69.2 & **81.8** & 71.4 \\  & & CAD & **72.2** & 80.3 & **73.3** \\ \cline{1-1}  & & Reg. & 82.0 & **85.5** & 73.0 \\ \cline{1-1}  & & CAD & **88.7** & 82.5 & **77.1** \\ \hline \hline \end{tabular}
\end{table}
Table 3: CAD outperforms the regular decoding method (Reg.) in all settings except for FLAN-T5 on NQ.

 

fact triples extracted from source documents Cao et al. (2018); Zhu et al. (2021), optimizing summarization models towards a factual consistency metrics Nan et al. (2021); Cao and Wang (2021), learning a post-editing error corrector Dong et al. (2020) and removing noisy training samples Kang and Hashimoto (2020); Goyal and Durrett (2021). These methods require additional fine-tuning and are not directly suitable for zero-shot and few-shot prompting scenarios. King et al. (2022) and Sridhar and Visser (2022) propose to alleviate the issue by constraining beam search algorithms.

Knowledge conflictsWhen presented with an updated document with conflicting knowledge, we expect language models to generate responses based on the provided contexts rather than relying solely on outdated parametric knowledge. This setting is especially valuable to retrieval-augmented language models Khandelwal et al. (2020); Shi et al. (2024); Min et al. (2023); Yasunaga et al. (2023), where documents retrieved from external databases are used as additional input to provide LMs additional knowledge. However, simply adding documents does not always change the model predictions, as current LMs often overlook the contexts and rely heavily on their prior parametric knowledge Longpre et al. (2021); Chen et al. (2022). Existing approaches for improving model's faithfulness to the context, such as the prompting-based method Zhou et al. (2023), are limited in that they could only apply to large-scale instruction-finetuned LMs like OpenAI's text-davinci-003. In contrast, our work investigates a decoding strategy to tackle this problem, applicable to any LM.

## 7 Conclusion

Language models suffer from an insufficient attention to the given context compared to its prior knowledge, leading to an unfaithful generation to the input context. We present CAD, a simple inference-time method that downweights an output probability associated with the model's prior knowledge to promote models' attention to the context. We experiment on two families of tasks that require a strong attention to the context and show that CAD provides more faithful outputs across different language models of various sizes.

Figure 3: Effect of the adjustment level \(\alpha\). The y-axis is the performance and the x-axis is \(\alpha\).

Figure 2: OPT models of varying sizes consistently benefit from CAD. The x-axis indicates the size of language models and the y-axis is the performance.

 



### Limitations

Our proposed CAD method requires the output logits from language models in order to contrastively calculate the probability distribution with and without contexts. However, API-based language models like ChatGPT and GPT-4 may not provide output logits. Consequently, it is not feasible for CAD to be directly applied to such fully black-box models. Furthermore, CAD introduces a hyperparameter \(\alpha\), which serves to regulate the level of contrastive adjustment. While we have observed that \(\alpha=0.5\) yields consistent enhancements compared to regular decoding, different models applied to various tasks may have distinct optimal values for \(\alpha\). If there exists a very small demonstration set of in-domain examples, we would consider the selection of \(\alpha\) similar to other decoding parameters like the top-p or temperature values.

## Acknowledgement

We thank Alisa Liu and Jiacheng Liu for providing insights during discussions of the project. This research is supported in part by the Office of the Director of National Intelligence (ODNI), Intelligence Advanced Research Projects Activity (IARPA), via the HIATUS Program contract #2022-22072200004. This material is also funded in part by the DARPA Grant under Contract No. HR001120C0124. We also gratefully acknowledge support from NSF CAREER Grant No. IIS2142739, NSF Grants No. IIS2125201, IIS2203097, and the Alfred P. Sloan Foundation Fellowship. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of ODNI, IARPA, or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for governmental purposes notwithstanding any copyright annotation therein.

## References

* S. Black, G. Leo, P. Wang, C. Leahy, and S. Biderman (2021)OPT-Neo: large scale autoregressive language modeling with mesh-tensorflow. External Links: Link Cited by: SS6.
* S. Cao and L. Wang (2021)CLIFF: contrastive learning for improving faithfulness and factuality in abstractive summarization. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Online and Punta Cana, Dominican Republic, pp. 6633-6649. External Links: Link Cited by: SS6.
* Z. Cao, F. Wei, W. Li, and S. Li (2018)Faithful to the original: fact aware neural abstractive summarization. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018, pp. 4784-4791. External Links: Link Cited by: SS6.
* A. Chen, M. Zhang, and E. Choi (2022)Rich knowledge sources bring complex knowledge conflicts: facilitating models to reflect conflicting evidence. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, Abu Dhabi, United Arab Emirates, pp. 2292-2307. External Links: Link Cited by: SS6.
* H. W. Chung, L. Hou, S. Longpre, B. Zoph, Y. Tay, W. Fedus, Y. Li, X. Wang, M. Dehghani, S. Brahma, A. Webson, S. Shane Gu, Z. Dai, M. Suzgun, X. Chen, A. Chowdhery, A. Castro-Ros, M. Pellat, K. Robinson, D. Valter, S. Narang, G. Mishra, A. Yu, V. Zhao, Y. Huang, A. Dai, H. Wu, S. Petrov, E. H. Chi, J. Dean, J. Devlin, A. Roberts, D. Zhou, Q. V. Le, and J. Wei (2022)Scaling instruction-finetuned language models. ArXiv preprintabs/2210.11416. External Links: Link Cited by: SS6.
* J. Davison, J. Feldman, and A. Rush (2019)Commonsense knowledge mining from pretrained models. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China, pp. 1173-1178. External Links: Link Cited by: SS6.
* Y. Dong, S. Wang, Z. Gan, Y. Cheng, J. Chi Kit Cheung, and J. Liu (2020)Multi-fact correction in abstractive text summarization. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Online, pp. 9320-9331. External Links: Link Cited by: SS6.
* S. Feng, V. Balachandran, Y. Bai, and Y. Tsvetkov (2023)FactKB: generalizable factuality evaluation using language models enhanced with factual knowledge. In Proceedings of

the 2023 Conference on Empirical Methods in Natural Language Processing_, pages 933-952, Singapore. Association for Computational Linguistics.
* Goyal and Durrett (2021) Tanya Goyal and Greg Durrett. 2021. Annotating and modeling fine-grained factuality in summarization. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 1449-1462, Online. Association for Computational Linguistics.
* Guo et al. (2023) Meiqi Guo, Rebecca Hwa, and Adriana Kovashka. 2023. Decoding symbolism in language models. In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 3311-3324, Toronto, Canada. Association for Computational Linguistics.
* Ho and Salimans (2022) Jonathan Ho and Tim Salimans. 2022. Classifier-free diffusion guidance. _ArXiv preprint_, abs/2207.12598.
* Holtzman et al. (2020) Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020. The curious case of neural text degeneration. In _8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020_. OpenReview.net.
* Holtzman et al. (2021) Ari Holtzman, Peter West, Vered Shwartz, Yejin Choi, and Luke Zettlemoyer. 2021. Surface form competition: Why the highest probability answer isn't always right. In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_, pages 7038-7051, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
* Kang and Hashimoto (2020) Daniel Kang and Tatsumori B. Hashimoto. 2020. Improved natural language generation via loss truncation. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 718-731, Online. Association for Computational Linguistics.
* Khandelwal et al. (2020) Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. 2020. Generalization through memorization: Nearest neighbor language models. In _8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020_. OpenReview.net.
* King et al. (2022) Daniel King, Zejiang Shen, Nishant Subramani, Daniel S. Weld, Iz Beltagy, and Doug Downey. 2022. Don't say what you don't know: Improving the consistency of abstractive summarization by constraining beam search. In _Proceedings of the 2nd Workshop on Natural Language Generation, Evaluation, and Metrics (GEM)_, pages 555-571, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
* Kryscinski et al. (2020) Wojciech Kryscinski, Bryan McCann, Caiming Xiong, and Richard Socher. 2020. Evaluating the factual consistency of abstractive text summarization. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_, pages 9332-9346, Online. Association for Computational Linguistics.
* Kwiatkowski et al. (2019) Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019. Natural questions: A benchmark for question answering research. _Transactions of the Association for Computational Linguistics_, 7:452-466.
* Laban et al. (2022) Philippe Laban, Tobias Schnabel, Paul N. Bennett, and Marti A. Hearst. 2022. SummaC: Re-visiting NLI-based models for inconsistency detection in summarization. _Transactions of the Association for Computational Linguistics_, 10:163-177.
* Li et al. (2016) Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016. A diversity-promoting objective function for neural conversation models. In _Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 110-119, San Diego, California. Association for Computational Linguistics.
* Li et al. (2023) Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori Hashimoto, Luke Zettlemoyer, and Mike Lewis. 2023. Contrastive decoding: Open-ended text generation as optimization. In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 12286-12312, Toronto, Canada. Association for Computational Linguistics.
* Lin (2004) Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In _Text Summarization Branches Out_, pages 74-81, Barcelona, Spain. Association for Computational Linguistics.
* Liu and Liu (2023) Alisa Liu and Jiacheng Liu. 2023. The memotrap dataset. [https://github.com/inverse-scaling/prize/blob/main/data-release/README.md](https://github.com/inverse-scaling/prize/blob/main/data-release/README.md).
* Liu et al. (2021) Alisa Liu, Maarten Sap, Ximing Lu, Swabha Swayamdipta, Chandra Bhagavatula, Noah A. Smith, and Yejin Choi. 2021. DE Experts: Decoding-time controlled text generation with experts and anti-experts. In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_, pages 6691-6706, Online. Association for Computational Linguistics.
* Longpre et al. (2021) Shayne Longpre, Kartik Perisetla, Anthony Chen, Nikhil Ramesh, Chris DuBois, and Sameer Singh. 2021. Entity-based knowledge conflicts in question answering. In _Proceedings of the 2021 Conference 

on Empirical Methods in Natural Language Processing_, pages 7052-7063, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
* Malkin et al. (2022) Nikolay Malkin, Zhen Wang, and Nebojsa Joic. 2022. Coherence boosting: When your pretrained language model is not paying enough attention. In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 8214-8236, Dublin, Ireland. Association for Computational Linguistics.
* Maynez et al. (2020) Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. 2020. On faithfulness and factuality in abstractive summarization. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 1906-1919, Online. Association for Computational Linguistics.
* Min et al. (2023) Sewon Min, Weijia Shi, Mike Lewis, Xilun Chen, Wentau Yih, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2023. Nonparametric masked language modeling. In _Findings of the Association for Computational Linguistics: ACL 2023_, pages 2097-2118, Toronto, Canada. Association for Computational Linguistics.
* Nan et al. (2021) Feng Nan, Cicero Nogueira dos Santos, Henghui Zhu, Patrick Ng, Kathleen McKeown, Ramesh Nallapati, Dejaao Zhang, Zhiguo Wang, Andrew O. Arnold, and Bing Xiang. 2021. Improving factual consistency of abstractive summarization via question answering. In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_, pages 6881-6894, Online. Association for Computational Linguistics.
* Narayan et al. (2018) Shashi Narayan, Shay B. Cohen, and Mirella Lapata. 2018. Don't give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization. In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_, pages 1797-1807, Brussels, Belgium. Association for Computational Linguistics.
* Pagnoni et al. (2021) Artidoro Pagnoni, Vidhisha Balachandran, and Yulia Tsvetkov. 2021. Understanding factuality in abstractive summarization with FRANK: A benchmark for factuality metrics. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 4812-4829, Online. Association for Computational Linguistics.
* See et al. (2017)bigail See, Peter J. Liu, and Christopher D. Manning. 2017. Get to the point: Summarization with pointer-generator networks. In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 1073-1083, Vancouver, Canada. Association for Computational Linguistics.
* Shi et al. (2021)Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Rich James, Mike Lewis, Luke Zettlemoyer, and Wen tau Yih. 2021. REPLUG: Retrieval-augmented black-box language models. In _Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_.
* Sridhar and Visser (2022) Arvind Krishna Sridhar and Erik Visser. 2022. Improved beam search for hallucination mitigation in abstractive summarization. _ArXiv preprint_, abs/2212.02712.
* Touvron et al. (2023) Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothee Lacroix, Baptiste Roziere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. LLAMA: Open and efficient foundation language models. _ArXiv preprint_, abs/2302.13971.
* van der Poel et al. (2022)Liam van der Poel, Ryan Cotterell, and Clara Meister. 2022. Mutual information alleviates hallucinations in abstractive summarization. In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_, pages 5956-5965, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
* Yasunaga et al. (2023)Michihiro Yasunaga, Armen Aghajanyan, Weijia Shi, Rich James, Jure Leskovec, Percy Liang, Mike Lewis, Luke Zettlemoyer, and Wen-tau Yih. 2023. Retrieval-augmented multimodal language modeling. In _International Conference on Machine Learning (ICML)_.
* Zhang et al. (2022) Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewwan, Mona Diab, Xian Li, Xi Victoria Lin, et al. 2022. Opt: Open pre-trained transformer language models. _ArXiv preprint_, abs/2205.01068.
* Zhang et al. (2024) Tianyi Zhang, Faisal Ladhak, Esin Durmus, Percy Liang, Kathleen McKeown, and Tatsunori B. Hashimoto. 2024. Benchmarking Large Language Models for News Summarization. _Transactions of the Association for Computational Linguistics_, 12:39-57.
* Zhou et al. (2023) Wenzuan Zhou, Sheng Zhang, Hoifung Poon, and Muhao Chen. 2023. Context-faithful prompting for large language models. In _Findings of the Association for Computational Linguistics: EMNLP 2023_, pages 14544-14556, Singapore. Association for Computational Linguistics.
* Zhu et al. (2021)Chengang Zhu, William Hinthorn, Ruochen Xu, Qingkai Zeng, Michael Zeng, Xuedong Huang, and Meng Jiang. 2021. Enhancing factual consistency of abstractive summarization. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 718-733, Online. Association for Computational Linguistics 



## Appendix A Appendix

### Qualitative Analysis

We provide qualitative examples for XSUM and Memotrap in Table 4. In XSUM, the regular decoding generates texts that is not mentioned in the article, whereas CAD produces output exclusively based on the information in the input article. For MemoTrap, the standard decoding disregards the instruction and generates the memorized ending, while CAD adheres to the instruction within the given context and produces the desired output.

### Additional Related Work

Contrastive decoding methodsContrastive decoding methods have been extensively explored for text generation. Coherence boosting Malkin et al. (2022) and CPMI van der Poel et al. (2022) demote a short context from a full context, focusing on the longer-range context for coherence and overall better generation quality. MMI-based decoding Li et al. (2016) uses a contrastive formulation to improve output diversity in dialog generation. In this work, we adopt a same intuition and focus on analyzing the _knowledge conflict_ scenarios where the faithfulness to the context is particularly important but difficult for the regular decoding methods. We also extensively experiment the setup with a diverse set of language models and scales. DExperts Liu et al. (2021) demotes the output distribution of an _anti_-expert (e.g., exposed to toxic language) to help lead the generations free from the unwanted attributes. Contrastive decoding Li et al. (2023) demotes an _amateur_ model (e.g., models with a very small number of parameters) to help distill the expert knowledge learned in the larger, more competitive models. In general, contrastive decoding has shown to be a general way to control model outputs, which we reinforce by considering the new case of factual consistency with the textual context.

Pointwise mutual information in text classificationThe concept of Pointwise Mutual Information (PMI) is extensively examined in text classification and reranking, serving to adjust the weighting of various classification choices based on the increased likelihood of an answer given a question within a specific task domain. Past research has applied it to zero-shot multiple-choice tasks Holtzman et al. (2021), as well as the reranking of candidates for commonsense and symbolic knowledge extraction Guo et al. (2023); Davison et al. (2019).

\begin{table}
\begin{tabular}{l l} \hline \hline  & **MemoTrap** \\ \hline Input & Write a quote that ends in the word “early”. \\ Regular & \begin{tabular}{c} Insver \\ early \\ \end{tabular} \\ \cline{2-3} CAD & 
\begin{tabular}{c} Occidental \\ early \\ \end{tabular} \\ \hline \hline \end{tabular}
\end{table}
Table 4: Qualitative examples of contrast-aware decoding. The nonfactual or inconsistent texts are highlighted in yellow.