DeCoRe: Decoding by Contrasting Retrieval Heads to Mitigate Hallucinations

Authors: Aryo Pradipta Gema, Chen Jin, Ahmed Abdulaal, Tom Diethe, Philip Alexander Teare, Beatrice Alex, Pasquale Minervini, Amrutha Saseendran
Primary area: generative models
Keywords: Large Language Models, Hallucination, Constrained Decoding, Retrieval Heads

Abstract:
Large Language Models (LLMs) often hallucinate, producing unfaithful or factually incorrect outputs by misrepresenting the provided context or incorrectly recalling internal knowledge. Recent studies have identified specific attention heads within the Transformer architecture, known as retrieval heads, responsible for extracting relevant contextual information. We hypothesise that masking these retrieval heads can induce hallucinations and that contrasting the outputs of the base LLM and the masked LLM can reduce hallucinations. To this end, we propose Decoding by Contrasting Retrieval Heads (DeCoRe), a novel training-free decoding strategy that amplifies information found in the context and model parameters. DeCoRe mitigates potentially hallucinated responses by dynamically contrasting the outputs of the base LLM and the masked LLM, using conditional entropy as a guide. Our extensive experiments confirm that DeCoRe significantly improves performance on tasks requiring high contextual faithfulness, such as summarisation (XSum by 18.6\%), instruction following (MemoTrap by 10.9\%), and open-book question answering (NQ by 2.4\% and NQ-Swap by 5.5\%).

Decision: reject
Reviewer scores: 5, 5, 6, 6

Reviews and discussion:
<thread>
creation date:
1730523891142

thread_type:
official_review

<thread_object>
creator: reviewer_1dqs

<content>
summary:
The paper proposes a new variant of contrastive decoding for reducing hallucinations in language models (LMs). At high level, contrastive decoding uses the difference between two LMs’ likelihood as the final decoding criteria. Typically, there is a baseLLM that generates a basic sequence of tokens which might contain hallucinations. To reduce the amount of hallucination, the probabilities of each generated token is then offset by the corresponding token probabilities from another weaker or corrupted LM which might contain more hallucinations. By using the offset to calibrate the sequence likelihood, the resulting decoding can potentially avoid undesirable failures from the second LM.

In this paper, the authors focus on building the corrupted LM leveraging the so-called retrieval attention heads found by recent work. Specifically, by masking those retrieval attention heads, the resulting corrupted LM is more likely to ignore context information, leading to more faithfulness issues (hallucinations).

Thus, the hypothesis is that using the token probability difference between the base LM and the corrupted LM can lead to better grounded responses.

Experiments are conducted on several representative datasets across different open source LMs (Llama 3 8B & 70B in main text), including long-form (summarization) and short-form (question answering) generation tasks.
Overall, the proposed method is found to be effective in making LMs more sensitive to provided context while not interfering with the LM’s parametric knowledge.

strengths:
The paper is easy-to-follow with necessary background.

Given the wide range of LM applications, developing efficient and reliable hallucination mitigation methods is a very important research direction. 

The proposed method is a nice extension of contrastive decoding with recent attention head function insights.

The experiment design are mostly reasonable with several recent baselines to validate the effectiveness of the proposed method

weaknesses:
Although it is good to build upon recent attention head insights for contrastive decoding, the generalizability of those identified retrieval attention seems very limited. 

The improvement over baselines is at odds and not pronounced enough. For example, the proposed method does not provide consistent improvements over the base LM and CAD on fluency and coherence (lower ROUGE-L and BERTScore-F1). It is good to analyze the response difference to drive deeper understanding, e.g., does the proposed method lead to shorter and more concise generation? Typically, shorter answers are preferred by question answering datasets, e.g., NQ, TriviaQA.

As reported by the authors in Fig 3, masking more heads is not necessarily helpful. It is good to explore alternative attention selection strategies, e.g., randomly masking, selecting heads based on the required task solving capabilities (profiling using question answering datasets), or layer-based selection (e.g., balanced masking out across different layers).
That said, it might be interesting to see whether certain attention heads are more related to shorter vs longer context retrieval. 

There are some further improvements can be done for the experiment part:
Most of the datasets studied are of shorter input context. It is good to explore some datasets with longer context (e.g., retrieving more passages) and compare the method there. In addition to longer context, it is also good to explore more involving contexts, e.g., contextual conflicts [1].

It is good to report the robustness of those chosen hyperparameters. For example, can the proposed method benefit from different sampling temperatures and varying number of heads to be masked. As in-context examples are used for certain tasks, it is good to study the robustness of that decision, e..g, different set of shot and less or more examples.

[1] Wang et al., Resolving Knowledge Conflicts in Large Language Models

questions:
Is there any difference in terms of generations from the proposed method vs others, e.g., fluency, length?

What is the smaller LM used for DeCoRe lite? 

What is the alpha value for DeCoRe static? How that is determined?

What is the computational cost for all compared methods, e.g., tflops?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730556594179

thread_type:
official_review

<thread_object>
creator: reviewer_6h8b

<content>
summary:
The paper introduces DeCoRe, a decoding strategy for reducing hallucinations in LLMs. DeCoRe works by contrasting between a LLM (that can be augmented with additional context), and the same LLM where the attention heads that are responsible for extracting information from the context (i.e., retrieval heads) are masked, assuming masking these heads induces hallucinations and thus contrasting between the two distributions increase factuality. The weight given to each distribution can be controlled by a hyperparameter that estimates the model’s uncertainty using the entropy of the model. Experimental results show that DeCoRe significantly improves contextual faithfulness on multiple tasks including summarization and open-domain QA.

strengths:
- The methodology of contrasting between a variant where retrieval heads are masked is interesting and has benefits over previous methods, for example in applicability.
- The experimental setting is thorough, comparing against six strong baselines on three tasks (reading comprehension, summarization, and instruction-following) with multiple datasets, including XSum, MemoTrap, IFEval NQ, PopQA, and TruthfulQA.
- The advancements on MemoTrap in comparison to the baselines are significant.
- The paper is overall well-written and easy to follow.

weaknesses:
- It seems that in several experiments, the improvements from DeCoRe are either marginal or it even performs significantly worse than other methods. For example, as presented in Tab.1 CAD significantly outperforms DeCoRe on NQ-Swap whilst performing similarly on other benchmarks (within a one point margin). While I agree with the paper that DeCoRe has higher applicability I am not entirely sure that CAD cannot be used for MemoTrap (see questions).

- The paper also claims that DeCoRe can increase factuality (Tab.2) and multi-hop reasoning with CoT (Tab.3) but from looking at these tables, in most cases that DeCoRe is best-performing it is within a one-point margin to the second-best performing model (as stated in the paper, there are also cases where the baselines perform better). Additionally, in all experiments three DeCoRe variants are compared against the baselines, which can be a bit misleading. I believe the paper might benefit from an aggregated score for each method and error bars or statistical tests for the main claims. This can simplify understanding the significance of the main results.

- I believe the paper might benefit from a qualitative analysis that supports the main claims. For example, when is using conditional entropy beneficial and when does it hurt performance, and when does DeCoRe outperform the baselines?

questions:
Please see weaknesses, I am willing to reconsider my score.

Regarding CAD and MemoTrap, maybe I am missing something, but perhaps it’s possible to use the task as the additional context, e.g., for the task - “Write a quote that ends in the word "talk": Don't bark if you can't”, the additional context can be “Write a quote that ends in the word "talk".

Again, maybe I am missing something, but lines 330-331 state that DeCoRe has ‘competitive performance’ on NQ-Swap, but CAD outperforms DeCoRe by 8+ points. What gains do you consider substantial in the context of the experiments and the datasets in the paper?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730707373949

thread_type:
official_review

<thread_object>
creator: reviewer_ywx4

<content>
summary:
This paper introduces a training-free decoding approach named DeCoRe for reducing hallucinations in LLMs. By identifying specific retrieval heads that influence the generation of grounded content, DeCoRe employs a masking strategy to induce hallucinations, then applies contrastive decoding to distinguish accurate from hallucinatory outputs. Experiments show improvements in contextual faithfulness on datasets like XSum and factual accuracy on tasks such as TriviaQA.

strengths:
1. The paper is clearly structured, making complex ideas accessible and easy to follow.

2. The paper introduces a novel decoding approach that contrasts outputs from the original and masked LLM, effectively reducing ungrounded content.

3. The paper  incorporates an entropy-controlled mechanism, allowing the method to adjust the level of contrast applied depending on the model’s uncertainty.

weaknesses:
1. The model's performance improvements are inconsistent across different datasets, and it fails to outperform baseline models in many settings (e.g., XSum, IFEval). This is inconsistent with the summary conclusions presented in the abstract.

2. The paper only compares a limited number of baselines, primarily focusing on contrastive methods, without evaluating against many related models.

3. The approach in this paper appears to be related to self-consistency and should better include a discussion on this topic, along with relevant baseline comparisons.

questions:
n/a
</content>
</thread_object>

</thread>
<thread>
creation date:
1730732631611

thread_type:
official_review

<thread_object>
creator: reviewer_9xyh

<content>
summary:
The paper addresses the issue of hallucinations in Large Language Models (LLMs), where outputs may be unfaithful or factually incorrect. It identifies specific attention heads, called retrieval heads, within the Transformer architecture that are crucial for extracting relevant information. The authors hypothesize that masking these heads can induce hallucinations. To tackle this, they propose a novel decoding strategy named Decoding by Contrasting Retrieval Heads (DeCoRe), which does not require additional training. DeCoRe works by comparing outputs from the base LLM and a masked version to reduce hallucinations, guided by conditional entropy. Experiments show that DeCoRe significantly enhances performance in tasks demanding high contextual accuracy, such as summarization, instruction following, and open-book question answering, with notable improvements in various benchmarks.

strengths:
1.	The paper is well-written.
2.	The research includes thorough experiments that effectively demonstrate the capability of DeCoRe in mitigating hallucinations across various tasks.
3.	The proposed method, DeCoRe, is training-free and useful for hallucination mitigation.

weaknesses:
1.	The paper only provides evaluations on the effectiveness of mitigating hallucination. However, as a decoding method, it is also important to show the generated texts are coherent and fluent. Therefore, it is required to provide evidence to show this decoding method can generate text with high quality with low hallucination.
2.	It seems that in the experiments, many baselines can achieve better results than DeCoRe.
3.	The paper seems a combination of the existing findings, which applies the retrieval heads (Wu et al., 2024) to the Contrastive Decoding (Li et al., 2023) method, potentially limiting the originality of the contribution.

questions:
See weaknesses
</content>
</thread_object>

</thread>
<thread>
creation date:
1732279198116

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General response

comment:
We thank the reviewers for their time and detailed reviews.

## Strengths

We appreciate that the reviewers recognise the strengths of our work:

| Category | Comment | Reviewer |
|---|---|---|
| **Importance and Novelty** | "Given the wide range of LM applications, developing efficient and reliable hallucination mitigation methods is a very important research direction." | 1dQs |
| | "The methodology of contrasting between a variant where retrieval heads are masked is interesting and has benefits over previous methods, for example in applicability." | 6H8B |
| | "The proposed method, DeCoRe, is training-free and useful for hallucination mitigation." | 9XYH |
| | "The proposed method is a nice extension of contrastive decoding with recent attention head function insights." | 1dQs |
| | "Introduces a novel decoding approach that contrasts outputs from the original and masked LLM, effectively reducing ungrounded content." | yWX4 |
| | "Incorporates an entropy-controlled mechanism, allowing the method to adjust the level of contrast applied depending on the model’s uncertainty." | yWX4 |
| **Experimental Validation and Results** | "The research includes thorough experiments that effectively demonstrate the capability of DeCoRe in mitigating hallucinations across various tasks." | 9XYH |
| | "The experimental setting is thorough, comparing against six strong baselines on three tasks (reading comprehension, summarization, and instruction-following) with multiple datasets, including XSum, MemoTrap, IFEval NQ, PopQA, and TruthfulQA." | 6H8B |
| | "The experiment designs are mostly reasonable with several recent baselines to validate the effectiveness of the proposed method." | 1dQs |
| | "The advancements on MemoTrap in comparison to the baselines are significant." | 6H8B |
| **Clarity/Presentation** | "The paper is easy to follow with the necessary background." | 1dQs |
| | "The paper is well-written." | 9XYH |
| | "The paper is clearly structured, making complex ideas accessible and easy to follow." | yWX4 |
| | "The paper is overall well-written and easy to follow." | 6H8B |

## Amendments

We deeply appreciate their feedback, and to that end, we made several amendments to the manuscript, which we believe significantly improved its quality.

| **Category** | **Amendment** | **Reviewer** |
|---|---|---|
| **Experimental Results** | Added Lost-in-the-Middle experiments in the Results section (in the “DeCoRe Mitigates Faithfulness Hallucinations amidst Distractor Documents” paragraph and in Table 2). | 1dQs |
| | Added average performance in Tables [1](https://anonymous.4open.science/r/decore-4FB7/docs/assets/decore_rebuttals/table_1.png), [2](https://anonymous.4open.science/r/decore-4FB7/docs/assets/decore_rebuttals/table_2_and_explanation.png), [3](https://anonymous.4open.science/r/decore-4FB7/docs/assets/decore_rebuttals/table_3.png), [4](https://anonymous.4open.science/r/decore-4FB7/docs/assets/decore_rebuttals/table_4.png), and overall average performance in [Table 5](https://anonymous.4open.science/r/decore-4FB7/docs/assets/decore_rebuttals/table_5.png). | 6H8B |
| **Statistical Analysis** | Added statistical test analyses in Appendices [H.1](https://anonymous.4open.science/r/decore-4FB7/docs/assets/decore_rebuttals/appendix_h1_table_22.png), [H.2](https://anonymous.4open.science/r/decore-4FB7/docs/assets/decore_rebuttals/appendix_h2_table_23.png), and [H.3](https://anonymous.4open.science/r/decore-4FB7/docs/assets/decore_rebuttals/appendix_h3_table_24.png) and toned down claims of significant improvements when they were not statistically significant in several factuality evaluation tasks. | 6H8B |
| **Computational Efficiency** | Added a section in [Appendix K.4](https://anonymous.4open.science/r/decore-4FB7/docs/assets/decore_rebuttals/appendix_k4_table_33.png) to show the computational efficiency (TFLOPS) of DeCoRe, referred to in the Results section. | 1dQs |
| **Qualitative Evaluation** | Added a reference to Appendix L.2 at the end of the Results section; Appendix L.2 shows generation examples for qualitative evaluation purposes. | 9XYH, 6H8B, 1dQs |
| | Added an analysis of the average length of generations in Appendix [L.1](https://anonymous.4open.science/r/decore-4FB7/docs/assets/decore_rebuttals/appendix_l1_table_34.png). | 1dQs |
| **Additional Clarification** | Restructured the Experimental Setup section (3.4) to include details about DeCoRe_entropy-lite. | 1dQs |
| | Added a discussion on the relationship between DeCoRe and self-consistency in the Related Works section. | yWX4 |

All changes are highlighted in blue in the revised draft.
</content>
</thread_object>

</thread>
<thread>
creation date:
1733225508502

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Summary of Contributions

comment:
We thank the reviewers for their insightful comments. Here is a **summary of our contributions**:

- **A Novel Contrastive Decoding Mechanism**: We introduce **DeCoRe**, which **contrasts** outputs from the base LLM and a variant where its **retrieval heads are masked**, effectively reducing hallucinations without requiring fine-tuning or significant computational overhead;
- **Conditional Entropy Guidance**: DeCoRe utilises **conditional entropy** --a predictor of potential hallucinations-- to dynamically modulate the contrastive weight hyperparameter during the decoding process;
- **Accuracy Boost in Most Tasks**: DeCoRe achieves more accurate results compared to most baselines in the majority of tasks (**10 out of 17 tasks**).

**Limitations and Future Work**:

- While **DeCoRe may not outperform specific baselines** on every task (e.g., ITI in TruthfulQA or CAD in MemoTrap and NQ-Swap), these **baselines often experience significant performance drops** on other tasks. This variability is a common limitation in the field, as methods may be tailored to excel in particular areas. In contrast, **DeCoRe offers consistently strong performance with very minimal drops** across tasks.
- DeCoRe uses token-level conditional entropy to modulate the contrastive weight. Future work may consider **other hallucination detection methods** beyond token-level conditional entropy such as semantic entropy.
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper introduces a variant of contrastive decoding to mitigate hallucinations in language models. The proposed DeCoRe identifies specific retrieval heads that influence the generation of grounded content and uses a masking strategy to induce hallucinations. It then applies contrastive decoding to differentiate accurate outputs from hallucinatory ones. Experiments demonstrate improved contextual faithfulness on datasets like XSum and enhanced factual accuracy on tasks such as TriviaQA.

Pros:
1. The paper is clearly structured, making complex ideas accessible and easy to follow.
2. Experimental results demonstrate the capability of DeCoRe in mitigating hallucinations across various tasks.

Cons:
1. Limited Novelty. The paper seems a combination of the existing findings, which applies the retrieval heads (Wu et al., 2024) to the Contrastive Decoding (Li et al., 2023) method, potentially limiting the originality of the contribution.
2. Effectiveness. The improvement over baselines is at odds and not pronounced enough. It seems that in the experiments, many baselines can achieve better results than DeCoRe. The model's performance improvements are inconsistent across different datasets.
3. Insufficient evaluation. The paper only provides evaluations on the effectiveness of mitigating hallucination. However, as a decoding method, it is also important to show the generated texts are coherent and fluent. Therefore, it is required to provide evidence to show this decoding method can generate text with high quality with low hallucination.

While the reviewers agree this paper is fairly well written and easy to follow, there are several major drawbacks that are not fully addressed. Therefore, I believe this paper needs another round of major revision before it can be published.