Paper: Self-contradictory Hallucinations of Large Language Models: Evaluation, Detection and Mitigation (ICLR-2024)

Reviewer comment (Experiments):
Correct me if I'm wrong, the authors only evaluated aLM to detect output from gLM, how robust are aLMs to text generation tasks? that are not generated by "generating initial text, defining context and trigger" process?

Author response:
> ### Is aLM robust to another text generation task?

Yes. We have added an evaluation on question answering, with the PopQA benchmark [1] suggested by reviewer a7zQ. The results show that our method can accurately detect a large number of self-contradictions, for both vanilla and retrieval-augmented generation. More details can be found in the global response.

[1] When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories. ACL 2023

Reviewer comment (Writing):
Last highlight in Revise, Section 5 is wrong.

Author response:
> ### Is the last highlight in paragraph Revise of Section 5 wrong?

We have re-checked it and are confident that it is correct. According to https://wiki.factsider.com/william-t-freeman/, Freeman’s birth country is indeed the USA. If the reviewer still disagrees, we kindly request the reason.

Reviewer comment (Experiments):
does the authors try other temperature in between? 0.5, 0.7 for both aLM and gLM?

Author response:
> ### Can you provide evaluation on different temperature values, for both aLM and gLM?

Yes. We have provided this evaluation in the global response. The results show that across various temperatures (0, 0.25, 0.5, 0.75 and 1.0), self-contradictions are consistently prevalent and our method for detecting self-contradictions is robust.

Reviewer comment (Theory):
The paper's central premise is that LLMs can produce self-contradictory hallucinations, and the proposed method aims to rectify using a prompting-based approach with LLMs. While it's acknowledged that LLMs can operate as zero-shot reasoners, their potential to induce hallucinations is also evident. Asserting that the authors correct LLM-induced hallucinations using LLMs (without external knowledge) appears paradoxical. This paper should incorporate a theoretical analysis explaining the efficacy of the proposed method in mitigating hallucinations.

Author response:
Our mitigation approach **does not correct** hallucinations. Instead, it **removes** contradictory information that already exists in the two sentences. Our removal can be achieved by comparing the two sentences and does not rely on external knowledge. For instance, in Figure 3, the two sentences provide conflicting information about the birth date for William T. Freeman. This inconsistency can be removed without knowing Freeman’s actual birth date, by dropping the information about the birth date entirely from the sentence. It is rare but not impossible that LLMs introduce a new hallucination when generating the revised sentence. Since our mitigation algorithm is iterative, these new hallucinations are resolved in subsequent iterations.

Our removal approach offers a novel perspective on mitigating hallucinations compared to the traditional correction approach. The motivation is to prevent LMs from generating mistakes when they are uncertain about specific facts. It aligns well with machine learning approaches that incorporate a reject or abstention option [2].

Reviewer comment (Experiments):
In the experiments, the verification of the text against Wikipedia is assessed, and various ratios are presented. However, a more in-depth exposition of the experimental setup should be shown. Specifically, manual checks were mentioned, but it's questionable whether the annotators thoroughly scanned the entirety of Wikipedia.

Author response:
The annotators read relevant Wikipedia articles and perform verification through a combination of human understanding and text searches. To enhance coverage, they are asked to examine various articles: (i) articles of all entities in the contradictory sentences; (ii) articles of relevant entities mentioned in (i); (iii) If non-English articles provided more information than English counterparts, the annotators used Google Translate to review non-English versions translated to English. To improve annotation accuracy, each piece of contradictory information must be examined by two annotators independently. Then, the two annotators discuss to resolve inconsistent cases and reach final decisions.

We believe that the above protocol has resulted in high-quality verification annotations. We will incorporate this discussion into the paper.

UPDATE: As suggested by reviewer 3teD, we additionally ask the annotators to use web search and examine broader online text. However, since Wikipedia already has a high coverage, the unverifiability numbers do not change significantly and our claims still hold.

Reviewer comment (Experiments):
The proposed iterative mitigation algorithm (Algorithm 3) requires an input n, which represents the number of mitigation iterations. The value of n seems to fluctuate based on the size of |x|. The paper should elucidate the chosen value for n and expound on how mitigation performance is influenced by this value.

Author response:
The initial submission includes an ablation study on this. Please refer to the paragraph “Ablation Study on Mitigation” in Appendix B. Our ablation study shows that each iteration progressively removes self-contradictions, while preserving fluency and informativeness. The mitigation converges quickly after three iterations, which is why we choose $n=3$.

We believe that there is a misunderstanding. The value of n is fixed at $n=3$ throughout all experiments presented in the paper and we do not intend to suggest at any point that n should be adapted based on |x|.

Reviewer comment (Experiments):
How does the annotator manually verify contradictory information using Wikipedia?

Author response:
The annotators read relevant Wikipedia articles and perform verification through a combination of human understanding and text searches. To enhance coverage, they are asked to examine various articles: (i) articles of all entities in the contradictory sentences; (ii) articles of relevant entities mentioned in (i); (iii) If non-English articles provided more information than English counterparts, the annotators used Google Translate to review non-English versions translated to English. To improve annotation accuracy, each piece of contradictory information must be examined by two annotators independently. Then, the two annotators discuss to resolve inconsistent cases and reach final decisions.

We believe that the above protocol has resulted in high-quality verification annotations. We will incorporate this discussion into the paper.

UPDATE: As suggested by reviewer 3teD, we additionally ask the annotators to use web search and examine broader online text. However, since Wikipedia already has a high coverage, the unverifiability numbers do not change significantly and our claims still hold.