Self-contradictory Hallucinations of Large Language Models: Evaluation, Detection and Mitigation
Authors: Niels Mündler
Jingxuan He
Slobodan Jenko
Martin Vechev
Venue: ICLR 2024 poster (2024)
Keywords: language model
hallucination
trustworthy artificial intelligence
reasoning
Primary area: societal considerations including fairness, safety, privacy
TL;DR: We present a comprehensive analysis showing that state-of-the-art LLMs frequently produce self-contradictory hallucinations. We then design prompting methods that effectively detect and mitigate self-contradictions.

Abstract: Large language models (large LMs) are susceptible to producing text that contains hallucinated content. An important instance of this problem is self-contradiction, where the LM generates two contradictory sentences within the same context. In this work, we present a comprehensive investigation into self-contradiction for various instruction-tuned LMs, covering evaluation, detection, and mitigation. Our primary evaluation task is open-domain text generation, but we also demonstrate the applicability of our approach to shorter question answering. Our analysis reveals the prevalence of self-contradictions, e.g., in 17.7% of all sentences produced by ChatGPT. We then propose a novel prompting-based framework designed to effectively detect and mitigate self-contradictions. Our detector achieves high accuracy, e.g., around 80% F1 score when prompting ChatGPT. The mitigation algorithm iteratively refines the generated text to remove contradictory information while preserving text fluency and informativeness. Importantly, our entire framework is applicable to black-box LMs and does not require retrieval of external knowledge. Rather, our method complements retrieval-based methods, as a large portion of self-contradictions (e.g., 35.2% for ChatGPT) cannot be verified using online text. Our approach is practically effective and has been released as a push-button tool to benefit the public at https://chatprotect.ai/.

Published as a conference paper at ICLR 2024
SELF-CONTRADICTORY HALLUCINATIONS OF LLMS: EVALUATION, DETECTION AND MITIGATION
Niels Mündler, Jingxuan He, Slobodan Jenko & Martin Vechev Department of Computer Science, ETH Zurich, Switzerland wet.ettueuetttt.t...eeee ethz. ch, jingxuan. he@inf. ethz. ch, sj sjenko@student. ethz. ch, martin. vechevOinf. ethz ch
ABSTRACT
Large language models (large LMs) are susceptible producing text that contains hallucinated content. An important instance of this problem is self-contradiction, where the LM generates two contradictory sentences within the same context, this work, we present comprehensive investigation into self ceffccontraiittion for various instruction-tuned LMs, covering evaluation, detection, and mitigation. Our primary evaluation task is open-domain text generation, but we also demonstrate the upplicability of our approach to shorter question answering. Our analysis reveals the prevalence of coffcconraaiitionn,, c.g., in 17.7% of all sentences produced by ChatGPT. We then propose a novel prompting- based framework designed to effec- tively detect and mitigate seff-contradictions. Our detector achieves high accuracy, e.g., around 80% Fl score when prompting ChatGPT. The mitigation algorithm iteratively refines the generated text remove contradictory information while preserving text fluency and informativeness. Importantly, our entire framework is applicable black-box LMs and does not require retrieval of external knowledge. Rather, our method complements retrieval based methods, as large portion of seff-contradictions (e.g.. 35.2% for ChatGPT) cannot verified using online text. Our approach is practically effective and has been released as push-button tool to benefit the public at https: pettss//chatppe http:///hhatpottt.atat
INTRODUCTION
Large language models (large LMs) are pretrained on massive text corpora (Chowdhery et al., 2022; Zhang al., 2022: Touvron al., 2023a) and fine-tuned to follow human instructions (Ouyang et al., LMs, such as ChatGPT (OpenAI, 2023a), have demonstrated remarkable
2022). Instruction-tuna zero-shot capabilities in solving natural language tasks (Bang et al., 2023; Qin et al., 2023; Zhong et al., 2023; OpenAI, 2023b). Therefore, they are being increasingly integrated into various aspects of daily life, including online search and professional environments (Mehdi, 2023; NerdyNav, 2023; Spataro. 2023). However, generating nonsensical or unfaithful content, commonly referred to as hallucinations (Bang et al., 2023; OpenAI, 2023b; Borji, 2023). This raises significant concerns regarding the trustworthiness of LMs (Weidinger et al., 2021; Zhuo et al., 2023; Reuters, 2023; Bose & Shepardson, 2023). Detecting and mitigating hallucinations still remains open challenge (Ji et al., 2023), especially for state-of-the-art, proprietary LMs accessible only as black boxes (OpenAI, 2023a;b; Anthropic, 2023).
widely adopted LMs such ChatGPT and GPT-4 are prone to
Reasoning about Self-contradictory Hallucinations This work focuses on an important type of hallucination called celf-contradiction, which occurs when an LM generates two logically inconsistent sentences given the same context. Our key insight that self-contradiction can be leveraged to conveniently tackle non-factual model outputs. We exploit the fact that self- sef-ccotraadicton is guaranteed to reveal non-factuality, because two contradicting sentences cannot be simultaneously correct (Dowden, 2011). Moreover, removing conflicting information from a pair of contradictory sentences strictly decreases non-face -onttacuull,ty.. Importantly, both contradiction detection and removal can be accomplished by performing logical reasoning, particular strength of modern LMs (Liu et al., 2023). This frees us from relying on externally retrieved knowledge (Peng et al., 2023) or comparing tens of samples (Manakul al., 2023b), which can be difficult or too expensive in practice.

---

Published as a conference paper at ICLR 2024
Table Two seff-contradictory hallucinations generated by ChatGPT. Sentences marked with green color (resp., red color) are factually correct (resp., wrong). Our framework successfully triggers. detects, and mitigates both hallucinations. We provide additional, longer examples in Appendix C.
Prefix
The FlayStation (P84) is a home video gaze console developed by Sony. Released 2013, is the eighth generation of consoles in the PlayStation series. It 1a the fourth goneration of the PlayStation consale series. PlayStation (P84)
Sentence pair
Nitigation
2013.
Prefix
Guen Jorgensen is a retired professional tristhlete from the United Statee. She currently lives in Kinnesota with her husband, Patrick Lenieux, and their children. She currentlyy 1ives in Portland, Oregon with har husband and too children. currently with
Sontence pair
Nitigation
children.
Based on the above insight, we propose a three-step pipeline for reasoning about self-contradictions. To trigger celf-contrdiictions, we employ appropriate constraints to generate relevant sentence pairs. Then, we explore various existing prompting strategies to detect self-contradictions (Wei et al., 2022; Kojima et al., 2022; Wang et al., 2023). Lastly, develop an iterative mitigation procedure that makes local text edits to remove contradictory information, while preserving other important text qualities such as fluency and informativeness. Since framework operates through prompting, it is directly applicable to state-of-the-aa black-box LMs.
Significance of Self-contradiction We perform an extensive evaluation on four modern LMs: GPT-4 (OpenAI, 2023b), ChatGPT (OpenAI, 2023a), Llama2-70B- BhChtt (Touvron al., 2023b), and Vicuna- 13B (Chiang et al., 2023). We primarily focus on open-domain text generation, a task that involves producing long text using LMs' internal knowledge (Petroni et al., 2019; Cohen et al., 2023a), where trustworthiness is highly desired but challenging to achieve (Ji et al., 2023). Our evaluation highlights the significance of self-contradiction. Specifically, self-contradictions are prevalent across the evaluated LMs, e.g., in 17.7% of all sentences generated by ChatGPT in open-domain text generation. A large portion of these (e.g., 35.2% for ChatGPT) cannot be verified using Wikipedia or text obtained by web search. In other words, our work serves as valuable complement retrieval-basee approaches (Shuster et al., 2021; Peng al., 2023; Min al., 2023).
Effective Detection and Mitigation Our proposed framework is highly effective at detecting (e.g.. ~80% Fl score) and mitigating colfacontraiictions (c.g.. reducing up to of 89.5% coff-contrdicttion while maintaining text informativeness and fluency). In Table we present two instances of ChatGPT generated self-contradictions that are successfully triggered, detected, and mitigated by our method. In Appendix C, we provide various longer examples, both sentence and text levels.
Generality We further apply our method to question answering (Mallen et al., 2023). The results show that our approach accurately (e.g., 74.2% to 83.8% precision) detects significant number (e.g.. 12.7% 38. 88.0%) of self-contradictions, for both vanilla and aetrenel-augmeented question answering.
User-friendly Tool and Open-source Repository The effectiveness and generality of our frame- work underscore its practicality. We release a user-friendly tool that produces warnings for halluci- nations and automatically mitigates them, accessible https://chatpprotee. ai/. Our code and datasets are publicly available on GitHub at btttss://itthnn cww/eth cach-srr//Oe ari/ChatProtect.
2 RELATED WORK
In this section, we discuss works that are closely related ours.
Large Language Models Training modern LMs requires two steps: pretraining (Chowdhery et al., 2022: Zhang et al., 2022: Touvron et al., 2023a) and instruction-tunic -unnin (Ouyang et al., 2022). Commercial instruction-tuned LMs (OpenAI, 2023a;b; Anthropic, 2023) are typically proprietary and only provide black- box accesses. Starting from Alpaca (Taori et al., 2023), open-source instruction- tuned LMs have emerged, such as Vicuna (Chiang et al., 2023) and Llama2 (Touvron et al., 2023b). Instruction-turn tuned LMs are increasingly used in daily life (Mehdi, 2023; NerdyNav, 2023; Spataro, 2023), because they are powerful in text reasoning (Bang et al., 2023; Qin et al., 2023; Zhong et al.,
2

---

Published as a conference paper at ICLR 2024
2023; Liu et al., 2023). Comprehensive benchmarks, such as MMLU (Hendrycks al., 2021) and HELM (Liang et al., 2022), have been proposed to thoroughly assess LMs' overall capabilities. Our work extends beyond these by focusing specifically on LMs' power to address hallucinations.
Hallucinations in Natural Language Generation According to the survey by Ji et al. (2023), hallucination is a common issue in different natural language generation tasks, including translation (Zhou et al., 2021), summarization (Maynez et al., 2020; Kryscinski et al., 2020), dialogue (Dinan et al., 2019; Shuster et al., 2021), and question answering (Lin et al., 2022; Kuhn et al., 2023; Cohen al., 2023b; Mallen al.. 2023). Two other works have explored hallucinations in open-domain text generation (Lee et al., 2022; Manakul et al., 2023b). While Lee et al. (2022) require access to internals of training and inference. our work is designed to be applicable black-box LMs. Compared with Manakul et al. (2023b), our framework is significantly more accurate in hallucination detection (as shown in Appendix B) and additionally addresses mitigation.
DEFINING AND MOTIVATING SELF-CONTRADICTIONS
This section defines self-contradictioon of LMs and presents motivations for addressing them.
We consider a language model (LM) that generates text consisting of a sequence of sentences: [x1, d22 /x/]]- Typically, the text generation process is conditioned on user prompt that specifies the task to accomplish. For the generation of multiple sentences x, we denote this process as x LM(- p). Similarly, for single sentence æ, we use x LM(- p).
Language Models
Self-contradiction of LMs We examine pair of sentences (r, x') generated by the LM. For x and x' to be contradictory, they must describe the same subject. Therefore, we constrain their generation by providing a context e in the form of a prompt. That is, æ ~ LM(- and x' LM(- c). We define 2') ) to be a self-contradiction of the LM when the two sentences are logically inconsistent. Our definition requires both sentences be generated from the same LM, in contrast prior works that use external sentences (i.e., those not generated by the LM) for hallucination evaluation (Elazar al., 2021; Azaria & Mitchell, 2023). As opposed self-consistency over different chain-of-thoo of-thoughtt reasoning paths (Wang et al., 2023), our definition captures factual statements in broader contexts.
Self-contradiction vs. Non-factuality When x and x' contradict each other, at least one of them guaranteed to be factually incorrect (Dowden, 2011). Based on our annotations, in 63.1% of the self-contradiction generated by ChatGPT, even both sentences are non-factual. Since both æ and x' originate from the same LM, their contradiction must expose the LM's non-factuality. Exploiting this insight step further, removing conflicting information from pair of contradictory sentences strictly decreases non non-facture factuallty.. These key properties empower us to detect and mitigate non-factuality via self-contradiction, using only two sampled sentences each time.
Like ours, related works leverage sampling- 2023; Manakul et al., 2023b; Cohen et al., 2023b). However, they do not benefit from the insights discussed above regarding the connection between non-factuality and self-contradiction. As a result, they require tens of samples to achieve reasonable detection accuracy and cannot handle mitigation.
methods to detect non-factuality (Kuhn et al.,
ceff-contradiction vs. Knowledge Retrieval One common approach for addressing hallucinations depends on the retrieval of external knowledge to identify non-factual content or guide factual text generation (Dinan et al., 2019; Shuster et al., 2021; Peng et al., 2023; Min et al., 2023; Mallen et al., 2023). However, effective knowledge retrieval challenging and costly in practice (Zhou et al., 2022: Ji et al., 2023; Mallen et al., 2023). Meanwhile, the detection and mitigation of coltacontrai can be achieved solely through logical reasoning, a particular strength of the latest LMs even in zero-shot manner (Liu et al., 2023). Indeed, our evaluation shows that significant portion of real-world self-contradictions, e.g., 35.2% for ChatGPT, cannot be verified or refuted using relevant online text. Moreover, demonstrated in Section 6.3, our approach is able to detect a large number of self colfaccotraiissoon even in retrieval- auggeente generations (Mallen et al., 2023). This makes our work valuable complement retrieval-bace approaches.
3

---

Published as a conference paper at ICLR 2024
4 TRIGGERING, DETECTING AND MITIGATING SELF-CONTRADICTIONS
We consider two LMs, gLM, which generates text x 1 22.1.22..... , xxx] and aLM, an analyzer LM. In this section, describe our algorithms for triggering self-contradictions of gLM, as well for detecting and mitigating them using aLM. From the algorithms, we decouple four important utility functions that involve solving different text reasoning tasks. Below, we provide generic definitions of these utility functions. In Section 5, we discuss their concrete implementations.
extract_ prrract_contextstz;; x): extracts a list of contexts for sentence 2,. possibly using metadata information from x, such as the entity of x.
gLM.gen_ sen_eeentec)((): queries gLM to generate a new sentence x' compatible with context c.
aLM. detect(zx, z/z c): invokes aLM to predict if z; and z are contradictory within context
aLM. die c): receives a pair of contradictory sentences (ær, ær)) within context The method invokes aLM to generate a revised version of z; where the conflicting information between I; and o is removed. The revised sentence also expected to retain as much non-conflicting information as possible and to be coherent with context c.
Trigger In typical usages, gLM generates each sentence only once. To trigger self contradiiad,,,,ns, we need to uppropriately query gLM to generate second sentence to form pair. We develop Algorithm I to achieve this purpose. At Line 2, it iterates over the sentences of x. For each sentence T;, it calls extract, contexts obtain list of contexts (Line 3). Then, Line 4, it runs gLM. gLM.gen_sseee senteneee that returns an alternative sentence x! that aligns with context c. At Line 5, the resulting sentence pair (æ,, x() and its context e are yielded following Python semantics. The context plays a crucial role in the generation of zi. Our goal is for c to effectively constrain x' to have the same scope as T;, Meanwhile, we desire c to provide an appropriate level of freedom, allowing gLM to generate 2 that contradicts Z4. We elaborate on achieving this dual objective in Section 5.
Detection Detecting self-contradictions is done by calling aLM. detect on the output of Algo- rithm 1. This procedure is closely related to natural language inference (NLI) (Bowman et al., 2015; Dagan et al., 2005), which involves predicting the relationship between a premise and a hypothesis: entailment, neutrality, or contradiction. Our detection differs from general NLI: (i) we require that both sentences are generated from the same LM; (ii) we consider the context in which the two sentences appear during detection; (iii) our detection output is binary: contradictory or not.
Mitigation Our mitigation approach performs local revisions of all sentences in x using the process shown in Algorithm 2 to remove self-contradictory information. If aLM. detect predicts a self- contradiction, we utilize aLM. revise eliminate the contradictory information from the sentence pair. We do not change sentences without detected self-contradictions to maintain other important text qualities of x, such fluency and informativeness.
To further minimize ceff-contradictions, we repeat the above step on the updated version of x, creating an iterative procedure outlined in Algorithm 3. In each iteration, Algorithm for all sentence pairs (z;, xt) and contexts e (Line 6). The output of mi mitigate_orr _eeeateeeeeeeeeeeteeeeeeeee is re-assigned x; and ultimately written back into the text at x. After a certain number of iterations (e.g.. 3 in our experiments), the mitigation process is expected to converge, even though there may be a small number of remaining self-contradictionn identified by aLM. detect. In such cases, we remove the corresponding sentences. This generally does not compromise the quality of the resulting text, as shown in Table of Appendix B. After this step, all predicted ceff-contradictions are eliminated.
invoke mi mitigate_aeee one from
INSTANTIATION TO TEXT GENERATION TASKS
Our framework built on prompting, making it suitable for state-of-the-art and proprietary black-box LMs. In this section, we instantiate these prompts on our primary evaluation task, open-domain text generation (Manakul et al., 2023b; Lee et al., 2022), where we ask LMs to generate long text with dozens of facts for various entities. Focusing closely aligns with the practical use of modern LMs in generating text across diverse domains (Mehdi, 2023; NardyNav, 2023; Spataro, 2023); (ii) it is well-suited for analyzing seff-contradiction it.
this task offers three key advantages: (i) it
4

---

Published as a conference paper at ICLR 2024
Algorithm 1: Triggering self-contradictions
Promp::
the start description about Millian Preenss:
for text generated by gLM.
Villin
field
procedure trigger(x) input text generated by gLM.
Intells gence (AT) conputer visten.
Please geserate the next sent ence of this description. penerated
output: sequence of tuples (zi, atd, c). z;
fill gap this triple:
(Re: bas bornl;
do
for c: in extract_ cecrract_contextarr.... x) do
gll: vas barn in 1980.
a!} gLM..ge_sentten gen_sentence(() yield (Ii, x1. c)
Figure 1: Our prompt for gen. senteeenc..
5
Proup::
Here the
description Millian Preenan.
Algorithm 2: Mitigating self-contradictions for pair of -eM--geeettedd sentences.
Then follow two statament..
Description:
Willian Fresran ix renouned researcher the field Artificia? Intelligence (AT) and cosputer vislos.
procedure mitigate. __ee(_, v/. c) input (zz, m)), pair of gLM-generated sentences coherent with context e output: a revised version of z; with self-contradictions removed. if aLM, MLM.detect(as, xt, c) then return aLM reviseeiee,,,, e).c)
He vas born on Angust 15, 1965, in the United States
Staterent
Please azplain if the statements about Willian T. Fresras controcictory. Provide your explanation
else return 24
alM:
The statenents about are contradictory. Staterent 1 states that be born Angust 15, 1955, that
statement These teo statements cannct both be tree at the same tine.
Algorithm 3: Iterative mitigation of self- contradictions for generated by gLM.
Promp:: Please conclude with
aiM: Yes.
procedure mi sitigate_iter(( n1)
Figure 2: Our prompt for aLM detect.
input gLM--generatd text.
the number of mitigation iterations. output: revised version of with greatly
Promp::
Baru is the atazt of descráption about Millian Freunan.
reduced ceff-cotradiictons.
resovzed researcher the Arbsficial Intelligonce (AI) and conputer vision
repeat n times with = =0
for 2; in x do for in wwwract_tccctttceccc contexte(25, x) do
Original Seatence folloved descriptien: August 1965,
States
5
x! gLM.gen_sentence((
Hosever, there is contradiction with this sentence: 1960.
= wettigateeoneeeeee 4/,
y- append(z;;)
Remove the conflicting information from the sentence.
Nainrain other information and text fisency.
==y
States
return x
Figure 3: Our prompt for aLM revise.
requires LMs to untangle their internal knowledge (Petroni al., 2019; Cohen al., 2023a); (iii) is challenging to deal with long text (Ji et al., 2023). At the end of this section, we also discuss the generic instantiation of our approach to other tasks, such as question answering.
Generating Initial Text First, we query gLM to generate text x for user-given prompts. For our open-domain text generation task, the user prompt takes the form "Please tell me about t", where t an entity from certain domain. After receiving the user prompt, gLM generates a sequence of sentences that reflect its internal knowledge about / in an encyclopedic style.
Defining Context Given the gLM-generated text x and a sentence in we use the notation X to refer to x; 's preceding sentences, that is, X = [ 22. xt-1]. We define the context of a; to be a tuple of three elements. The first two are the entity i of and the prefix X The third element a relation triple (s, r, 0) extracted from I; using an information extraction (IE) system (Banko et al., 2007). We choose CompactlE (Bayat et al., 2022) because it is suitable for open domains and constructs triple elements from spans within the input sentence. Note that CompactlE is small BERT model with 110M parameters that does receive any other external information input.
5

---

Published as a conference paper at ICLR 2024
Trigger To instantiate gLM. C.MM.gen_senteete gen_sentence((), format into prompt. An example of our prompt is shown in Figure (shortened for presentation purposes; the full prompt can be found in Appendix D). It includes c's elements, such as the entity & (William T. Freeman, professor from MIT EECS), the prefix x and the first two items of the relation triple (s: He and was born). These elements serve to constrain the scope of the output sentence x' Importantly, we omit the third element of the relation triple, introducing degree of freedom for gLM's generation process. The prompt then resembles a cloze test where the LM fills in the blank based on its internal knowledge (Petroni al., 2019; Cohen et al., 2023a). To make gLM generate strictly one sentence, we change the system message and provide few-shot demonstrations. Given the prompt. gLM (in this case ChatGPT) successfully generates a sentence that aligns with the context, which highlighted in red color. In Table 8 of Appendix B, we run alternative prompting strategies for gLM. FLM.gen_see ren_seenence, such as rephrasing (over-constraininng) and asking for continuation (under-constraining). Our prompt significantly outperforms all baselines because it enforces an appropriate level of constraint.
Detect We leverage zero-shot prompt to instantiate aLM. detect(r, xí,, c). This prompt includes the entity 8. the prefix x and the sentence pair (z,, zz)). We apply mhain-of-thoughe prompting (Wei et al., 2022), asking aLM to provide first an explanation and then a conclusion. This enables the model to decompose the complex problem of contradiction detection into smaller reasoning steps. In practice, the generated explanation, together with z; and x', can be returned to the user to improve the interpretability
the detection result. The binary conclusion (Yes or No) facilitates convenient parsing of the final detection result. An example of such a prompt is illustrated in Figure 2. The two gLM- -eM-generated sentences result in different birth years for William T. Freeman, clearly indicating contradiction. aLM (ChatGPT) correctly predicts the inconsistency for this case. According to Freeman's 's Wikipedia page (Wikipedia, 2023b), he was born in 1957, so both sentences are factually incorrect. In Table of Appendix B, examine other prompting strategies (Kojima al., 2022; Wang et al., 2023) and find that chain-of-thought outperforms them.
Revise To instantiate aLM. revise(z, z/ c), we stick to zero-shot prompting. Given the entity t and the prefix X-g. our prompt instructs aLM to remove the conflicting information between xy and ry We also provide specific instructions in the prompt such that the output is informative and coherent with the context. An example is shown in Figure 3. aLM (ChatGPT) successfully revises the sentence by eliminating the problematic birth date, and returns faithful and fluent sentence.
Generic Instantiation The only component specific to open-domain text generation within the prompts in Figures to is the first sentence (underlined in the figures). To adapt our approach to general tasks expressed by free-form user-provided prompt p. replace the sentence with generic "Here is the start of an answer to the prompt p". In the case of question answering, p takes the form of question, such "What is the birthplace of William T. Freeman?".
EXPERIMENTAL EVALUATION
In this section, we present an extensive evaluation of our work on open-domain text generation (Sections 6.1 and 6.2) and question answering (Section 6.3).
6.1 EXPERIMENTAL SETUP FOR OPEN-DOMAIN TEXT GENERATION
Models We experiment with four popular instruction-tumed LMs: GPT-4 (gpt-4-0314) (OpenAI, 2023b), ChatGPT (gpt-3.5-turbo-030) (OpenAI, 2023a), Llama2-70B-CII (Touvron et al., 2023b), and Vicuna- 13B (version 1.1) (Chiang et al., 2023). We evaluate these LMs as both gLM and aLM.
To construct evaluation data for open-domain text generation, we sample gLM to generate encyclopedic text descriptions for Wikipedia entities. Our primary test set, referred to as MainTestSet, consists of 360 descriptions spanning the four gLMs mentioned earlier and 30 diverse entities. For each gLM and each entity, we collect three different descriptions to enhance the reliability of our evaluation. In Table 2, show that the generated descriptions are long, consisting of between 7.3 and 12.5 sentences on average. sentence contains 1.6 fact triples on average. Table also shows the average perplexity of the descriptions, measured using OPT-1.3B (Zhang et al., 2022).
Data and Annotation

---

Published as a conference paper at ICLR 2024
The entities carefully chosen from multiple Wikipedia-basee sources (Lebret al., 2016; Wiki- media, 2023; Wikipedia, 2023a) to capture different domains and varying levels of popularity. In Figure 4 of Appendix A, we list the 30 selected entities, highlight their diversity in domains and popularity, and detail our selection procedure.
Three authors (Ph.D. Master's students in CS) perform human annotation to obtain the ground truth for MainTestSet. To improve knowledge coverage, our protocol instructs the annotators to review various online resources. Moreover, ensure annotation quality, each sample undergoes independent annotation by two annotators. Then, they discuss to resolve mistakes and inconsistencies. We detail our annotation process in Appendix A. We achieve a Cohen's Kappa value of 88.9% and 82.7% for annotating colf-contraiiction,, which indicates nearly perfect agreement (Cohen, 1960).
For validation, we use 12 entities. We compile second test set called 2ndTestSet, which contains a comprehensive set of 100 entities. For this set, we report the prediction results in Appendix B.
Evaluation Steps and Metries Our pipeline consists of triggering, detecting, and mitigating self-contradictions.. To evaluate these steps, we leverage the following metrics:
Trigger: We Algorithm to trigger self cafaccontraiiiions.. We calculate the frequency of self-contradictions: the ratio of sentences for which at least one celf-contradiction triggered. Then, we evaluate how self-contradiction complements retrieval-based methods (Shuster et al., 2021; Peng et al., 2023; Dinan et al., 2019; Min et al., 2023). To this end, we manually check if the contradictory information can be verified using Wikipedia or text obtained through web search. Our protocol for ensuring high-quality manual verification can be found in Appendix A.
Detection: We run aLM. detect on the result obtained the previous step. Then, we calculate and report standard classification precision (P), recall (R), and Fl score.
Mitigation: We run Algorithm on the gLM-generated text to obtain a revised version. To evaluate this step, we compare the original and the revised texts on the ratio of removed self- contradictions, informativeness, and fluency. When sentence does not induce contradiction, we consider as informative. The informativeness of a revised text is determined by comparing the number of informative sentences in the original text with the revised text and calculating the ratio. Note that this ratio might exceed 100% as our mitigation can produce new informative sentences. For fluency, we compute the increase in perplexity from the original text to the revised text.
Sampling Temperature We leverage sampling for LM decoding, where temperature is an important parameter (Holtzman et al., 2020) and our evaluation involves multiple decision points for temperature. The ablation studies in Appendix B show that our results are robust to these choices.
We fix the temperature for the experiments in this section to investigate the effect of other variables. When generating the text descriptions, we use temperature 1.0 because it aligns with practical aspects of L.M usage: (i) the default temperature in OpenAl APT is 1.0 (OpenAI, 2023c), and (ii) users have the option to check multiple responses, highlighting the importance of sampling diversity (OpenAI, 2023a). When running gLM. gen_ sentence, aLM detect, and aLM. revise, we use temperature 0 for ChatGPT and GPT-4. This is because we desire maximum confidence for these functions. For Llama2-70B-Chal and Vicuna-13B, we have to use temperature 1.0 avoid repetitive text.
6.2 EVALUATION RESULTS FOR OPEN-DOMAN TEXT GENERATION
Self-contradictions are Significant Table 2 highlights the significance of self-contradictiion. First, our approach triggers prevalence of ooffccntraaictions, ranging from 15.7% to 22.9% of all sentences depending on the gLM. The more powerful the model, the fewer self-contradictions generates. For instance, GPT-4 has the lowest ratio of seff-contradictoons In Figure 5 of Appendix B, we show that these self-contradictions spread across popular and lesser-known entities, instead of concentrating on small subset of entities. Moreover, a substantial portion of these self-contradictions (e.g., 35.2% for ChatGPT) cannot be verified using online text, making them highly unlikely to be resolved by retrieval-baie baseed techniques (Shuster et al., 2021; Peng et al., 2023). Thus, work stands as a valuable complement to retrieval- based methods. Vicuna- 13B's 's self-contradiction are often about basic facts that easily verifiable online, while other LMs produce more nuanced information.
7

---

Published as a conference paper at ICLR 2024
Table 2: Statistics of generated open-domain text descriptions and triggered self- contradictions, for different gLMs on MainTestSet.
gLM
avg. num. sentences
perplexity
self-contra. triggered (L)
unverifiable online
GPT-4
12.5
7.02
15.7%
27.5%
ChatGPT
9.6
5.72
17.7%
35.2%
Llama2-70B-Chh
11.3
5.44
19.0%
30.1% %
Vicuna-13B
7.3
6.82
22.9%
20.5%
Table 3: Results of using the same aLM (ChatGPT in this case) to detect and mitigate self- contradictions produced by different gLMs on MainTestSet.
Detection
Mitigation
gLM
(1)
R (t)
(1)
self-contra.
removed (t)
informative facts retained (1)
perplexity
increased (L)
GPT-4
80.1% 1%
79.7%
79.9%
76.3%
99.9%
0.67
ChatGPT 84.2%
83.2%
83.7%
89.5%
100.8%
0.44
Llama2-70B-CMn
72.9%
89.0%
80.2%
85.4%
95.5%
0.86
Vicuna-13B 74.4%
83.2%
78.6%
82.7%
95.1%
1.78
Table Results of using different aLMs detect and mitigate self-contradictions produced by the same gLM (ChatGPT in this case) on MainTestSet.
Detection
Mitigation
aLM
(f)
(f)
F1 (f)
self-contra. removed (t)
informative facts retained (t)
perplexity increased
GPT-4
91.3%
82.1%
86.5%
79.7%
97.7%
0.52
ChatGPT
84.2%
83.2%
83.7%
89.5%
100.8%
0.44
Llama2-70B-On
95.3%
45.8%
61.9%
44.4%
98. 88.8%
0.22
Vicuna-13B 90.0%
25.1%
39.3%
25.5%
100.2%
0.28
Our Detection and Mitigation Methods are Effective Next, we instantiate our method with Chat-
GPT as aLM. The results in Table 3 show that our approach is consistently effective across different gLMs. The detection achieves high Fl score ~80%. The mitigation is able to remove significant portion of seff-contradictions (76.3% to 89.5%) without harming text fluency (perplexity increase is small). also maintains informativeness, i.e., the number of informative facts maintained.
Performance Varies across LMs In Table 4, assess different aLMs in detecting and mitigating self-contradictions produced by the same gLM, ChatGPT. The results reveal that both proprietary LMs (GPT-4 and CharGPT) have strong performance. Open-source LMs (Lamm22-0BB-hha and Vicuna-13B) show limitations, particularly in low detection recall and inability to effectively remove seff-contradictions. Fortunately, state-of-the-an open-source instruction-tuned LMs have substantially advanced in recent months, with 2lamm22-70B-0- the-art) significantly outperforming Vicuna-133 (version 1.1, released in early May 2023 and then state-of-thhe of-te--f-te--rt according to LMSYS (2023b)). We anticipate that this positive trend will continue.
(relcased in late July 2023, current state-of-
Efficiency Analysis For a text x = [x1 32 ..., xx]] our method makes a constant number LM queries for each sentence 2; and the total number of LM queries grows linearly with |x|. For each 2; we include its prefix x in our prompts. Thus, the total length of the prompts grows quadratically with xx]. our experiments, we monitor the number tokens consumed by the OpenAl API, which reflects both speed and compute in practice. For ChatGPT, generating text description for an entity costs 259 tokens on average. Detecting and mitigating self-contradictions come with cost of 79x and 90x, respectively. Considering the price of 0.002$ per 1k tokens, the expense of detection and mitigation amounts to 0.04S and 0.05S, respectively. We consider these costs acceptable, especially for scenarios where high level of trustworthiness is demanded.
8

---

Published as a conference paper at ICLR 2024
Table Question answering using the PopQA benchmark (Mallen et al., 2023) with aLM ChatGPT.
Vanilla
Retricval Augmented
gLM
self-contra. predicted (L)
self-contra, predicted ())
P (T)
(1)
ChatGPT
18.2%
83.2%
12.7%
83.8%
Llama2-70B-O8 20h-hhat
38.0%
79.6%
21.5%
74.2%
More Details in Appendix In Appendix B, provide additional experimental results for open- domain text generation. We first show that self-contradiction is common across different entities. Next, we show that our detection method outperforms SelfCheckGPT (Manakul et al., 2023b). Moreover, we perform ablation studies to show the robustness of our results across different temperatures and to validate our design decisions discussed in Sections 4 and 5. Finally, we explore the relationship between the choice of gLM and aLM, and showcase the usefulness of our pipeline on 2ndTestSet.
In Appendix we provide more real-world examples on the sentence level and the result of the method on full-text examples. Further, we showcase a seff-contradiction that cannot be verified using online text, and a failure case of aLM. detect where aLM Vicuna- 133..
6.3 EXPERIMENTS ON QUESTION ANSWERING
We adapt our approach question answering as described the end of Section We focus on 1.5K randomly sampled questions from PopQA (Mallen et al., 2023). We perform experiments in two settings, vanilla generation, and retrieval augmentation, where generation and detection are provided with adaptively retrieved text snippets as proposed by Mallen et al. (2023). We use ChatGPT as aLM and the default temperatures (see the end of Section 6. To compute the precision of our detection method, manually annotate all sentence pairs that predicted to be contradictory.
The results for the two evaluated gLM, ChatGPT and Maamaa-77B-Chann are presented in Table 5. Even though self-contradictions decrease in the retrieval augmented setting by around 30% in ChatGPT and almost 50% in -lama2--7B-Chha a significant amount of seff-contradictions (12.7% to 38.0%) detected in all settings with high precision (74.2% 83.8%). This implies that with retrieval augmentation, at least 10.6% of ChatGPT and 16.0% of Mlama2-70B-Char generated answers contain self-contradictions.. A qualitative analysis shows that self-contradictions often arise when the retrieved knowledge is irrelevant to the posed question or lacks the crucial information for correct answer. This reaffirms our claim that our method serves valuable complement to retrieval sarieval-bbsed methods.
7 CONCLUSION AND DISCUSSION
In this work, we presented a comprehensive investigation into self-contradiction, an important form of hallucination produced by LMs. Self-contradictions arise when the LM generates two contradictory sentences within the same context, thereby revealing the LM's lack of factuality. To address this issue, we developed an algorithm trigger, detect, and mitigate self-contradictions. Importantly, our approach is built upon prompting strategies, making it applicable to black-box LMs without requiring retrieval of external knowledge. We conducted extensive evaluation four modern instruction- tuned LMs and two different tasks, demonstrating the benefits of our approach: it effectively exposes, accurately detects, and appropriately mitigates ciff-cotrraiittons..
Our framework currently handles the contradictions of two sentences sampled at the same text position. future work item to efficiently detect contradictions across different positions. Moreover, our approach serves as a post-processing method on LMs' output text. It is open to explore the potential of fine-tuning smaller open-source LMs to achieve high performance as aLM or even training LMs to avoid generating contradictory content. Finally, we note that other kinds of hallucinations than self-contradictions exist. For example, LMs can consistently generate incorrect content. As self- contradictions prevalent and our approach complements retrieval- based augmentation methods at hallucination detection, we believe that this work is significant and important for understanding and enhancing the trustworthiness of modern LMs.

---

Published as a conference paper at ICLR 2024
REPRODUCIBILITY STATEMENT
All our work is open source and available at https://github. cww/ethhssseeeeeeeee wwweeeetssscchtptreeeeeeeeeeeeee sri/chatprotect. The ex- periments were conducted against LMs that publicly available, either fully open-source through model parameters or via public APIs and we set the random seeds where possible to ensure repro- ducibility. Moreover, both the main paper and the Appendix, we give details on the experimental setup used, and how and which hyperparameters were selected.
REFERENCES
Anthropic. Introducing Claude, 2023. introducing- claude.
URL
Amos Azaria and Tom M. Mitchell. The internal state of an LLM knows when its lying. CoRR, abs/2304. 13734, 2023. URL https: httpaaaar..rraarra//a//aaaaa argg/bbs/3044 13734.
Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, Quyet V. Do, Yan Xu, and Pascale Fung. A multitask, multilingual, multimodal evaluation ChatGPT on reasoning, hallucination, and interactivity.
CoRR, abs/2302.04033 044020 2023. URL https: pectps://arttn //arxiv....i/////aa/a/i org/ oogg/ba////aeeeeeee abs/2302. 040233.
Michele Banko, Michael Cafarella, Stephen Soderland, Matthew Broadhead, and Oren Et- zioni. Open information extraction from the web. In IJCAI, 2007. URL http://ijcai org/ Proceedigss/77/PPpeers//// pdf.
Farima Fatahi Bayat, Nikita Bhutani, and H. V. Jagadish. Compactie: compact facts in open informa- tion extraction. In NAACL, 2022. URL https: /tto:::://oooooo////// http:///ooooor/1.../ 18653/11/01665/1///2/2 naacl-mall 65.
Ali Borji. A categorical archive of chatgpt failures. CoRR, abs/2302.03499, 2023. URL https:
//arxiv.orr owoo/ogg//a. 2322 03494.
Nandita Bose and David Shepardson.
Biden meets Microsoft, Google CEOs cow.eerss..cmmmeccooooggo/ - ggoggee---oasaa---gggr--00000-00
AI dangers, 2023. white-
URL
Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. A large annotated corpus for learning natural language inference. In EMNLP, 2015. URL https://dol: o0g/10.
18653/v1/ http////d111111///11
Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. Vicuna: an open-source chatbot impressing GPT-4 with 90%* ChatGPT quality, 2023. URL https:
//imsys. wwtgb//g/////ooooooooooooooooolo 22ap//2233-000 -0-2--03-0--100 300---vv0----vaaa
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: scaling language modeling with pathways. CoRR, abs/2204.02311, 2022. URL https://arxiv brg/abs//a /2004. 02311.
Jacob Cohen. A coefficient of agreement for nominal scales. Educational and psychological measurement, 20(1):37 1960.
Roi Cohen, Mor Geva, Jonathan Berant, and Amir Globerson. Crawling the internal knowledge-base of language models. In Findings of EACL, 2023a. URL htt.hht:/acclattholllaeee wwwp:atwooooooooooooooooe findings-eacl. 139.
022/2033.
Roi Cohen, May Hamri, Mor Geva, and Amir Globerson. LM vs LM: detecting factual errors via cross examination. CoRR, ftns/2355.32811 2023b. URL.
wttp://wwww...//////....../ 13281.
Ido Dagan, Oren Glickman, and Bernardo Magnini. The PASCAL recognisiig textual entailment challenge. In MLCW, 2005. URL httph:tt//:/tte //doi. 0tt.://0rr//011100/177777775735
10

---

Published as a conference paper at ICLR 2024
Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, and Jason Weston. Wizard
of wikipedia: showledge-powered conversational agents. ICLR, 2019. URL https:// hwtpr///www.ntttfooor///d11111/.
Bradley H. Dowden, Logical Reasoning, chapter 9, pp. 283-315, 2011. URL https://www... sfu. ca/" start///ggccalrraaaonnng//oooooo
200pp2283-200 283-315, pdf.
Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhilasha Ravichander, Eduard H. Hovy, Hinrich Schütze, and Yoav Goldberg. Measuring and improving consistency in pretrained language models. URL https://ddee /doii. org/10.1162
Trans. Assoc. Comput. Linguistics, 9:1012-1031, 2021.
tacl_a_0044
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In ICLR, 2021. URL https:
//openreview..oo mett//torumm20004/8888111333..1
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The curious case of neural text degeneration. In ICLR, 2020. URL https:// net//orumniddrryeQpppp
Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu. Dan Su. Yan Xu, Etsuko Ishii, Yejin Bang, Andrea Madotto, and Pascale Fung. Survey of hallucination in natural language generation. ACM Comput. Surv., 55(12). 2023. URL https://dooi. http://///..oooo...../ 1144//5571330
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. In NeurIPS, 2022. URL https://arxiv. org/abs/ 2205. 11916.
Wojeiech Kryscinski, Bryan McCann, Caiming Xiong, and Richard Socher. Evaluating the factual consistency of abstractive text summarization. In EMNLP, 2020. URL https://doi. org/10. 18653/v1/2020, emnlp-me mmnlp-main.. 750.
Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation. ICLR, 2023. URL https://
openreview. wetpp/ddddddd--/-Yeeeeeee
Rémi Lebret, David Grangier, and Michael Auli. Neural text generation from structured data with application to the biography domain. In EMNLP, 2016. URL https://doi: org/10. /0/./11..0 18653/ v1/d16- 1128.
Nayeon Lee, Wei Ping, Peng Xu, Mostofa Patwary, Pascale N. Fung, Mohammad Shoeybi, and Bryan Catanzaro. Factuality enhanced language models for open-ended text generation. In NeurIPS, 2022. URL http:/ts////rtr //abxii////xi...//// org/ abs/ /ew.///....22.m. 04624.
Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, et al. Holistic evaluation of language models. CoRR, abs/2211.09110 2022. URL https://arxiv: http://aaxvvrrro//a///// argg/bs//2// 22111. 09110.
Stephanie Lin, Jacob Hilton, and Owain Evans. TruthfulQA: measuring how models mimic human falsehoods. In ACL, 2022. URL https: //doi 18g/10018865//122022 ac11 acl-long. 229.
Hanmeng Liu, Ruoxi Ning. Zhiyang Teng, Jian Liu, Qiji Zhou, and Yue Zhang. Evaluating the logical reasoning ability chatgpt and GPT-4. CoRR, abs/2304.03439, 2023. URL https //arxiv. how.:.org/aaaoo httg/abs/2000 ab3/ 03439.
LMSYS. FastChat GitHub, 2023a. URL http://gtttub.ccommmsssss/ss////
LMSYS. Chatbot arena leaderboard updates (week 2), 2023b. URL http::: 02-3-05-10-10 10-1eee-reaorrddd...
Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi. and Hannaneh Hajishirzi. When not to trust language models: Investigating effectiveness of parametric and non- parammett memories. In ACI, 2023. URL https: htttps://dol httt::::///o.oogg//../ orr. 18653/v11/2222 1all-oogg.555 546.
11

---

Published as a conference paper at ICLR 2024
Potsawee Manakul, Adian Liusie, and Mark F. Gales. MQAG: multiple-choice question answering and generation for assessing information consistency summarization. CoRR, abs/2301.12307, 2023a. URL httpttt/tariiiiii wttp:///rrg///////a/////o abt////...././//////.aaa...a.. 12000 12307.
Potsawee Manakul, Adian Liusie, and Mark J. Gales. SelfCheckGPT: zero-resource black-box hallucination detection for generative large language models. CoRR, abs/2303.08896, 2023b. URL 08896.
https://arxiv. http://aaaxio..raaaaaaa//
Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan T. McDonald. On faithfulness and factuality in abstractive summarization. In ACL, 2020. URL https://ddi.orr/// wtt...................... 2020 acl main. 173.
Yusuf Mehdi.
Confirmed: the new Bing runs OpenAl's GPT-4, 2023. //bloge bing. com/ -2X800/99s-GPT-4
URL https:
irwd/oooffo/dddeeeeeeeeoaon new-ning runs- on oneooonOOOppeee on-OpenAI%
Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Wei Koh, Mohit lyyer. Luke Zettlemoyer, and Hannaneh Hajishirzi. Factscore: Fine-gres tineigraieed atomic evaluation of factual precision in long form text generation. CoRR, Abs/2305.142511 2023. URL https btttps///arx: //arxiaa.vvv
org/abs/2305. 14251.
NerdyNav. ChatGPT statistics, 2023. URL htt.tpp/s/eeerraaaaccccoo////a/s
OpenAl. Introducing ChatGPT, 2023a. URL
OpenAl. GPT-4 technical report. CoRR, abs/2303.08877, 2023b. 2303.08774.
20236, URL htt.tttps.//arxw...orr///a/
OpenAl. OpenAI API, 2023c. URL http:p///ppaaatt..o..ppaaa..cc//
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang. Sandhini Agarwal, Katarina Slama, Alex Ray, al. Training language models to follow instructions with human feedback. In NeurIPS, 2022. URL https: //arxiv////xvvv///// org/ httg//b//22033 02155.
Baolin Peng, Michel Galley, Pengeheng He, Hao Cheng, Yujia Xie, Yu Hu, Qiuyuan Huang, Lars Liden, Zhou Yu, Weizhu Chen, and Jianfeng Gao. Check your facts and try again: improving large language models with external knowledge and automated feedback. CoRR, 124/23302.181133 2023. URL https: /ttp:pp::///iiiiii /arxivv...v org/ /232 12813.
Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick H. Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander H. Miller. Language models as knowledge bases? In EMNLP-IJCNLP, 2019. URL
https://doi.org 18653/v1//19
Chengwei Qin, Aston Zhang, Zhuosheng Zhang, Jiaao Chen, Michihiro Yasunaga, and Diyi Yang. Is chatgpt general-purpose natural language processing task solver? CoRR, abs/2302.06476, 2023.
URL https: //rxxvvxvvv////////v/////////aa/ org/ abs//30s/2 02302.06766.... 06476.
Reuters.
Google
cautions
against "hallucinating" chatbots,
port says, 2023. google-cautions- www.nnssagannntta.aaaa
URI. https: /tops://wste http://wwww.eeeeeeeeeeee
reuters. wwweerss....eeeeeeeeeeeeeeeeeeee cow/technooogy
Kurt Shuster, Spencer Poff. Moya Chen, Douwe Kiela, and Jason Weston. Retrieval augmentation
reduces hallucination in conversation. In Findings of EMNLP, 2021. URL. https: .rg/
10. ww.11655/11222111 http://////222.....nnggg.// findings---mmmmpp
Jared Spataro. work. 2023.
Introducing Microsoft 365
your copilot for URL https://blogs microsoft. hom//logg/223/0//66/ work.
Copilot
introducing wwsiinggmiico -onesoott3665..o wwt.offtt--6---oooo-tto- your
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford Alpaca: an instruction-folloiing LLaMA model, 2023. URL https://t///iiiee cow/tatsu- lab/stanford alpaca.
12

---

Published as a conference paper at ICLR 2024
TogetherAL. Together AI API, 2023. models inference.
URL https://ddos. aateerrai///oo ai/docs/
Hugo Touvron, Thibaut Lavril, Gautier Izacard. Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurélien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. LLaMA: open and efficient foundation language models. CoRR, abs/2302.13971, 2023a. URL https: pectps//arriic www./og./////ooooooooo....o....t aa////b//2/00 13971.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. CoRR, abs/2307.0922 2023b. URL https: //txx//xxvvv////////////a//aaav/ org/abs/ $307.09288
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, and Denny Zhou. Self- consistency improves chain of thought reasoning in language models. In ICLR, 2023. URL https://arxiv. httpp/////axi.org/bbb/////// wwwps/..orrbbb/////ee/// 11171.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc V. Le, and Denny Zhou. Ohain-of-thoughh prompting elicits reasoning in large language models. In NeurIPS, 2022. URL WRLLh:ts:///araee http///aaiii.orga/////////// 11903.
Laura Weidinger. John Mellor, Maribeth Rauh, Conor Griffin, Jonathan Uesato, Po-Sen Huang, Myra Cheng, Mia Glaese, Borja Balle, Atoosa Kasirzadeh, et al. Ethical and social risks of harm from language models. CoRR, abs/21 12.04359, 2021. URL owiimeedii..o////ttt///e//ae--ee /ttpo///ttt//////oaaaaaaaaaaaaaa
wwtp:/v..rrgbbb///122/ 04359.
Wikimedia. Wikipedia titles, 2023, URL https: //dumps://d/pp...ee
Wikipedia. Wikipediar.eattreed articles/by length, 2023a. URL htttp:://en.wikippeee org/
wiki//Wikipedia: Feature /et.ap/ppeeppffeuuuueeetrtttt/// EttiiceesBy__enngtee
Wikipedia. William T. Freeman Wikipedia, 2023b. URL httttp::/eenwiikippdia.rrr////
William_ Freeman..
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona T. Diab, Xian Li, Xi Victoria Lin, et al. OPT: open pre-trained transformer language models. CoRR, abs/2205. 1is/2205500068, 2022. URL https://arxiv.ortt
a22/22///222222. 01068.
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. Bertscore: evaluating text generation with BERT. In ICLR, 2020. URL https: nwt/forumm/eeeccc SkeHuCVFDr.
Qihuang Zhong, Liang Ding, Juhua Liu, Bo Du, and Dacheng Tao. Can chatgpt understand too? A comparative study chatgpt and fine-tuned BERT. CoRR, abs/2302.10198, 2023. URL https://arxiv. wtg/abs/////ooooooogo...oo....at 2302 10198.
Chunting Zhou, Graham Neubig, Jiatao Gu, Mona T. Diab, Francisco Guzmán, Luke Zettlemoyer, and Marjan Ghazvininejad. Detecting hallucinated content in conditional neural sequence generation. In Findings of ACL, 2021. URL https: //t.///o.ii/o 100gg/10.. 18653/6663vv
findings-saal aaainngs-cll..... 120.
Pei Zhou, Karthik Gopalakrishnan, Behnam Hedayatnia, Seokhwan Kim, Jay Pujara, Xiang Ren, Yang Liu, and Dilck Hakkani-Tun Turr Think before you speak: explicitly generating implicit commonsense knowledge for response generation. In ACL, 2022. URL https://doo org/10. 18653/v1/ 2022. acl long. 88.
Terry Yue Zhuo, Yujin Huang, Chunyang Chen, and Zhenchang Xing. Exploring Al ethics of chatgpt: A diagnostic analysis. CoRR, abs/2301, 12867, 2023, URL https:/ http///rxvvvv.. oeriiiiorrg// org/abs/200 12867.
13

---

Published as a conference paper at ICLR 2024
A MORE DETAILS ON EXPERIMENTAL SETUP
Selecting Diverse and Representative Entities To construct our test sets, we select entities from Wikipedia titles. We carefully assemble the test sets to ensure the diversity and representativeness of the entities. The majority of entities are randomly sampled from the list of human biographies in WikiBio (Lebret et al., 2016) and the list of all Wikipedia articles (Wikimedia, 2023) (mostly non-human). This strikes a balance between human and non-human entities. We also randomly sample from the list of featured articles (Wikipedia, 2023a) to include more popular entities. After sampling, make sure that the selected entities cover different degrees of popularity and different domains. To measure the level of popularity, we quote the entire phrase of each entity and search on Google, which returns webpages that match the exact phrase. Moreover, we do not include entities where the evaluated LMs refuse to provide any useful information or return irrelevant text.
The 30 entities in MainTestSet and their statistics are presented in Figure 4. The 100 entities in 2ndTestSet and their popularity are listed in Figures 7 and 8. Our validation set consists of 12 entities: Angela Merkel, Augustin-Louis Cauchy, Eike von Repgow, Hernån Siles Zuazo, Marbel Arch, Marillac College. Mikhail Bulgakov, Okolo N N'Ojie, Ruy de Se- queira, Shahar Pe er, Spain, Tan Chorh Chuan.
Details on Human Annotation Process Our human annotation process aims to derive three labels: self-contradiction, factuality, and verifiability of contradictory information. We are aware that annotating these labels is challenging task, even for human beings. To enhance annotation quality, we ensure that each annotation decision is a joint effort of two annotators. Specifically, we have three annotators in total. One annotator annotates the entire dataset and the other two annotate two non-overlapping splits. After that, the annotators discuss together to fix mistakes and resolve inconsistencies. We find that this discussion step greatly boosts annotation quality. After the discussion, small number of disagreements remain due to their subjective nature and we use the first annotator's label as the final decision.
While the annotation of self-contradictions does not require external knowledge, annotating the verifiability of contradictory information requires the annotators to thoroughly scan the Internet to obtain all useful information. Since our test entities are all from Wikipedia, we first instruct the annotators to diverse Wikipedia articles: (i) articles of all entities in the contradictory sentences; (ii) articles of relevant entities mentioned in (i); (iii) If non-English articles provided more information than English counterparts, the annotators used Google Translate to review non-English versions translated to English. We find that Wikipedia has a high coverage of necessary information for verification, which also supported by a study by Min et al. (2023) (Appendix A.5 in their paper). For the minority of cases where Wikipedia is insufficient, extensively for important keywords such as mentions of entities and their relationships. Note that the purpose of this annotation is to evaluate how our work complements retrieval-based methods. Therefore, the annotators focus only on content that can be handled by existing retrieval-based methods (Shuster et al., 2021; Peng et al., 2023; Dinan et al., 2019; Min et al., 2023), excluding considerations of figures, images, files, etc.
ask the annotators to search the Internet
To annotate factuality, we ask the annotators check all relevant online resources they could find. If a piece of information can be supported by text on the Internet, we mark a sentence as factual. Otherwise, we label it as non-factual.
Compute For ChatGPT and GPT-4, we query the OpenAI API (OpenAI, 2023c). We run Vicuna- 13B with the FastChat API (LMSYS, 2023a) on NVIDIA A100 80GB GPUs. We use the service of Together AI (TogetherAl, 2023) for running Llama2-70B-Chat.
B MORE EXPERIMENTS ON OPEN-DOMAIN TEXT GENERATION
Distribution of Self-contradictions To provide insight into the distribution of self-contradictions, Figure 5 provides a breakdown of self-contradictions generated by ChatGPT among individual entities. It shows that self-contradiction is a common issue among different entities and popularity.
14

---

Published as a conference paper at ICLR 2024
Comparison with SelfCheckGPT We compare our method with SelfCheckGPT (Manakul et al., 2023b) on the task of detecting self-contradictions. SelfCheckGPT (Manakul et al., 2023b) leverages three black-box methods for detecting hallucinations. They are based on unigram probabilities, BertScore (Zhang et al., 2020), and multiple-choice question answering and generation (MQAG) (Manakul et al., 2023a). We perform thorough comparison between our prompting-basee detection approach with these three methods in two settings: (a) our setting of detecting self-contradictions of sentence pairs, and (b) their setting of detecting non-factual sentences given 20 alternative samples. We use ChatGPT as both gLM and aLM. To enable setting (b), we annotate the factuality of each generated sentence, sample 20 alternative sentences, and adapt our prompt in Figure 2. We change our prompts to output a score ranging from 0 to 10. Our prompt adapted for setting (b) can be found in Figure 16. The results are plotted Figure and show that our approach substantially outperforms all three variants of SelfCheckGPT, especially in our setting of detecting self-contradictions.
Robustness across Different Temperature Values We have tested temperature values 0, 0.25, 0.5, 0.75, and 1.0 for both gLM and aLM (ChatGPT for both cases in this experiment). The results show that the claims of our paper hold across different temperature values: (i) self-contradictions are consistently prevalent; (ii) our approach for addressing seff-contradictions is robust.
We first fix the temperature of aLM to the default value 0 and vary the temperature of gLM in generating the initial text. For each temperature, gLM produces new text descriptions, which require new expensive human annotations. To make human annotation feasible, we annotate all sentence pairs predicted by our detection method to be contradictory and report detection precision. The results are presented in Table 6. We find that both the ratio of sentences predicted be cellacontra and the precision of our detection are consistent across different temperatures.
Then, we fix the temperature of gLM to the default values and vary the temperature of aLM in detecting relf-contradictions. We reuse existing human annotations in this experiment. The results are presented in Table 7, showing that the precision, recall, and F1 scores have very small variance.
Ablation Study on Trigger We consider three alternative prompting strategies as baselines for gLM. sentence: (i) Continue: we directly ask gLM to generate continuation of x-; as xi-- This method imposes insufficient constraints on ay often resulting in non-contradictions where I; and z refer to different subjects. (ii) Rephrase:
provide gLM with x and 2;, and request gLM to rephrase I; to x/,. This strategy overly restricts x because rephrasing inherently preserves the semantic meaning of an. (iii) Q&&: provide gLM with x<i and rt, prompting gLM to generate questions regarding z4. We then instruct gLM to answer the questions. With this prompt, the level of constraint decided by gLM and is not controlled by us. We compare these alternatives with our prompt in Table 8. The results clearly show that our prompt outperforms the baselines, by triggering more self- contraddddiss and leading higher detection precision. This success can be attributed to the appropriate level of constraint our prompt enforces on the generation of z/c.
Ablation Study on Detection For aLM. detect, we compare our prompting strategy in Figure 2 with various baselines in Table 9. The first baseline involves straightforward prompt which directly asks aLM to answer Yes/No whether a sentence pair is contradictory based on the context. This baseline is inaccurate and cannot provide interpretability of the decision decision-making making process. Following Kojima et al. (2022), we prompt aLM reason step-by-step manner with a single query that elicits the model to print reasoning steps before coming to a definite conclusion. We find that step-by-step reasoning is not well applicable to contradiction prediction, where asking for concrete steps often leads the model to follow wrong reasoning paths. On the contrary, our chain-of-thoupe prompting strategy (Wei et al., 2022) asks the model in two turns first provide a free-form explanation only and then conclude with Yes/No, which boosts accuracy. We also explore the use of multi-pati reasoning (Wang et al., 2023), which extends over chain-of oh-thought prompting by querying aLM to provide multiple explanations before concluding via vojo We find that multi-path reasoning does not provide benefits over our prompt, while drastically increasing the cost.
Ablation Study on Mitigation In Table 10, we present a breakdown of the results obtained from running our iterative mitigation procedure in Algorithm 3. Each iteration progressively removes self-contradictions. After the third iteration, we remove the sentences corresponding to the remaining small amount of predicted seff-contradictions, This removal preserves fluency and informativeness.
15

---

Published as a conference paper at ICLR 2024
In contrast, baseline method that naively drops all predicted self-contradictions from the beginning results in significantly lower informativeness and fluency.
Relationship between gLM and aLM In Section 6.2, we present results on the performance each LM as either gLM aLM. We now further explore the performance between different combinations of gLM and aLM. To this end, we try all possible combinations of GPT-4 and ChatGPT as gLM and aLM. The results are presented in Table 11. We draw two observations. First, combining different gLM and aLM achieves a higher F1 score than using the same LM, underscoring the benefit cross-examination (Cohen et al., 2023b). Second, content generated by GPT-4 is difficult to classify: in terms of FI score, ChatGPT-ChatGPT outperforms GPT-4-ChatGPT and ChatGPT-GPT-44 outperforms GPT-4-GPT-4, We find that the differences in the sentence pairs generated by GPT-4 are often more nuanced and harder to identify.
Results on 2ndTestSet We now discuss the results of running our approach on 2ndTestSet. Given the large size of 2ndTestSet, we do not perform human annotation and only report prediction results, In this series of experiments, both gLM and aLM are ChatGPT. Overall, 12.7% of the sentences are predicted be self-contradictory. The breakdown plotted in Figures 7 and showing again that coffrcontraiittion is a common issue across entities. The ratio of detected self-contradictions negatively correlates with entity popularity. Our mitigation successfully removes all predicted self- contradictions, while maintaining informativeness (we retain 100.9% sentence pairs that are predicted to be non-contradictory) and fluency (perplexity increase is only 0.42).
C MORE REAL-WORLD EXAMPLES
Next, we provide more long examples for better understanding of the effectiveness of our approach.
Sentence-level Examples In Table 12, we present additional examples of self-contradictory sen- tence pairs. We can see that these LMs produce diverse hallucinated information, such as geographical location and wedding date. For all cases, our mitigation successfully eliminates eeff-contradiction and generates a coherent revision. It may happen that the first sentence in pair is factually correct, but the second one is factually wrong, e.g., the example for the entity "Black Mirror". Such cases reflect LMs' internal uncertainty, which is removed by our method to improve certainty and factuality.
Text-level Examples In Figures to 1, we show texe- examples. Each example consists of the original text and the revised text for a given entity. The examples are generated by different gLMs and revised by the same aLM (i.e., ChatGPT). Our method applies mitigation locally on sentences detected, keeping other sentences unchanged. Enforcing local changes
where coff-contradiction is important for maintaining the overall fluency and informativeness of the text. Our mitigation successfully removes self-contradictory information in all but one case (the purple sentence in Figure 10). In that case, the revised sentence has the same semantic meaning as the original sentence.
Self-contradiction Unverifiable using Online Text In Table 13, we show an example of self- contradiction for which neither the corresponding Wikipedia page nor results from web search contain relevant information. When prompting ChatGPT as aLM, our method detects this seff-contradiction and decides remove the sentence, which improves the overall factuality.
aLM detect is far lower than other considered LMs. To understand the reason, we inspect false negatives of Vicuna- 13B and compare its decision with ChatGPT's. We find that Vicuna- 1euuna-13B usually struggles to identify conflicting information. As a result, it tends to make up erroneous explanations that lead to false negatives. We provide such a case in Figure 12, where ChatGPT outputs the correct response in succinct and confident manner. In contrast, Vicuna- makes up three flawed reasons.
Failure Case of Vicuna-13B The detection recall of Vicuna-133
D COMPLETE PROMPTS
We now provide the full prompts of our method and the ablation baselines.
16

---

Published as a conference paper at ICLR 2024
Prompts of Our Method The full prompts of our method are shown in Figures 13 15. They respond to the shortened versions Figures to To ensure that gLM generates strictly one sentence in Figure 13, change the system message and provide the LM with three-shot demonstrations using entities "Douglas Adams", "Kayne West", and "Angela Merkel".
Our Prompt Adapted for Detecting Non-factuality To compare with SelfCheckGPT (Manakul et al., 2023b) in Figure 6b, we adapt our prompt for aLM detect to the task of predicting if sentence non-factual given first asks the model if there are contradictions between the original sentence and the alternatives. Then, queries the model to conclude whether the original sentence is factually correct.
alternative samples. The adapted prompt shown in Figure
Prompts of Ablation Baselines In Figures 17 19, provide the prompts for the baselines for gLM.gen_ seeneeeneeceeeeeeeeeeeeeeee in Table 8. For "Continue" (Figure 17) and "Q&A" (Figure 19), we generate Two alternative sentences match the number of sentence pairs produced by method. For "Continue" (Figure 17) and "Rephrase" (Figure 18), we use the same three-shot demonstrations as our method in Figure 13. The prompts of the baselines for aLM detect Table are presented in Figures 20 to 22. The baseline "Multi-path" in Figure 22 uses the same prompt as our method in Figure 14. The difference is that "Multi-path samples 5 explanations with temperature 1, while our method generates one explanation with temperature 0.
17

---

Published as a conference paper at ICLR 2024
Vaivefieid,
Architecture
Artis
Historical
Polinies
Dolog, Pereeus (constellsston).
Mick dazzia, Love, Inc. (TV Beries), Latri Elvan, John Holt (author), Diane Ledley, Seguin, Ridonhouz, Baglish Arts and Crafts archa Lecture, Doawy Parker (actor), Auguzca Cl8singe3,
10a
Sports
Television
Otteers
10"
Bavlings.
of Google
10a
Bordes Colleh, Todo Charptonshipe kg, Bellsaghürden, Baloas Molla, Thiesy= First Arsy (Japan), Prince Albart relstiong, Bailay-Theeee Thergaon House, Roologist Party for the Developsen: of Buckina, Patsisura Stadiux, T. Blackfan, 1990 91 Austria Leagus
Enilies MainTestSet
Figure 4: The entities in MainTestSet (left). They have dif- ferent levels of popularity according the number of Google research results with exact phrase match (the bar chart). They also cover different domains (indicated by the different colorss.
45
# of sontences
15
3
3
Entities in Main TeatSet
Figure Breakdown of ChatGPT-gennratee coffacontras (red) and con-contradictory sentences (green). The entities are sorted by their popularity, i.e., the same order as in Figure
Precision
Procision
(AtIC-PR-87)
100
(AUC-PR=700.
100
SeUCCE-MQAG
75
SeIfCK-MQAG
75
(AUC-PR=31.0)
(AUC-PR-61.9)
SelCCk BertScore
25
SeliCk Berreer (AUC-PR-53. 5)
25
(AUC-PR=16.6)
SelICL-Unigram
25
75
100
Sallk-Unngram (AUC: PR=S2.7)
50
100
(AUC- PR=12.9)
Recall
Recall
(a) Our setting: detecting colf-contraiictions among pairs of sentences.
(b) SelfCheckGPT's setting: detecting non- factual sentences given 20 alternative samples.
Figure Comparison between our work and SelfCheckGPT (Manakul al., 2023b) in two settings.
18

---

Published as a conference paper at ICLR 2024
Table 6: Different choices of temperature values for gLM during text generation.
self-contra. predicted
temperature
P (1)
0
17.4%
84.1%
0.25
17.8%
81.3%
0.5
17.2%
79.2%
0.75
16.6%
83.3%
18.3%
83.4%
Table 7: Different choices of temperature values for aLM detect.
temperature
P (1)
R (1)
Fl (1)
0
84.3%
83,8%
84.0%
0.25
83.4%
84.4%
83.9%
0.5
83.3%
83.8%
83.6%
0.75
80.9%
82.7%
81.8%
84.2%
83.2%
83.7%
Table 8: Ablation study gLM.gen_sentence
self-contra.
triggered (t)
(
Continue
5.1%
76.8%
Rephrase
0.2%
33.3%
Q&A
10.3%
81.9%
Ours
17.7%
84.2%
19

---

Published as a conference paper at ICLR 2024
Table 9: Ablation study on aLM detect.
P (T)
R (1)
FI (1)
Directly ask
83.0%
65.4%
73.1%
Step-by-step (Kojima et al., 2022)
89.0%
54.2%
67.4%
Multi-path (Wang et al., 2023)
86.9%
77.7%
82.0%
Ours
84.2%
83.2%
83.7%
Table 10: Ablation study on our mitigation algorithm in Algorithm
self-contra. removed (t)
informative facts retained (f)
perplexity increased (U)
Original
0%
100.0%
Iteration
69.3%
101.6%
0.42
Iteration 2
74.5%
102.2%
0.45
Iteration 3
75.8%
102.3%
0.46
Our final result
89.5%
100.8%
0.44
Naively drop
94.8%
89.4%
0.72
Table 11: The relationship between gLM and aLM for aLM, detect.
gLM
aLM
(t)
(1)
Fl (f)
CharGPT
ChatGPT
84.2%
83.2%
83.7%
ChatGPT
GPT-4
91.3%
82.1%
86.5%
GPT-4
ChatGPT
80.1%
79.7%
79.9%
GPT-4
GPT-4
88.3%
65.7%
75.3%
20

---

Published as a conference paper at ICLR 2024
Entities
of Google search results
# of sentences
10
10
10
10
HIV
Coral
pupiter
PlayStation
Cloopatra
Assan
On the Side of the Road
Mirroz
Bastille
Ryan Gosling
BloShock
Skapje
American Life
Twunton
Alexander
Castro
Chric Paul
Stanley Rubrick
Number namw
Baya Like Girla
Adizya Thackeray
Chr Catistopeer
Anabel Bernindez
Lux Intarior
History of imericane Angelee
Bobby Coleman
Hazid Dabaahi
Razon Margalet
International Migration (journal)
Jorgensen
Bloodist
Scott Bloomquiet
Trucks
Tascos Papadopoulos
Bteven Staynez
Rubén Bada
Willian T. Freeras
Minnespoiie
Loa Kataa
Cercopsthecus nitis
Edgar Reateria
Sycling Profictency Teet
lydropbasianue
Figure 7: The first 50 entities in our 2ndTestSet. The second column shows their popularity. The third column shows the number of sentences predicted to be sell-contradictrry or non-contradictory. when we use ChatGPT as both gLM and aLM.
21

---

Published as a conference paper at ICLR 2024
entities
of Google search results
of sentences
10
10
10t
10a
Jaeques Rouviez
Insity relationehsp modele
Nagnette
VGRG
Thoras Chapais
Black-and-red Broadbill
Flona. Sbackletos
Idle Varship
Combretastacin A-1
ilepharophizonie, Ptonte, Spicanthus Inverous Syndrome
Jaques Lazier
Indonstreonie
Maraball Xanble
Sigo
Undar the Tuacan Sun (book)
Walter Cantilupe
Manny Sanguillán
Louis Ferdinand Prince Pruscia
Prangoia-Naria-Banjjee Richard
St Jeseph's Colloge, Lochiavar
Minuscule regrosentation
Ballance,
Seraung Bwhold TI (80H-1050)
General Foldrachtacseter
(CHI)
Hackled orb heaver
Gary
Manx Electric Bailway Stations
Blohn Vosa P8188
Tareq el
Dryandra longitolia
Leoh (avinaeer)
Dickte Moore (sce hockey)
Argentina nen's national ice hockay tean
Reprosentativee
Len-Volkhard Freibere vos Vitigenatein
York Islanders records
Follover (Australian foothall)
List of star names Grue
Phousiduua lissesotoides
Preddy Gareia (football masager)
Geozge Bell (pitcher)
Pellegrint (19th-cestary painter)
Robia's Beckoning: Part (Butzan: The Anisated Beriea)
1989-69 FA qualstying rounde
(american football)
Grey Breasted Waitesye
Open
Figure 8: The second 50 entities in our 2ndTestSet. The second column shows their popularity, The third column shows the number of sentences predicted to be sell-contradictryy or non-contradictory. when we use ChatGPT as both gLM and aLM.
22

---

Published as a conference paper at ICLR 2024
Table 12: Real-world seff-contradictory hallucinations generated by different gLMs (left-most column). Sentences marked with green color (resp., red color) are factually correct (resp., wrong). We leverage ChatGPT as aLM for detection and mitigation.
Jerunalan, Tarael, actraas, producer, and director. She is beet knom for her roles in filns ruch as "Black Bran, "Star Vars, " and 'V for Vendetta," " Vith dual. Anerican and Israel: estizenship, Portzan began her acting talent agent in pizza parlor. Portnan's career young age after being discovered Protessional, where ehe played a young gixl film debut va5 Lec Besson's 1996 filn nentored by a hitran- Later, she gained international fare for her role ne Padna Iaidala in the "Star Varo" preguel trilogy, starting 2ith "Star Hare: Episode I The Phanton Kenace' 1888, ler performance in "Black Sean' (2010) von her ceveral awards, including Acadeay Actress. addition of muxerous social and political causes, notabl7 advocaring for anteal righte, environnestal 1ssuet, and vomen'e rights, She 15 aleo a Harrvadd edtccated psychologiet,, having earned her paychology in 2003.
Natalin
June 9,
[Profix]
GPT-4
for
acting career, Portnan
dedicated supporter
bachelor's
[Sartence pair] [istigation]
2011, Portaan sarried French dancer and choreographer Benjarin Killopied. Portman tarried Benjanin Millepied in 2012.
Portasn rarried Benjamin Hill Klllepded
Cathol clargyaan as the Archbishop of Paris from 1896 to 1900. le vas born on March 1, 1819, in Santes, Prance, and wan ordained priest in 1845, During his early career, Richard served ar parish priext and vicar general in the Diocese Manten. He appointed the Riwhop Bolley 1872 and hold that position until 1886. Novenber 3, 1886, he appointed the Coodjutor Archbsehop of Parte, and fee nomche later, on January 6, 1887, he became the Archhi of Paria following the death of Archbishop Jusuph--tppolltt Gushert.
Maanciia-Parie--enjjja kichard (1819-1908)
[Peefix]
GPT-4
(Sencence pair)
Be was rade cardinal by Pope Lee XIII in 1890. cardinal in 1896.
le
Distigation]
by Pape
[Prafik]
Skopje the capital
largeat, city of Morth Macedonia, It 16 located in the central part of the country, on the Vardar River: It. 18 located in the northern part. of the comntry, on the Varder Rivar,
ChatePT
[Senrence pair]
[itiggttion]
It located the Vardar River:
Louis Furdinand von Prenzsen (1907-1964) was a samberr of the House of Hohanzollern and the head the Prussian royal House from 1951 until his death. le the aldest xon of Nilbals, Crown Prince Gereany,
Louis Fordizand surved in Gersan ary as a Potadas, Garaany During Korld Var cossander und sounled tvice. After
[Predix]
the var, he bacave a strong advorate for united Garsany. Be was also an active suportter of various charitable caussa and founded the Princw Louia Perdinand Poundatinn pronote cultural and historical preservation in Gersany.
ChaLGPT
[Sentence pair]
Fardirand married
Ruzsta, they
Louis Fardinand, Princw of Pruxxis, had xix childrwn.
(ittgaatoo)
Louia Fordizand was narried to Grand Duchean Kiza of Russia.
Pattinura Stadiux pultp-prrpoe stadius located Yaisssarr, Indonesta, naned after Pattimura, Kati mnal Baro of Indonwxia who fought against: the Datch colonial rule the 19th century. The stadiun was built in 1967 and has ceating capacity of approxinately 000 spectatore. Pattinura Stadiur primartly intarnational football
[Pewfix]
Llama2-708
for football natches, and har
-Chat
The acadiun is hase to the Indosesia nat tonal footbull cean and has also hosted concerta festivals, and other events.
[Sentence pair]
Pattisura Stadiux ia houe to the Makassar football club and has also bosced several Interna tional
[Nitigation]
The stadiun has also bosted concerte, festivale, and other evente
Bon Ridenhour was forner US Army coldser who becane prominent anti-var activist and whistleblover during the Vietuan Var. Be is best knoon for his role in exposing the My Lai Maccacre, a nase killing of hundreds of uaarzd Vietnanese civilians by US coldiers in 1968. Ridenhour deployed Viotnan in 1968
[Protix]
[l 2-70
Chat
in 1966 in Louisiana and enlisted in the Aray in 1966. gunsor holicopter.
[Sentence pair]
vas deployed Victnas in 1967, Be served as a dooz gannes en helicopter.
[Mitigation]
[Prwfix]
Black Kirror in a Britishh acience fiction anthology talevizion saries craated by Charlie Brooker.
Vicusa-13R 18
[Sentence pair]
The debuted Channel in 2011 and later noved Netrlix subrequent reacons.
The series debused on Metfllx in 2011.
[Nitigation]
The series noved to Notflix for subsequent 5635085
Anerican Life Yee song fron their alvmn the nane, releaced in 1986. The about the stress of modera life in the United States and the longing for simpler, nore fulrilling existence.
[Pewfix]
Vicuma-13B
[Sentance pair]
The cong reached numbor on the Billboard Hos 100 chart. The reached number 21 on the Billboard Hot 100 chart.) song charted Billboard 100.
[Mirigation]
23

---

Published as a conference paper at ICLR 2024
Chapsia Conadias lasyer, bistorias, and politicimn Ix-Boutai Maris, Quebar; He vas rember of Canadien par! lament for Charplain conesituency from 1898 to 1904. Chapaia vas alac prolific vriter and sistorian. He crote several booka on Canadian h history, including *The Greas Intendant: A Chronicle of Joan Talon in Canada, 1665-1672" and "La Bourgele de Quebec et sou fuubour Seint-Jean""
Decerbur 24,
Balat-Denia-e
1910 to 1911. Chapala vas proxinest Figurw the formation the Quebee national ixt nnverast. He argoed that Frwnch Canadtane needed to assert their cultural and political rights within Canada. in 1899, be war a co-foinder of the Ligus national sate canadianne, organization that nought to promota Prench Canadian nationalier. Thonax age
Royal Saciaty
its prexident
Chpais Auguat 1946 Quebes City
Thomas Chapsie was born in Dents de Boutelllerse, Quebec, Be nember Canadian parliament Champlatn conssituency from 1896 1904, Chapais wad prolific weiter historsan, crota books on Canadian history, including "The Great Intendant: Chrenicle of Jeas Talon in Canada, 1985 1672* and "La Bourgade de Quebec et son faubourg Saint-Jean" Be vas nember of the Royal Society of Canada. Caa quebec natfonalis: cultural and political rights vithin Canada. 1899, he was co founder of the Ligue nationaliste canadiemne, an organization that nought to pronote Fronch Canadian nationalien.
argued that French Canadians needed
their
Figure 9: The original text and the revised text for the entity "Thomas Chapais". We leverage ChatGPT as both gLM and aLM. Red color represents sentences removed from the original text, while other colors represent sentences revised to mitigate self-contradictions.
Los Natas Argentine atonar rock hand that vas forned in 1983 Bomncs Aires, The band consista of Sargio Chorzourian (guitar vocals), Claudio Filadoro (bace), and Kalter Broide (druns). Thoir mucic combines elexente of stoner rock, doon netal, peychedelie rock, and heavy rock with hints of Argentine folk music. The hand Bane, "Lax Eabas, ix derived from the vord "mata" spalled backaard (*natas" in Spantah) which ahouraans the band's interest the darz nystical acpects stoner rock bande who nostly sing in inglish.
mucic.
band cinge lattn bas releared several albums
sete Spanich,
spart from nany and EFs throughout their career, including notable alburs such as "Delmar" (1929), "Cludad de Brahman" (1999), "Coreario Megro" (2002), and 'El Honbre Nonrata' (2906). Their music i5 characterized pounding base lines, and thunderous droming, often accorpanted by smooh and nellow heary guitar riffe, aclodic complexity in contrast to pozerful instrumanals. In 2009, taking a histur, which reculted in cevoral ride projecte for each of the nembor. Is vaza's until 2011 that Las latae officislly diabanded, largely due to other projects and commitaent undertaken by the band membere Over the
that add layer of bazd announced that they would be
has acquired large base. Thoir records resain important staples within the collections of fane of geare. Loe listss remembered as one of the aafor ploneers of stones rock and doen aetal in the spanish-speakinn world, paving the vay for other bands from the region to make naze for thenselves in the international rock music scese.
yeare, Los Natas bees influcntial
doca
Los Hatas is an Argentiins atoner rock band forred in Buasox Airea. The band consista of Sargic Chotamurian (guitar and vocula), Claudio Filadoro (bass), and Valter Broide (druss). Thair music cosbinea elasents of stoner rock, netal, psychedelic rock, Natus," in derived from vord "antan" spalled beckward ("natas* in Spantah)) The shich sets thas apart fros rasy other stoner zock banda sho mostly sing in Esghish. Lea Matas has relessed several alhura and EPs throughout their carser, including notable allmm auch "Dalmar" (1998), "Cindad de Brahsan' (1925), "Corseric Negra" (2002), and 'gl Hosbrw Montafia" (2006). Their ausic cheracterized by heavy guitar ziffa, pounding basa lines, and thunderous drussing, often accospanied by amooth and zellov vocala that add a layer of ralodin complerity in contrant to the powerful instrommtala. In 2009, the hand announced that they sould be taking hiatus, which reaulted in awceral side projecta for each of the nesber. Loa Natas offi 2011, largely
heavy rock with hists of Argentine folk ausic. The band's naxe, "Los sings nainly in Spanish,
nabbers Over years, Lox Matan hax been lennial in the storer rock and doon scann and has acqui large fan base. Thair recorda ressin isportant staples within the collections of funa of the ganss. Los Katas is resambered as one of world, paving the way for other bands
eially disbunded
other projecta
undertakon
the zajor pioneezs of atoner rock and Goon zetal in the Spanish-speall muxic.
the ragion
thersnlves
international
Figure 10: The original text and the revised text for the entity "Los Natas". gLM GPT-4 and aLM is ChaIGPT. The colors represent sentences revised to mitigate self-contradictions..
24

---

Published as a conference paper at ICLR 2024
Beptenbor 1968, Skorshas, Fent, England. She gree up in Australia and began her acting career on Australian telavixion the sarly 1890s. She later noved to the Uhited States and gained vorldside recagiition for her parformancea in films auch as 'Mulholland Drive" (2001), "21 Graxs" (2003), "The Painced Teil" (2006), and 'The Isposaible" (2012). Naoni Vatta has been praised
Nanni Vatta Britiah-Austzalion
who
played vide range roles, charncterz to lighter and norw coxedic onea. She has received nurerous nuarda and neninations including Acadeyy Avard and BAFTA nonisationa for Bast Actrusa. In addicion to her work on the big aczeen, Maoxi Vatea hax almo starred in "Birda Pray" (2020). She currently stazzing the
versatility
actress,
her work,
critically acclained teleciatio serins, inel uding "Poin Panka" (2017). "hee
Loudeat Voice" (2019), Son" (2921): Naom: Vatta in known for her private and loe-zey personal life. She van narried to filmakeer Lies Schewiber from 2007 to 2021 and has two shildres with hin: She la committed hameitariin and ix invel ved with several charitable organizations, including the David Lynch Foundation and Plan Intersational.
"The Noon
Maoni Vatto 15 Britich-lustraaian actrese who van born on September 20, 1968, in Shorehan, Kest, England. She lustralia and United States gaised sorldvido recogaition for her perforaaces filxs (2001), "21 Grame" (2005), "The Painted Yesl" (2006), and "The Imporetble" (2012). Nooxt Vatts has been prateed for her versacility as an actress, as she has played vide range of roles, from dranatic and intense charactero comedic noninations
Australion television the early 1980s. She later
"Bulbolland Drive"
lighter and Acadexy Avard and televisten series, including "Tvin Peazs" (2017), "The Loudoss Veice" (2019), and "Birds of Pxoy" (2020), Fassi Vatts ie imown for her private and lov koy perconal 1ife. Ste was rarried to #ilmaker Lier Schzesber and has two children him, She 15 connitted huranstarian and involved with several charstable organtzations, including the David Lynch Foundacion and Plan International.
received
Actress. Naoms awards nominations for her voris, including starred reveral critically
The original text and the revised text for the entity "Naomi Watts". gLM is Vicuna- and aLM is ChatGPT, Red color represents sentences removed from the original text, while other colors represent sentences revised to mitigate self-contradictions.
Figure 11:
Table 13: A self-contradiction generated by Llama2-70B-Chen that cannot be verified using Wikipedia or text obtained by web search. ChatGPT detects this self-contradiction and decides to remove the improve sentence, which improves the overall factuality of the text description.
Prince Albert of Saze Alltenbrg (1913 1902) vas a German prince and the last reigning duise Steebbug,,,, enall duchy located Alteaburg, the capital Saze Alteaburg, and vas the eldest son of Duke Braest and his first eifo, Princess Marie of Hildbugghsssen Prince Albert was educated at home and later studted las and politiccs at the University ot Jena, lle sacoeeded his fasher duks but due ago. 1863, vas officially recognized During his zoign, Prince Albert focused os nodernizing the duchy and isproving the lives of his subjecte. He promoted the construction of railwaye, the expansicn of agriculture industry, and the developnent of education and Sealthcare. Be also took interest art and culture, and vas patron of several artista and musicians. In 1866, Prince Albert played a significant role in the Auctro Pruccian Par, shore he sided with the luctrian Empire againet Prossis, Bovever, his troope vere defeated, and Saze Altenburg wae he exiled
eastern
of Gerzany. le born
minority, regency council ruled adult and began ruling in oen right.
place
he
[Peclix]
subsequently smmexed Prussia. Prince Albert forced abdicate, exile, Prince Albert continued to involved
estate the justrian Expire. Despite in policics and waa prosinent figure is European socisty. Be vas known for his cosservative vieus and his support for the Austrias nonarchy. Be vas known for his lsberal policies and his support for the arts.
[Bentance pair]
[Mitigation]
25

---

Published as a conference paper at ICLR 2024
Prospt:
give the beginning description about Dians Arkenatone. follow
Description:
Diane Arkenstose Anerican muxician, composer, and singer-somguri
Statement 1: She wax born on May 3, 1959, in Kebranka, DSA.
born Califarnia.
Please
the statesente about Diane Arkenstone contradictory. your arplenation only.
CharGP? as alM:
The atatwments about Diane Arkenatoon contradictory. Statement: she born in saye
Nebraska,
They
zame sime.
Vicuna-130 ald::
contradictary. was born in Febraska, statement states that she vas born in California. These too statemente do not contradict each other because lebraszs and California are both states Unitedd States. Alternasively,
mays
possible that Dians Arkesstion have kistaren about ber place birth when interviesed or quoted in the past. in either case, the too statements do not contradict each other.
born Nebraska but latar simply
An example where Vicuna-13B fails to correctly deduce contradictiveness. Vicuna-133 13B makes up three flawed reasons, highlighted by three different colors. In contrast, ChatGPT makes succint and correct response.
Figure 12:
Systax:
You description gonerator.
description question should by the sentence. return sentance description.
Three-shot denonetrationn
entisies 'Douglas Adaza", 'Kayne Weat"', and "Angela Keriee"",
Procpt:
Here is the start of description about Hillsan T. Preersan:
Billian Fresram renouned rasearcher the field of Arcificial Incell (AI) and vixion.
Please genarate next sestance this description.
The panerated Lente sust fill the gep in this Pobjictt:rediitte triple: (Be; Dax bors;
The sentence should contain as litile other inforation possible.
giN: 1980.
Figure 13: Our full prompt for gLM. w..M.gen_seentece
26

---

Published as a conference paper at ICLR 2024
Proapt:
I give you the beginning of description about Willian T. Freeran. Then follow two statements.
Description:
0illian T. Freeran is renoveed researcher in the rield of Aecificial Inselligence (A2) and computer vision.
Statereut Augast 16, 1855,
States.
Staterent 2:
1980.
Please exploin if the stateneuta about Willian T. Freeras are contradictory. Provide your explanation only.
aIN: statenmne
7, Freeman costradictory. Staceneat states vas August shile statement states that he vas born in 1960, There too stateneats cannot both be true at the saxe tine.
Proupt: conclude whether
contradictory with
alN: Yea.
Figure 14: Our full prompt for aLM. detect.
Prompt:
Here 1a the start of description about Hillsan T. Preenan. Millian Freseam renoumed
the field Arsificial Intell (41) and
Original Sentence:
Be was born on August 15, 1955, in the United Statas;
originally
description,
1980.
Rezove the conflictinn information from the sentence, preserving much valid inforration possible. only.
given description. The new centence mast only contain int information from the original sestence.
with
alN: Rillian Freemsn born the Unieed Statws:
Figure 15: Our full prompt for aLM. revise.
27

---

Published as a conference paper at ICLR 2024
Prospt:
I give you the begiming of text ansvering the preapt "Please tell ze about 2008 World Judo Chappiensips Kea's 50 Eg. follow several aubseqeent staterents
the evidence you verity correctness the original statement. If there is contradiction beiseen the original taxt and any the additional wvidences, should conclude that the original acatenent incorract.
original
The 2003 Wozld Zudo Chanpionaliga Men's 60 kg was beld in Dankes, Japan frox Beptember 12 Septembez 14, 2003. The tournanent sax the participation of 62 judokas from different
Original Staterent::
vas van Japan's Tedshiro Komura, who defaated The geld medal in the nasts 60 ky Korea's Choi
rinal.
Evidence 1:
The gold xedal in the ran'a 60 log category caa von Tordanis Arancibia from Cuba.
Evidence
The gold radal. in the men's 60 kg whe von Hirrahh Lxumt from Japan.
Further suspled
onitted for brevity
Arw there contradictions betzeen the original statement and the provided evidence?
Based ressoning about the above queation and the wvidenows, what is your coaclusion your explanation only.
regarding
original
aLM:
There contradictions betseen the original statement and the provided eridence. gold
claina Tadahiro Namura from Japan the nen's a 80 ig category, but the eridences provide different naner ar the vinner, including Yordanis Arencibia from Gaba, Hiroshi Tzuni from Japan, Ludvig Paincher from Austria, tadovic Chamertin of Seitzerland, and Maninjen Andullawy of Bibakistan. original Prorgt: Please conclude shather the staterent incorrent acore batuwen (entirely incorrect) and 10 (fully correct). Ansve: just. "Boorw: gt where is your acorN
original
Figure 16: The prompt for detecting non-factuality in a sentence using 20 sampled alternatives using gLM, gen_ sentence.
28

---

Published as a conference paper at ICLR 2024
You description generator, given the srart descraption. return potontial centences for the description.
Three desonstrations entitics "Douglas Adaxe", leett', "Ingola Kerael"",
Prompt:
Beru the acart of description with the entity Millian T. Fresmani:
Billion
the field Arcificial igence
Please generace too valld costimuttons of this description.
gLM:
extensive research contributions to the National Acadexy of Eagineering in 2021.
nuterous uccol adla, lncluding
Fzvemanta work creating
developing algorichaa interpeet visual
proswasing, particular vay do.
Figure 17: The full prompt of the bascline "Continue" for gLM, sentennc in Table 8.
are discription generator. are grven the start description, raturn the sentance
the description.
using antitieo "Dauglax "Kayne
Markel"
Prompt:
Harn the start of description with the antlty Millian Fresnan:
the field Arcificial
Planse genarate the nest sectance of this dencription. It. should be raphrased version of this sentence: 1996, United States
gIN:
Born in the United States on Anguet 15, 1866, Villisr V. Preezan bas becoze dietingtsseeed figurs
the reals Artificial Intalligence (A1)
Figure 18: The full prompt of the baseline "Rephrase" for aeal.genssentenn Table
Prompt:
description
Hillian
Pillian
Arsifiessl Insolligence
conputer vieson.
Please read the follosing cestence. Vrite least two questionzz that can be answered the information presenzed the folloving contence.
Benceuce:
Be was born on Auguss 16, 1955, in the Ubited States,
glM:
ess
2. Where 930 Killisa
T. Freeman bora?
Prompt:
going wak about Milliin T. Froeran:
Ligence computer
Please acsser the following queation
Sentence:
glN:
Th Language model, do not have to real-tine information. Rouwver, Villian T. Preseen
18, according public recorde.
Figure 19; The full two-turn prompt of the baseline "Q&A" for gLM. J.MM.genssentenn in Table 8.
29

---

Published as a conference paper at ICLR 2024
Proopt:
give you beginning description about
two statemente.
Description:
P5ilian Freezam renovned researcher the field Arcizicial Intelligence
Statement
Be was born August 15, 1955, in the United Statwx
Stacenent
Be was born in 1960.
the aLM: Yea.
Willian Freezam contradictory? Ansvor with oither Yeu
Figure 20: The full prompt of the baseline "Directly ask" for aLM detect in Table
Procptt:
I give you the beginning of description about Willian T. Freezan Then follow two statesents.
Desce iption:
Pillian Freemas resovned researcher
Arcificial Incelligence
computez vision.
Statereet
Ubited
Staterwat 1980.
He
two acalementa Then conclude with yeu or no.
Freusam contradictory? First, show your roasening atup-by-atep fushios.
sIN:
Stap The descriptioo states that Villinr Pronzan rencuned researcher the field
computer: vision.
Step Staterent provides additional information about birthdate, that born Angust
1955,
Stap Statament contradicta aistement by stating that born 1960, which different year than the onn provided in statemennt 1. Stop Tharwfore, the atatmments about Killiwe Presesn contradictory. Conclus
Nillian
costradictory.
Figure 21: The full prompt of the baseline "Step-by-step" for aLM. detect in Table 9.
Peocpt::
you the begizning description about Villian
follow
Description:
Billian Freunas in zenovned researcher the field of Aecifteial Ince1ligance (AZ) and computer Viaion.
Statereet
He wae born on Augrast 16, 1985, in the United States:
He was born in 1960.
Please explain if the statementa about Filliam Preuras contradictory. Provide explamation only.
(path 1):
Yeu, the statements about Villiam T. Preszan are contradictory. Statement giver the correct barthdate of August, different and incorrect birth year of 1960. Therefore, statesent accurate
1965, shile Statmment
while the
aLM (path 2):
Yeu, the staterents about Villiny T. Preszan are contradictory. Statement states that he saa born on August 15, 1955. while statement states that be born 1980. These statamett cannot both trow at the zame tire.
(path
Yen. the ataterents about Willine T. Preezan are omtradictory. Statement says was born in 1955, while statenent 2 caye be vas born in 1960. Both stateneate camoot be true at the came sime, they contradict each other.
aiN (path 4):
August 1955, while etatenent states that be vas born in 1960. Since the too statements provide different birth years, only one of then can be correct, raking then contratictory.
aIN (path 5):
Yer, the statements
Proczan contradictory. his birtbaste an Angust 15, 1955, whereas the second statement providee different birth year altagether (1980),
statement explicitly nontions
Since both statenents cannet be simuitaneously true, they are contradictory.
Prompt:
contradictory
aLN (path and 4): Yes.
aiN (path S and 5): Yes. statmmett contradictory.
Figure 22: The full prompt of the baseline "Multi-path for aLM. detect in Table 9.
30


Reviews:
[{"review_id": "zBNBcjMoVI", "reviewer": "Reviewer_3teD", "summary": "This paper proposes a pipeline to detect self-contradictory hallucinations generated by LLMs. Specifically designed prompts are used to ask LLMs to generate alternative answers to a question. Then an analysis LLM is asked to detect the potential contradiction between two answers and to revise them (to remove the contradiction).\n\nThe method is tested on a set of 30 topics and 360 descriptions generated by LLMs - a dataset created by the authors. The generated answers are verified manually. The method is compared with several baselines using other prompts. The results show that the proposed method can better detect contradictions than other baselines.", "questions": "Have you tested with different temperatures in decoding? This may produce different answers.\n\nCan you compare the method with other methods that leverage external resources?\n\nWould it be possible and meaningful to compare the proposed method with others on short answers?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper examines an important problem of LLMs - hallucination. Despite the huge impact of the problem, there is a limited number of studies about the its detection. This paper offers an interesting solution to one type of hallucination.\n\nThe proposed method relies on existing LLMs. It is easy to implement and to replicate.\n\nThe authors provide a dataset and the tool that can be reused by other researchers.\n\nThe experimental results show some level of success with the proposed method, which outperforms several other pipelines (prompts).\n\nSome ablation analysis is provided about the impact of different LLMs used in the generation and analysis steps.", "weaknesses": "The proposed method may detect one type of hallucination where alternative answers are contradictory. In reality, many hallucinations do not contain verifiable contradictions. The coverage of the method on different types of hallucination may be limited.\n(edit after rebuttal: This is partially explained in the rebuttal)\n\nThe experiments are performed on a limited number of cases. Although a larger set of data is created, it cannot be used in a strict evaluation because of the lack of manual verification. I acknowledge the potential high cost for manual evaluation. However, a larger set of cases would make the experiments more meaningful. One possible solution is to leverage the existing datasets of hallucination. For example, a recent large dataset HalluEval (arXiv: 2305.11747) could be possibly used.\n(edit after rebuttal: The number of topics used in the test is still limited. Therefore, the coverage is still questionable.)\n\nThe experimental results may suggest that there is a strong relationship between gLM and aLM. When ChatGPT is used as both, we can see that the performance is higher in most cases. This contradicts the intuition that GPT-4 is a more powerful LLM and should perform better than ChatGPT. Unfortunately, the paper does not report the case of GPT-4 as both gLM and aLM to see if the combination of GPT-4+GPT-4 is better than ChatGPT + ChatGPT. In any case, the fact that the combination of the same LLM for the two steps is better than when different LLMs are used for the steps may suggest that it would be better to use the same LLM for both steps. Some analysis is required to better understand the relationship between the two steps.\n(edit after rebuttal: The answer in the rebuttal does not provide a plausible reason for this. It just restates the experiments.)\n\nThe revision tries to remove the contradictory part of the answers. This is a conservative revision. If one of the answer is correct, it would be better to keep it instead of removing it. From this perspective, it would be useful to refer to some external information source (e.g. Wikipedia). The authors argue that not doing so can make the approach independent of any external resource, but this may also make the method less reliable. Indeed, looking at the example of William T. Freeman, the generated birth dates are all wrong. A comparison with Wikipedia data would be able to tell that. An external resource would be valuable to the detection of hallucination.\n(edit after rebuttal: Despite the explanation in the rebuttal, there is still not a strong motivation for the revision process.)\n\nYou mention that \"a substantial portion of these self-contradictions (e.g., 35.8% for ChatGPT) cannot be verified on Wikipedia\" to motivate the decision not to rely on external resources. However, you can verify on a much broader set of external resources. So the argument not to use an external resource is not very strong.\n(edit after rebuttal: It is true that Wikipedia has a good coverage for general topics. It may miss many more recent or local events. Other web documents may be useful. Why not using web search to find possibly relevant information?)\n\nThe following description about informativeness is unclear (please revise): For informativeness, we calculate the ratio of non-contradictory sentence pairs (i.e., informative facts) in the revised text compared to the original text. Note that this ratio might exceed 100% because our mitigation can revise contradictory sentence pairs into non-contradictory ones. \n(edit after rebuttal: the revised definition of informativeness is clearer.)\n\nThe paper proposes some reasonable prompts to generate answers and verify contradictions. The proposed prompts should be further motivated. The authors have not provided strong reasons to choose the specific prompts. The only way to see their superiority is through the comparisons in experiments with other alternative prompts.\n(edit after rebuttal: The explanation about the prompts is reasonable. It still remains unclear why these specific prompts are chosen instead of other alternatives.)\n\nThe papers target long text generation in contrast to short answers targeted by some other papers. Why is the length of answers a key factor that distinguish this work from a set of existing studies? Would it still be possible to compare the proposed method with others on short answers?\n(edit after rebuttal: If the method can be applied to both long and short texts, the paper should not emphasize the ability of the method for long texts. In fact, no specific means is taken to deal with long texts vs short texts.)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ej1G5bReil", "reviewer": "Reviewer_a7zQ", "summary": "This paper introduces a prompting-based method to detect and mitigate self-contradictory hallucinations. The proposed method generates texts based on a given prompt and subsequently evaluates these for self-contradiction by generating additional sentences. The text is then iteratively revised with the result of self-contradiction. Experimental results involving four LLMs, including GPT-4 and Llama-2-70B, suggest that the proposed approach effectively addresses self-contradictions.", "questions": "Q1. How does the annotator manually verify contradictory information using Wikipedia?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "The paper is clearly articulated and well-written. Moreover, it offers extensive experimentation with a range of LLMs, demonstrating a solid empirical foundation.", "weaknesses": "The primary issues with this paper include:\n\n- The paper's central premise is that LLMs can produce self-contradictory hallucinations, and the proposed method aims to rectify using a prompting-based approach with LLMs. While it's acknowledged that LLMs can operate as zero-shot reasoners, their potential to induce hallucinations is also evident. Asserting that the authors correct LLM-induced hallucinations using LLMs (without external knowledge) appears paradoxical. This paper should incorporate a theoretical analysis explaining the efficacy of the proposed method in mitigating hallucinations.\n\n- In the experiments, the verification of the text against Wikipedia is assessed, and various ratios are presented. However, a more in-depth exposition of the experimental setup should be shown. Specifically, manual checks were mentioned, but it's questionable whether the annotators thoroughly scanned the entirety of Wikipedia.\n\n- While the paper claims that self-contradictions can be tackled without relying on external knowledge, it does utilize an \"external\" information extraction system to gather context.\n\n- The term \"topic\" is employed in the paper, but Figure 4 indicates that experiments focused on specific entities. It seems that this paper is focusing on entities than on broader topics. If the text generated is primarily entity-centric, perhaps the experiments should have considered data from existing entity-centric QA research [1] even if the paper's emphasis on the open-domain text generation.\n\n- The proposed iterative mitigation algorithm (Algorithm 3) requires an input n, which represents the number of mitigation iterations. The value of n seems to fluctuate based on the size of |x|. The paper should elucidate the chosen value for n and expound on how mitigation performance is influenced by this value.\n\n[1] When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories, ACL 2023", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "q1mZdj1aBp", "reviewer": "Reviewer_gQAk", "summary": "This work address a specific hallucination problem in large language models: self-contradition. More specifically, the authors proposed prompt-based approach, including generating, detecting and revising generated text. Extensive analysis shows prevalence of self-contradictions when LMs generate text for open-domain topics and proposed detection and mitigation method are shown to be effective.", "questions": "1. does the authors try other temperature in between? 0.5, 0.7 for both aLM and gLM?\n\n2. please refer to 1st point in **Weakness** section.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The motivations of the paper is important.\n- This paper proposes a simple prompt-based method for triggering, detecting and migitating self-contradictions.\n- The authors proposed a new eval human-annotated dataset.\n-The authors provide code for reproducibility.\n- Although authors demonstrate the proposed method in open-domain text generation task, same approach could be applied to other NLG tasks.\n- The writing is clear", "weaknesses": "- Correct me if I'm wrong, the authors only evaluated aLM to detect output from gLM, how robust are aLMs to text generation tasks? that are not generated by \"generating initial text, defining context and trigger\" process? \n\n- Last highlight in Revise, Section 5 is wrong.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)