LM vs LM: Detecting Factual Errors via Cross Examination
Authors: Roi Cohen
May Hamri
Mor Geva
Amir Globerson
Venue: EMNLP 2023 Main (2023)
Keywords: knowledge
factuality
LLM
question answering
interpretability
interactivity
multi-agent
consistency
fact checking
dialogue

Abstract: A prominent weakness of modern language models (LMs) is their tendency to generate factually incorrect text, which hinders their usability. A natural question is whether such factual errors can be detected automatically. Inspired by truth-seeking mechanisms in law, we propose a factuality evaluation framework for LMs that is based on cross-examination. Our key idea is that an incorrect claim is likely to result in inconsistency with other claims that the model generates. To discover such inconsistencies, we facilitate a multi-turn interaction between the LM that generated the claim and another LM (acting as an examiner) which introduces questions to discover inconsistencies. We empirically evaluate our method on factual claims made by multiple recent LMs on four benchmarks, finding that it outperforms existing methods and baselines, often by a large gap. Our results demonstrate the potential of using interacting LMs for capturing factual errors.

LM vs LM: Detecting Factual Errors via Cross Examination
Roi Cohen'
May Hamri
Mor Geva²
Amir Globerson
Blavatnik School of Computer Science, Tel Aviv University
Google DeepMind 'Google Research
(roil, mayhamri j@mail. aa..ac. pipekêgoogle.com gamirêtauex. tau.ac.il
Abstract
where a witness is cross-examined in order check whether their statement is factually correct or not. In such a setting, the examiner asks questions that aim to lead towards contradictory statements by the witness, and a contradiction implies that the wit- ness lied at least in some of the statements. Hence the well known quote "Were you lying then or are you lying now?" (Wilder et al., 1957).
A promiment weakness of modern language models (LMs) is their tendeney to generate fac- tually incorrect text, which hinders their usabil- ity. A natural question is whether such factual errors be detected automatically. Inspired by truth-seeking mechanisms in law, we pro- pose a factuality evaluation framework for LMs that is based idea is that an incorrect claim is likely to re- sult inconsistency with other claims that the model generates. To discover such inconsis- tencies, we facilitate a multi-turn interaction between the LM that generated the claim and another LM (acting as an examiner) which in- troduces questions to discover inconsistencies. We empirically evaluate our method on factual claims made by multiple recent LMs on four benchmarks, finding that outperforms exist- ing methods and baselines, often by a large gap. Our results demonstrate the potential of using interacting LMs to capture factual errors.
cross-examination. Our key
To employ this mechanism LM factual calibra- tion, we propose the following setting, illustrated in Figure 1. Our goal is to check whether a statement made by an LM ("Augustus was the first Roman Emperor to sport beard") is factually correct. We refer to the model that generated this statement the EXAMINEE. To check whether this fact is cor- rect, we use another L.M, called EXAMINER, conduct a cross-examinn examinatainnnnon of EXAMINEF. Con- cretely, we craft designated prompts to facilitate a multi-turn interaction between the two LMs, where EXAMINER issues questions (e.g.. "Was it common for Roman Emperors to wear beards?") to EXAMI- NEE to check the veracity of the original statement. The examination is concluded by decision from EXAMINER as whether the original claim was correct or not.!
1 Introduction
Modern language models (LMs) often generate in- consistent (Elazar et al., 2021), atrtabbbibee (Rashkin et al., 2021; Bohnet et al., 2022; Liu et al., 2023a), or factually incorrect text (Tam et al., 2022; Devaraj et al.. 2022; Maynez et al., 2020), thus negatively impacting the reliability of these models (Amodei al., 2016; Hendrycks al., 2021). This has prompted the community develop methods that calibrate the confidence of model predictions to better align with their quality (Brundage et al., 2020). For example, prior methods have used prob- abilistic approaches (Tiang et al., 2020; Zablotskaia et al., 2023) clustering (Kuhn et al., 2023), fine- tuning (Kadavath et al., 2022; Lin et al., 2022) and in-context learning (Alivanistos et al., 2022; Cohen et al., 2023).
Our problem setting is related to that of calibra- tion (Guo et al., 2017), where the goal is to predict the probability at which model will err. How- ever, unlike previous approaches to this problem, we use text generated by LMs. Our approach mo- tivated by the intuition that calibration is actually an claborate reasoning process where one checks the level of support that a fact has, based on other statements the model believes. We argue that such complex reasoning is naturally performed via the strong conversational skills of modern LMs.
We use our method to detect errors LM gener- ation in the context factual question-answertin
In this work, we take different approach to this problem, motivated by truth-sccking mecha- nisms in law. Specifically, we consider the setting
'In practice EXAMINER and EXAMINEE can be the same LM (c.g., GPT-3) 3) that is prompted in two different ways to define its different roles.

---

(c) demonstrating improved factuality detection ac- curacy across several benchmarks.
Augustus was the first Rozan Enperor
sport beard.
1. Who was Augustus?
What is definition Roman Emperon?
LM Pross-Examination
Más it comron for Ronan Experons to wear boards?
previeus Emperors before
Our goal is to employ an "examiner" LM (EXAMINER) to evaluate claims generated by an- other LM (EXAMINEE). To this end, we leverage the recent success of prompting (Liu et al., 2023b), to facilitate cross-examination setting between the two LMs. In such setting, EXAMINER should introduce questions with the objective of reveal- ing inconsistencies with respect to an initial claim made by EXAMINEE. Such inconsistencies can be considered as a signal for uncertainty of EXAMI- NEE in its original claim, and thus can be used to assess whether its original statement was correct.
Augustus?
1. Augustus was the first Experor.
2. Experor politiccal leader who held supreme authority in the Ronan Enpire.
3. No, it wasn't connon for Ronan Experors
to wear beards.
previnus rulers before Augustus were known Ronan Republic's leaders, such as Julfus Caesar
there any historical records on paintings that depict Augustus with bcard?
There are no known historical records paintings that depict Augustus with a beard.
Given an EXAMINER LM and claim C gen- erated by an EXAMINEE, our method establishes a multi-turn interaction between the LMs, where at each turn the other LM is prompted with a des- ignated prompt that incorporates the outputs from previous turns. This interaction continues until the examiner has further questions and can provide its final decision. To establish meaningful interac- tion that reveals possible inconsistencies, we define three stages for the examination, each guided by a specific prompt. As part of each prompt for Ex- AMINER or EXAMINER, we provide the outputs generated in the previous rounds for context. We next describe the examination stages in detail, with the overall process illustrated in Figure 2.
first Roran Experon to sport beard is incorrect. fact, was corson for Roman Emperors
clais that Augustus
beards, Augustus was known for pronating clean- shaven look. Additionally, are no known historical records or paintings that depict Augustus with beard.
Figure I: An example of our LMVLM approach. The first line shows the statement made by the EXAMINEE LLM. Then an interaction between the EXAMINER and EXAMINEE takes place, and the EXAMINER arrives at a conclusion whether the original statement correct or not (here it concludes that it was a false statement).
Our experiments with several recent LMs ( CHAT- GPT, GPT-3 and LLAMA) show that cross- examination effectively detects factually incorrect claims generated by LMs. Specifically, across mul- tiple datasets and examination settings, it detects over 70% of the incorrect claims while maintaining a high precision >80, outperforming strong baselines by a large gap.
Stage 1: Setup The examination begins by "as- signing" the EXAMINER its role. Namely, describ- ing the task setting, providing it with the EXAM- INEE'S claim, and asking it to generate questions for the EXAMINEE.
Next, we feed the questions generated by EXAM- INER, one at a time, EXAMINEE, concatenated to the following instructions: Please answer the following questions regarding your claim. The response from EXAMINER yields a set of an- swers to the questions from EXAMINER.
Further analysis shows that examiner LMs in- troduce multiple questions throughout the exam- ination, and employ various strategies to reveal inconsistencies, including question paraphrasing, validation of implicated arguments, claim decom- position, and requests for evidence.
Stage 2: Follow-up Questions We next feed Ex-
AMINER with the answers generated by EXAMINEE to its initial questions, and ask EXAMINER whether has any follow-up questions. Notably, outputs from EXAMINER at this stage are conditioned on
To conclude. our contributions are (a) framing the task of factuality testing an interaction be- tween two LMs, (b) proposing a concrete imple- mentation of this interaction via the use of one LM with different prompts in a zhro setting, and
"We observe that this effectively steers EXAMINER to ask natural questions directly related to the given claim C' ($5).

---

including question generation (Wang al., 2020; Honovich et al., 2021; Scialom al., 2021), NLI (Thome et al., 2018; Welleck et al., 2019; Maynez et al., 2020; Dziri et al., 2022: Gao et al., 2022; Kamoi et al., 2023), data augmentation (Atanasova ct al., 2022; Wright et al., 2022; Gekhman et al., 2023), and planning schemes that allow the model to self-edit its own generation (Schick et al., 2022). Unlike these works, erence text or external knowledge bases. Instead, we directly check if the LM's claim is likely be correct, by probing the model for inconsistencies. Our approach also uses multi-turn dialogue as a key component.
(1) Setup
The claim is: <O,, what would you ask in order to validate that?
examination questions
Please answer the following questions regarding your clain: cquestions>
answers
assuming any ref-
Follow-Up Questions ...Do you have any fallaw-up questions?
Please with Yes No.
yes
(3) Factuality Decision nste) wh.tttt is your conclusion regarding the correctness of the clain?
Model Calibration A key challenge with pre- diction models is to provide a probability of the answer being incorrect, problem known as model calibration (Guo et al., 2017). The problem of factual-error detection can be viewed as variation of calibration, where instead of continuous prob- ability, we provide binary prediction for whether the model is correct not. This is also related the setting of selective prediction, where models can abstain from answering a query (Varshney et al., 2022; Kamath et al., 2020). Common approaches to calibration are to perform various transforma- tions on model logits (Desai and Durrett, 2020; Jiang et al., 2021), and measuring uncertainty (e.g.. Kuhn et al., 2023). More recent works have studied the use of LMs for providing calibration, by training them on statements known be factually correct or incorrect. This "supervised" approach has been explored via fine-tuning (Kadavath et al., 2022; Lin et al., 2022) and in-context learning (Co- hen et al., 2023; Alivanistos et al., 2022).
the fallow-up
questions?
Follow-up questions
finish
The stage process of cross-examination between the EXAMINER and EXAMINEE, where the factuality of a claim C' generated by EXAMINEE is esti- mated by EXAMINER.
Figure 2:
the previous output from EXAMINEF. If the an- swer from EXAMINER is "Yes", then further prompt it to obtain more questions. This phase is conducted iteratively, until either EXAMINER de- clares has no follow-up questions, or the number of turns has reached a threshold.
Stage 3: Factuality Decision Once fur- ther questions obtained from EXAMINER, prompt it conclude whether the claim C is true false. Specifically, we request it to reply with either "correct" or "incorrect" as its final conclu- sion. In cases where the examiner does not output either of "correct" or "incorrect", we consider its final decision to be rejection of the claim. 4 Typi- cally though, we observe that the examiner follows the instructions and indeed generates a definitive conclusion (see statistics in $5).
We focus on zero-shot factual error detection that involves two categories: predicting whether a model's s claim is correct or incorrect. While we focus on a binary setting, one could envision exten- sions of our approach to continuous outputs (i.e., the probability that the claim is correct).
3 Related Work
Using multiple LMs in an interactive manner is relatively new idea with many potential applications. It has been shown that LMs can utilize additional LMs or tools to enhance safety or better solve downstream tasks (Amodei et al., 2016; Irving al., 2018: Barnes and Chris- tiano, 2020; Schick al., 2023). Additionally, Park et al. (2022) showed that in a social setting, LMs demonstrate certain social skills that emerge from
Multi-Agent LMs
Attribution and Fact Checking Our goal is closely related to works that check if LMggnnerated texts are faithful al., 2022; Honovich al., 2022). This prob- lem has been addressed via several approaches,
given source text (Bohnet
We use a maximum five turns in our experiments. 'Wo also search for additional indicative strings such as "truc", "false", " "is not correct". "is not true".

---

this interaction, and Shinn et al. (2023) proposes that an LM can use different model to instruct it when to "reflect" on its recent action, while per- forming a planned sequence of actions aimed at solving a given query. Intuitively, this model de- tects signs of hallucination or inefficient planning within the LM's trajectory.
EXAMINEE
LAMA
TriviaQA
NQ
PopQA
LLAMA-7B
53.9
48.4
33.8
24.9
GPT-3
79.8
74.2
50.1
43,9
CHATGPT 80.9
77.2
53.3
45.6
Table 1: Portion of factually correct claims by every EXAMINEE LM on each dataset.
4.1 Experimental Setup
Consistency Across Generations LMs have been shown to generate inconsistent outputs given different prompt paraphrases (Elazar et al., 2021; Newman et al., 2021). Prior work showed that prompts can be automatically optimized to pro- duce factually correct claims more robustly (Lester et al., 2021: Zhong et al., 2021; Qin and Eisner, 2021). Hao et al. (2022) utilized multiple gener- ated paraphrases to gauge consistency (Hao et al.. 2022), and other works (Elazar et al., 2021; Zhou et al., 2022) further proposed training objectives to improve model consistency. Another approach handling multiple outputs via variants of de- coding strategies (Wang et al., 2022), or model ensembles (Sun et al., 2022). In our work, we build on these, assuming inconsistencies are more likely to occur with incorrect claims, and let an examiner model search for these by introducing questions to the examinee.
Factual Question Answering One key use-case of LMs is answering questions seeking factual knowledge. For example, "How old was Barack Obama when he was first elected?". In such cases, crucial for the model answer the question correctly, or to indicate that it does not know the answer. We thus evaluate our approach on several Question Answering and Fact Completion datasets. These are typically provided as set of (Q, A) pairs of question Q and its ground answer A. Having gold answers allows dicted answer factually correct or not, which can be used to evaluate our L.VVLM approach.
evaluate if pre-
To apply cross-examination in this setting, we first convert the answer predicted by the model into an EXAMINEE claim that can be provided as input to the examination procedure. Formally, given question Q, if Q is phrased fill-in-the- blank question (e.g. "Bailey Peninsula is located in __"), "), then we feed it to the EXAMINEE model to obtain a prediction that completes the sentence and forms a claim. cases where Q is phrased as question (e.g.. "Where is Bailey Peninsula located?"), we prompt the model to provide an an- swer in claim format with: "Please answer the foll following question: <Q> Please phrase your answer as claim." " This process results in a claim C' that states the model's "belief" about the answer to Q. We then evaluate the truthfulness of through cross-examination and compare the examiner's 's decision of whether C' is correct or not to the ground-truth correctness.
Chain of Thought Reasoning Recent work has shown that LMs can be prompted to elaborate on their reasoning process and that this could be ex- ploited to improve mathematical, multi-hop and common-sense reasoning skills (Wei et al., 2022: Press et al., 2022; Yoran et al., 2023), along with planning and problem-solving abilities (Huang et al., 2022; Long, 2023). Another interesting ap- proach to complex reasoning in LMs is recent work on Maieutic prompting (Jung et al., 2022), that an- swers question by recursively generating set of facts and reasoning over those.
Our approach may be viewed as constructing elaborate chain-of to-thugh explanation for the examinee's claim. However, we do not train this explanation via in-context or fine-tuning, and rather rely on different prompts for its generation.
To evaluate our method, it is necessary to have "gold decisions" to compare the examiner's decisions against. Such la- bels can be obtained from the ground-truthh answers in the data. Namely, the decision for claim C is correct if it matches an evaluation of C' against the gold answer A. To evaluate if the claim C ob- tained for question Q is correct with respect the ground-truth answer A, we first check if A or any its aliases (if provided as part of the dataset, e.g., "FC Tottenham" and "Tottenham Hotspur")
Factuality Evaluation Labels
4 Experiments
In this section, we conduct experiments on multi- ple datasets and models to evaluate our approach, focusing on the task of factual question-answente

---

appears as a sub-string in C' (Schick et al., 2023; Meng et al., 2022). Next, to avoid incorrect labels resulting from this automatic evaluation (Bulian et al., 2022), we manually review all the claims marked as incorrect in the first step, and fix any la- beling mistakes. We also filter out any ambiguous unclear claims generated by EXAMINEE.
False claim
Correct claim
"Wanlockhead is the highest village in France because it is
"Wanlockhead is the high- est village in Scotland, country Earope."
located in the French Alps."
GPT-3
"Louis Ousthuizen the 2010 "Louis Oosthuizen, the Open Golf Champion is Amer- 2010 Open Golf Cham- ican, because he was born in pion, is South African."
the United States."
"The screenwriter for "Smile" "The screenwriter for was definitely Steven Spiel- "Smile" was Jerry Bet- berg."
CHATGPT
Examiner Evaluation We evaluate how well the examiner detects claims that are factually incorrect, using the following metrics:"
son."
"Fontenay is located in the "Fontenay located in beautifu country of Antare- France."
Precision: the portion of incorrect claims, out of the claims rejected by the examiner.
lica." "
Table 2: Example false claims generated by CHATGPT for PopQA and by GPT-3 for TriviaQA.
Recall: the portion of incorrect claims rejected by the examiner, out of all the incorrect claims. F1: the harmonic mean of precision and recall. For completeness, we additionally report (in ŞE) the complementary Precision, Recall, and F1 scores with respect to detection of correct claims.
Models We use CHATGPT (gpt-3. (gpt-3.5-turbo) 5-turbo), GPT-3 (text-davinci-003) (Brown et al., 2020;
Ouyang et al., 2022), and LLAMA-7B (Touvron et al., 2023), in three EXAMINER vs. EXAMINEE cross-examination settings: GPT-3 vs. GPT-3, CHATGPT vs. CHATGPT, and CHATGPT vs. LLAMA. Notably, using the same LM as EXAM- INER and EXAMINEE (except for their prompts, which are different), provides cleaner setting where both LMs share the same knowledge. The prompts used for each LM at every stage of the examination are shown in Table 10.
Data We consider the following datasets: LAMA (Petroni al., 2019), TriviaQA (Joshi al., 2017).
Natural Questions (NQ) (Kwiatkowski et al., 2019)
and PopQA (Mallen et al., 2022). These cover a wide range of queries, from real user queries (NQ),
to trivia questions (TriviaQA), and subject-relation- object facts phrased as queries (LAMA, PopQA). We consider the closed-book open-ended setting, where we do not provide any context answer choices to the model. We evaluate our approach on 1, 1,000 random examples from the test set (or from the development set a test set not available). 6 In addition, created dataset of false claims further test approach. This "Falsehoods" dataset contains only wrong claims, created sep- arately for each model (GPT-3 and CHATGPT) and for each of the four QA datasets. Concretely, given a model and a question Q, we prompt the model to generate false answer (see $C for de- tails). We verify that these are indeed incorrect claims by checking that the gold answer (and any of its aliases, if they exist) does not occur in the generated text. This yields subset of examples that are realistic, namely, the answer matches the target type (e.g.. "a city"" but incorrect (see ex- amples in Table 2). The examiner's ' decision for these examples should always be to reject.
Baselines For each setting, we compare LMvLM with recent methods for uncertainty detection and variants of our approach:
Confidence-Based The prediction head of LMs outputs probability for the predicted token. It is common practice to use this probabil- ity as a measure of confidence in the prediction (Yoshikawa and Okazaki, 2023). In our case, the LM generates a multi-token claim, and we caleu- late the confidence for the claim as the product of probabilities for all predicted tokens of the answer only. In order to output a binary decision (i.e., is the claim correct or not), we optimize a threshold over the train dataset to maximize F1. Note that examination approach does not require tuning any threshold.
Are you sure? (AYS): Recent work (Kadavath et al., 2022: Cohen et al., 2023) has shown that LMs can be trained to estimate their certainty in generated facts. Here, we use zero-shot ver- sion of this approach where we directly "ask" the model whether it is sure. Specifically, we add the following prompt right after the claim generation:
'We say that examiner "rejects" a claim the examiner concludes that the claim is incorrect.
"We use only subset of examples due to the high cost of running large LMs like GPT-3 used in our experiments.

---

LAMA
TriviaQA
NQ
PopQA
R
F1
R
F1
P
R
F1
P
R
F1
AYS
82.3
25.2
38.6
79.9
17.9
29.2
85.2
43.3
78.4
35.7
63.9
IDK
49.1
52.4
50.7
48.7
66.5
56.2
62.5
60.7
61.6
70.0
61.1
65.2
LMvLM
85.1
70.7
76.7
82.8
71.6
76.8
74.5
74.9
77.7
83.6
77.1
80.2
LMvLM (Majority)
86.6
75.8
80.8
84.5
80.8
82.6
82.3
76.1
79.1
87.0
84.0
85.4
-Follow-up
83.8
68.1
75.1
82.3
69.7
75.5
74.8
72.1
73.4
82.0
73.3
77.4
Table 3: Precision (P), Recall (R), and Fl scores for LMVLM with CHATGPT as EXAMINER and EXAMINEE, compared to baselines. The last row shows an ablation of our method without the follow-up questions stage.
LAMA
TriviaQA
NQ
PopQA
F1
P
R
F1
AYS
74.8
17.9
28.9
80.3
19.8
31.8
74.9
20.7
32.3
74.6
22.7
34.8
IDK
43.0
42.1
42.5
47.9
45.7
46.7
60.9
45.3
32.0
52.1
37.6
43.7
Confidence- SanddennceBBsse
38.6
85.8
53.2
39.6
84.4
53.9
56.2
72.7
63.4
60.8
69.7
64.9
IC-IDK
71.5
16.3
56.2
70.6
49.7
60.1
70.0
57.6
63.2
76.9
37.7
50.6
LMvLM
78.8
69.9
74.1
64.6
72.1
70.3
66.6
68.5
75.5
69.1
72.2
LMvLM (Majority)
80.7
77.9
79.3
83.1
72.1
77.2
79.3
76.8
78.0
82.2
71.4
76.4
- -Follow-up
76.4
71.1
73.7
78.7
64.8
71.1
66.6
70.1
68.3
70.9
65.8
68.3
Table 4:
Precision (P), Recall (R), and Fl scores for LMvLM with GPT-3 as EXAMINER and EXAMINEE, compared to baselines. The last row shows an ablation of our method without the follow-up questions stage.
"Are you sure regarding the correctness of your claim? Please answer th Yes or No". Then we take the output as the prediction whether the claim is correct not.
text or "Don't know". Based on the output, we generate a factuality label as in the IDK baseline above. Notably, this baseline requires labeled data for the in-context demonstrations, which is not necessary for our approach.
don't know (IDK): Recently, Ganguli et al. (2023) showed that LMs might have the capabil- ity to self-correct themselves, when instructed to
LMvLM: A single execution of our method, where we accept reject the claim according to the examiner's final decision.
do so. Here we instruct the model output "/ don't 't know" if is uncertain, by concatenating the following sentence to the original query: "If you are not sure you know the answer, answer wi th "I don't 't know" only.". ". the model answers "I don't 't know" we label the corresponding claim as false, and otherwise true. In-context IDK (IC-IDK): We teach the model to output that it doesn't it know the answer, via in-context demonstrations. We follow Cohen et al. (2023) and test each of the queries within in-context setting. For each query, first provide the model with K demonstrations, with D of them labeled as "Don't 'tt know" examples, while the rest K- D are provided with their gold answer from the dataset. The "Don't 't know" examples are randomly selected out of a set of examples the model failed on, while evaluat- ing it on an held-out set of examples from the dataset in zero-shot setting. Intuitively, these examples' answers are likely to be unknown to the model, hence we labeled them with "Don't know". " The model predictions are either target
LMvLM (Majority): For a given claim, we apply method three times (with the same EXAMINER and EXAMINEE). using sampling generation for follow-up questions generation. We reject the claim in case at least two of the examinations concluded it is false.
Since output probabilities not provided part of the CHATGPT's API, we cannot provide results for the Confidence-Basse baselines in this case. Moreover, we observe that CHATGPT often fails to understand the task of IC-IDK.
4.2 Results
Tables 3. 4, 5 show the results for the the following pairs of EXAMINEE vs EXAMINER: CHATGPT vs. CHATGPT, GPT-3 vs. GPT-3, and LLAMA vs. CHAIGPT, respectively. We do not include results for LLAMA as an EXAMINER since it did not work well, possibly because it is less specialized instruction following.
Across all settings, our method outperforms the baselines, often by a large gap. For example, it ob-

---

LAMA
TriviaQA
NQ
PopQA
R
F1
P
R
F1
P
R
F1
P
R
F1
AYS
61.4
38.0
46.9
60.0
35.7
44.8
71.1
15.0
24.8
74.8
14.2
23.9
IC-IDK
56.6
49.0
52.5
58.9
52.5
55.5
66.2
53.4
66.8
57.3
IDK
61.6
44.8
51.9
62.0
32.9
43.0
20.4
66.7
16.8
26.8
Confidence- Based
54.9
64.0
56.9
68.4
63.9
64.6
53.6
58.6
LMvLM
81.1
66.4
73.0
80.1
70.8
75.2
79.3
65.5
71.7
84.9
73.6
78.8
LMvLM (Majority)
82.9
73.9
78.1
80.3
76.8
78.5
83.7
74.2
78.7
88.3
77.4
82.5
-Follow-up
79.7
65.7
72.0
80.0
69.8
74.6
79.4
63.7
70.7
83.3
71.8
77.1
Table 5: Precision (P), Recall (R), and Fl scores for LMVIM with CHATGPT EXAMINER and LLAMA EXAMINEE, compared to baselines. The last row is an ablation our method without the follow-up questions stage.
LAMA
TriviaQA
NQ
PopQA
4.3 Ablations
GPT-3
65.7
98.4
89.9
83.1
GPT-3 (Majority)
75.8
98.5
92.0
88.0
Follow-Up Removal remove the follow-up iterations in the examination process gauge their benefit. Results are reported for GPT-3 in Table 4 (last row), showing a large decrease performance (e.g. 78 - 68.3 Fl for NQ and 77.2 + 71.1 for TriviaQA). Notably, recall scores are decreased by 6%-10%. Overall, this shows the importance of the follow-up questions issued by the examiner to assess the examinee's claim.
CHATGPT
83.6
97.9
90.4
88.8
CHATGPT (Majority)
87.1
98.6
94.2
93.9
Table Accuracy of GPT-3 and CHATGPT as EXAM- INER on false claims generated for each dataset.
CHATGPT/ CHATGPT
GPT-3/ CHATGPT/ GPT-3
LLAMA
of questions
7.0 1.01:...
6.4 4.3 6.8.444
Detrieval-Augmenttr LMS Language models can be significantly strengthened when augmented with additional retrieved data (Khandelwal et al., 2019; Borgeaud et al., 2022: Zhong et al., 2022: Guu al., 2020). We next perform evaluation of LMs in this setting well. We focus on aug- menting the EXAMINEE, since this is the model that can presumably benefit from additional infor- mation when answering questions. Specifically, we used DPR (Karpukhin et al., 2020) for retrieval and took the top three passages it retrieved from Wikipedia, concatenated them, and instructed a GPT-3 EXAMINEE (in the prompt) to answer the question based the passages.
# of follow-up 1.3 1: 1.0
1.3 0.6 1.1 0.5
questions
per
iteration
# of follow-up iter- 1.9 - 1.2
1.3 1.6
ations
questions per 3.1 + 2.7 1.6
2.9 1.9
itcration
of inconclusive 14.8% examiner decisions
9.1%
10.3%
Table 7: Cross-cxaminaliy statistics for each setting (EXAMINER/XXAMINEEE) averaged across datasets.
tains 85.4 Fl compared < 65.2 by baselines for CHAIGPT on PopQA (Table 3), and 77.2 Fl com- pared < 60.1 for GPT-3 on TriviaQA (Table 4). Notably, the most substantial gains are in terms of recall, showing the superiority of our method in de- tecting factually incorrect claims (when compared to the baselines which achieve reasonable preci- sion too). Interestingly, we observe that CHATGPT generally outperforms GPT-3.
This resulted in improved accuracy of the Ex- AMINEE (i.c., the fraction of questions answered correctly) from 50.1 to 66.4 when augmented with DPR. Table 9 reports results factuality detection when using GPT-3 EXAMINER. It can be seen that our approach still outperforms the baselines. as in the case without augmentation.
Comparing the performance of the retrieval- augmented EXAMINEF, to the non-augmented one (see Table 4. NQ columns), the EXAMINEE with retrieval leads to substan- tial increase in precision (79.3 87.5) with only small decrease in recall (76.8 -> 75.7) and an overall improvement of 2.9% in FI. This shows that, alongside the improvement the EXAMINEE,
Last, Table 6 shows the accuracy of our method and baselines on our Falsehood dataset. For both CHATGPT and GPT-3, LMVLM successfully re- jects large majority of the false claims, obtaining 87%-98% accuracy with CHATGPT and 75%-99% with GPT-3 across all datasets.
that augmenting

---

Pattern
Example statements/qucstios generated EXAMINER during examination
Rephrasing the claim
Claim: "The first Fast and Furious film was released in 2001."
In which year was the first Fast and Furious film released?
Rephrasing Questions
Claim: "The screenwriter who is credited with writing the screenplay for Winner is Wendy Riss"
1. What is the name of the screenwr iier who is credited with writing the screenplay for Winner? 2. Who is credited with wr writing the screenplay for Winner?
Validation Implications
Claim: "The director of The Town was Ben Affleck."
Ben Affleck known for directing any movies?
Logical decomposition
"The second oldest the Pevensie children C Lewis's The Lion, the Witch and the Wandrobe is Edmand."
Claim:
What the birth order of the Pevensie children in Lewis's Lion, Witch the Wardrobe? What are their ages? Who appears second this list?
Request for attribution
Claim:
"The screenwriter of Cover Up is Bill Blake"
there any evidence or documentation that supports the claim that Bill Blake the screenwriter for Cover Up?
Wrong intermediate
Claim: "There eight vertices (corners) octahedron."
EXAMINER: How many vertices does octahedron have? EXAMINEE: An octahedron has eight vertices, each of which is the point where three edges meet.
answers
Table 8: Examples for frequent patterss of CHATGPT and GPT-3 observed through manual analysis of examinations.
F1
of claims (9%-15%) where the examiner LM does not arrive conerete final decision (i.e., it does not generate "correet" or "incorrect" as the final decision. We reject the claim in those cases). our qualitative analysis, we identify reasons that could explain these cases.
AYS
80.2
20.1
32.1
IDK
69.5
46.0
55.4
Sonfidence-Basee
62.6
71.9
66.9
LMvLM
87.5
75.7
80.3
Qualitative Analysis We manually analyze a sample of 96 examinations 48 by each LM, with correct and 6 incorrect examinations for each model and each dataset. We observe the following trends (examples are in Table 8):
Table 9: Precision (P), Recall (R), and scores for LMvLM with GPT-3 EXAMINER and retrieval augmented GPT-3 as EXAMINEE.
LMvLM still performs well in detecting factual errors when they occur.
1. Rephrasing the claim: In about 60% of the examinations, both LMs introduce questions which are paraphrases of the original question. This supports the assumption that the EXAM- INER seeks inconsistencies by generating vari- ants of the original claim.
5 Analysis of Cross-Examinaal
We analyze cross- coossexxminations by GPT-3 and CHATGPT to better understand the success and failure cases of our method. We find that exam- iner LMs typically ask multiple questions in the examination, and perhaps surprisingly, seems to apply different strategies to reveal inconsistencies. We note that furhter analysis is required in order to better understand whether the EXAMINER indeed utilizes the conducted examination in its factuality decisions.
In about half of the cases, both LMs introduce questions that are similar to previously asked questions are dif- ferently phrased. This is a desirable behavior can reveal inconsistencies the examinee pro- vides a different answer for the same question. 3. Validation of logical implications: The EXAM- INER asks EXAMINEE regarding implied argu- ments that must be true whenever the original claim is correct. This can be observed in 70% of the correct detections of GPT-3, and 87.5% out of the correct detections of CHATGPT.
2. Rephrasing Questions:
Examination Statisties Table 7 provides statis- ties on the cossinxaminstions performed by CHAT- GPT and GPT-3. Both models introduce multiple queries (6-7 on average) during an examination, with typically 1-2 steps of follow-up questions, which are important for the examiner's 's decision ($4.3). We also observe a non-negligible number
4. Logical questions: The EXAMINER decom- poses the claim into multiple sub-questions

---

which together compose a trajectory validat- ing it. Such decompositions appear in about 75% of the cases for CHATGPT but only 10% in GPT-3. We observe these in 33% of the correct detections of GPT-3, and 70% for CHATGPT.
process, rather the having only single EXAMINER. Finally, one can train the EXAMINER generate questions more effectively.
Limitations
5. Request for attribution: The EXAMINER ask the EXAMINEE about the existence external evidence to support the claim. This happens in about 30% of the cases for both LMs.
We note three limitations of our LMVLM method. First, it requires multiple queries of the examinee and examiner LMs, which could be costly when using external APIs such as those used this work. This could key considcration when scaling this approach to large numbers of claims.
6. Wrong intermediate answers: The EXAMI- NEE responds with factually incorrect answers to one or more of the questions asked by the EXAMINER. We observe this occurs mostly in cases where the original claim is false (it hap- pens in only in about 14% of the cases where the EXAMINER is correct). In both models, this can be observed in about half of the cases where the claim is false and has also been detected by the EXAMINER. Furthermore, it occurs in about 80% of the cases where the EXAMINER has accepted a false claim, and in 45% where the EXAMINER has rejected a correct claim.
Second, for our method to succeed, both LMs (EXAMINEE and EXAMINER), but mostly EXAM- INER, should be able to follow instructions and have the ability to reason over information in rel- atively long context. This skill is currently mostly demonstrated by larger models (>10B parameters) and thus our method may not perform as well for smaller models.
Last, any logical flaws in the examiner's opera- tion are likely to affect the overall examination, po- tentially leading to inaccurate decisions. However, our experiments show that, even if such flaws occur, our method is still useful average it substan- tially improves factuality detection. Nonetheless, developing safety mechanisms that detect and miti- gate logical flaws an important research direction, that we leave for future work.
We note that in most cases where LMvLM fails, EXAMINEE provides incorrect information to Ex- AMINER. This may indicate that in those cases EXAMINEL encodes large set of factually wrong facts that mutually consistent, thus making hard for EXAMINER to detect inconsistencies. Fi- nally, the fact that CHATGPT more commonly vali- dates the claim via logical questions might key factor in its superiority over GPT-3 in our setting.
A Acknowledgements
This work is supported in part by the Israeli Sci- ence Foundation. We thank Roee Aharoni and Avi Caciularu for valuable feedback and constructive suggestions.
6 Conclusion
We introduce LMvLM, a method for zero-shot detection of factuality errors. Our method uses prompting to facilitate multi-turn interactions be- tween an two LMs, to reveal inconsistencies that imply factually incorrect claims. LMvLM builds a fundamental connection between self- consistency (i.e., consistency of an LM with itself) and factual consistency (i.e., consistency between claims generated by an LM and ground-truth facts). We consider the LM as the source of information, and we test whether a claim it has generated is consistent with other beliefs it has.
References
Dimitrios Alivanistos, Selene Báez. Santamaría, Michael Cochez, Jan-Christoph Kalo, Emile van Kricken, and Thiviyan Thanapalasingam. 2022. Prompting as probing: Using language models for knowledge base construction. arXiv preprint arXiv:2208. 10008/11555 11057.
Dario Amodei. Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. 2016. Concrete problems in safety. arXiv preprint arXiv:1606.06566 065666660666......
Our work can be extended in several ways. First, LMvLM provides interpretable information about related beliefs of the model, which could be ana- lyzed to understand what makes the model commit certain mistakes. Sccond, one may incorporate several LM instances into the factuality detection
Pepa Atanasova, Jakob Grue Simonsen. Christina Li- oma, and Isabelle Augenstein. 2022. Fact checking with insufficient evidence. Transactions of the Asso- ciation Computational Linguistics, 10:746-763.
Beth Barnes and Paul Christiano. 2020. Writeup:
Progress on Al safety via debate.

---

Bernd Bohnet, Vinh Q Tran, Pat Verga, Roce Aharoni. Daniel Andor, Livio Baldini Soares, Jacob Eisenstein, Kuzman Ganchev, Jonathan Herzig, Kai Hui, et al. 2022. Attributed question answering: Evaluation and modeling for attributed large language models. arXiv preprint arXiv:221 2w..2222.22223
and Yoav Goldberg. 2021. Measuring and improving consistency in pretrained language models. Transac- tions of the Association for Computational Linguis- rics, 9:1012-1031.
Deep Ganguli, Amanda Askell, Nicholas Schiefer, Thomas Liao, Kamilé Lukošiité, Anna Chen, Anna Goldie, Azalia Mirhoseini, Catherine Olsson, Danny Hemandezz, et al. 2023. The capacity for moral self- correction in large language models. arXiv preprint urXiv:2302.07459
Sebastian Borgeaud, Arthur Mensch, Jordan Hoff- mann, Trevor Cai, Eliza Rutherford, Katie Milli- can, George Bm Van Den Driessche, Bean-Baptise Lespiau, Bogdan Damoc, Aidan Clark, et al. 2022. Improving language models by retrieving from tril- lions of tokens. In International conference on ma- chine learning, pages 2206-2244 2240. PMLR.
Luyu Gao, Zhuyun Dai, Panupong Pasupat, Anthony Chen, Arun Tejasvi Chaganty, Yicheng Fan, Vin- cent Y Zhao, Ni Lao, Hongrae Lee, Da-Cheng Juan, et al. 2022. Rarr: Researching and revising what language models say. using language models. arXiv preprint 1TX:::2200008726.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systemss, 33:1877-1901.
Zorik Gekhman, Jonathan Herzig, Roee Aharoni, Chen Elkind, and Idan Szpektor. 2023. Tructeacher; Learn- ing factual consistency evaluation with large lan- guage models. arXiv preprint arXiv:2305. 11171..
Miles Brundage, Shahar Avin, Jasmine Wang. Haydn Belfield, Gretchen Krueger, Gillian Hadfield, Heidy Khlaaf, Jingying Yang. Helen Toner, Ruth Fong, et al. 2020. Toward trustworthy ai development: mecha- nisms for supporting verifiable claims. arXiv preprint arXiv:2004.07213.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Wein- berger. 2017. On calibration of modern neural net- works. In Proceedings of the 34th International Con- ference on Machine Learning, volume 70 of Pro- ceedings of Machine Learning Research, pages 1321- 1330. PMLR.
Jannis Bulian, Christian Buck, Wojeiech Gajewski, Ben- jamin Börschinger, and Tal Schuster. 2022. Tomayto, tomahto. beyond token-level answer equivalence for question answering evaluation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 291-305, Abu Dhabi, United Arab Emirates. Association for Computa- tional Linguistics.
Kelvin Guu, Kenton Lee,, Zora Tung, Panupong Pasu- pat, and Mingwei Chang. 2020. Retrieval augmented language model pre-training. In International confer- ence machine learning, pages 3929-3938. PMLR.
Shibo Hao, Bowen Tan, Kaiwen Tang. Hengzhe Zhang, Eric P Xing, and Zhiting Hu. 2022. Bertnet: Har- vesting knowledge graphs from pretrained language models. arXiv preprint arXiv:2206. 14268.
Roi Cohen, Mor Geva, Jonathan Berant, and Amir Globerson. 2023. Crawling the internal knowledge- base of language models. In Findings of the Asso- ciation for Computational Linguistics: EACL 2023, pages 1856-1869, Dubrovnik, Croatia. Association for Computational Linguistics.
Dan Hendryeks, Nicholas Carlini, John Schulman, and Jacob Steinhardt. 2021. Unsolved problems in ml safety. arXiv preprint ar arXiv:2109. 13916.
Or Honovich, Roce Aharoni, Jonathan Herzig, Ha- gai Taitelbaum. Doron Kukliansy, Vered Cohen, Thomas Scialom, Idan Szpektor, Avinatan Has- sidim, and Yossi Matias. 2022. True: Re-evaluating factual consistency evaluation. arXiv preprint 2t0.:222044999..
Shrey Desai and Greg Durrett. 2020.
Calibra- tion of pre-trained transformers. arXiv preprint arXiv:2003.07892
Ashwin Devaraj. William Sheffield, Byron Wallace, and Junyi Jessy Li. 2022. Exaluating factuality in text simplification. In Proceedings of the 60th Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers), pages 7331-7345, Dublin, Ireland. Association for Computational Lin- guistics.
Or Honovich, Leshem Choshen, Roee Aharoni, Ella Neeman, Idan Szpektor, and Omri Abend. 2021. q²: Evaluating factual consistency in knowledge- grounded dialogues via question generation and ques- tion answering. In Proceedings of the 2021 Confer- ence on Empirical Methods in Natural Language Pro- cessing, pages 7856-7870, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Nouha Dziri, Hannah Rashkin, Tal Linzen, and David Reitter. 2022. Evaluating attribution in dialogue sys- tems: The BEGIN benchmark. Transactions of the Association for Computational Linguistics, 10: 1066- 1083.
Wenlong Huang, Fei Xia, Ted Xiao, Harris Chan, Jacky Liang, Pete Florence, Andy Zeng, Jonathan Tomp- son, Igor Mordatch, Yevgen Chebotar, et al. 2022. Inner monologue: Embodied reasoning through
Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhi- lasha Ravichander, Eduard Hovy, Hinrich Schüttze

---

planning with language models. arXiv preprint arXiv:2207.05008.
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Red- field, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Ken- ton Lec, al. 2019. Natural questions: a benchmark for question answering research. Transactions of the Association for Computational Linguistics, 7:453- 466.
Geoffrey Irving, Paul Christiano, and Dario Amodei. 2018. Al safety via debate. arXiv preprint arXiv:1805.00899
Zhengbao Jiang, Jun Araki, Haibo Ding, and Graham Neubig. 2021. How can we know when language models know? on the calibration of language models for question answering. Transactions of the Associa- tion for Computational Linguistics, 9:962-99 977.
Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. The power of scale for parameter-effictee prompt tuning. arXiv preprint (tt.::2004.0691...
Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. Teaching models to express their uncertainty in words. arXiv preprint 2205:22055.4334..
Zhengbao Jiang, Frank F. Xu, Jun Araki, and Graham Neubig. 2020. How can we know what language models know? Transactions of the Association for Computational Linguistics, 8:423-438.
Nelson F Liu, Tianyi Zhang, and Percy Liang. 2023a.
Evaluating verifiability in generative search engines. arXiv preprint arXiv:2304.098
Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zeulemoyer. 2017. Triviaqa: large scale distantly supervised challenge dataset for reading comprehen- sion. arXiv preprint 1rXiv:705500355
Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2023b. Pre- train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 55(9):1-35.
Jaehun Jung, Lianhui Qin, Sean Welleck, Faeze Brah- man, Chandra Bhagavatula, Ronan Le Bras, and Yejin Choi. 2022. Maieutic prompting: Logically consistent reasoning with recursive explanations. In Conference on Empirical Methods in Natural Lan- guage Processing.
Jieyi Long. 2023. Large language model guided tree-of- thought. arXiv preprint 230..0891.
Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Hannaneh Hajishirzi, and Daniel Khashabi. 2022. When not to trust language models: Inves- tigating effectiveness and limitations of paramet- ric and non-pan non-parametric memories. arXiv preprint arXiv:2212. 10511.
Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield Dodds, Nova DasSarma, Eli Tran-Johnson, et al. 2022. Language models (mostly) know what they know. arXiv preprint arXiv:2207.05221.
Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. 2020. On faithfulness and factu- ality abstractive summarization. of the 58th Annual Meeting of the Association for Computational Linguistics. pages 1906-1919, On- line. Association for Computational Linguisties.
Amita Kamath, Robin Jia, and Percy Liang. 2020. Se- lective question answering under domain shift, arxiv preprint (00::2000009466)
Proceedings
Ryo Kamoi, Tamya Goyal, and Greg Durrett. 2023. Shortcomings of question answering based factuality frameworks for error localization. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 132-146, Dubrovnik, Croatia. Association for Com- putational Linguistics.
Kevin Meng, David Bau, Alex J Andonian, and Yonatan Belinkov. 2022. Locating and editing factual associ- ations in GPT. In Advances in Neural Information Processing Systems.
Benjamin Newman, Prafulla Kumar Choubey, and Nazneen Rajani. 2021. P-adapters: Robustly extract- ing factual information from language models with diverse prompts. arXiv preprint 1000v2210.007280.
Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Eduumo, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open-domain question answering. arXiv preprint arXiv:2004.04900
Long Ouyang. Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, al. 2022. Training language models to follow instruc- tions with human feedback. Advances in Neural Information Processing Systems, 35:27730-27744,
Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. 2019. Generalization through memorization: Nearest neighbor language models. arXiv preprint 101000.7
Joon Sung Park, Lindsay Popowski, Carrie/ J. Cai. Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2022. Social simulacra: Creating popu- lated prototypes for social computing systems. In The 35th Annual ACM Symposium on User Inter- face Software and Technology, UIST 2022, Bend, OR,
Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. 2023. Semantic uncertainty: Linguistic invariances for un- certainty estimation in natural language generation. arXiv preprini arXiv:2302.0966

---

USA, 29 October 2022 2 November 2022, pages 74:1-74:18. ACM.
James Thome, Andreas Vlachos, Oana Cocarascu, Christos Christodoulopoulls, and Arpit Mittal. 2018. The fact extraction and VERiíicattion (FEVER) shared task. In Proceedings of the First Workshop on Fact Extrccion and VERification (FEVER), pages 1- 9. Brussels, Belgium. Association for Computational Linguisties.
Fabio Petroni, Tim Rocktäschel, Patrick Lewis, An- ton Bakhtin, Yuxiang Wu, Alexander H Miller, and Sebastian Riedel. 2019. Language models as knowl- edge bases? arviv preprint arXiv: wee...99900000 000601
Ofir Press. Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A Smith, and Mike Lewis. 2022. Measuring and narrowing the compositionality gap in language models. arXiv preprint arXiv:2210.03::
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix. Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023, Llama: Open and effi- cient foundation language models. arXiv preprint urXiv:2302.13971
Guanghui Qin and Jason Eisner. 2021. Learning how to ask: Querying LMs with mixtures of soft prompts. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, pages 5203-5212, Online. Association for Computa- tional Linguisties.
Neeraj Varshney, Swaroop Mishra, and Chitta Baral, 2022. Investigating selective prediction approaches across several tasks in IID, OOD, and adversarial settings. Findings of the Association for Compu- tational Linguistics: ACL 2022, pages 1995-2002, Dublin, Ireland. Association for Computational Lin- guistics.
Hannah Rashkin, Vitaly Nikolaev, Matthew Lamm, Lora Aroyo, Michael Collins, Dipanjan Das, Slav Petrov, Gaurav Singh Tomar, Iulia Ture, and David Reitter. 2021. Measuring attribution natu- ral language generation models. arXiv preprint arXiv:2112.12870.
Alex Wang, Kyunghyun Cho, and Mike Lewis. 2020. Asking and answering questions to evaluate the fac- tual consistency of summaries. In Proceedings of the 58th Annual Meeting of the Association for Compu- tational Linguistics, pages 5008 5000-5020, Online. Asso- ciation for Computational Linguistics.
Timo Schick, Jane Dwivedi-Y Yu. Roberto Dessi, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: Language models can teach themselves to use tools. arXiv preprint arXiv:2302.0476)
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Lc, Ed Chi, and Denny Zhou. 2022. ceff-consistency im- proves chain of thought reasoning in language mod- els. arXiv preprint arXiv:2203.1117.
Timo Schick, Jane Dwivedi- Yu, Zhengbao Jiang, Fabio Petroni, Patrick Lewis, Gautier Izacard, Qingfei You, Christoforos Nalmpantis, Edouard Grave, and Sehas- tian Riedel. 2022. Peer: A collaborative language model. arXiv preprint Arxiv:2208.11 11022088..166... 11663.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoe Le, and Denny Zhou. 2022. Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.
Sean Welleck, Jason Weston, Arthur Szlam, and Kyunghyun Cho. 2019. Dialogue natural language inference. In Proceedings of the 57th Annual Meet- ing of the Association for Computational Linguistics, pages 3731-3741, Florence, Italy. Association for Computational Linguistics.
Thomas Scialom, Paul-Alexin Dray, Sylvain Lamprier, Benjamin Piwowarski, Jacopo Staiano, Alex Wang, and Patrick Gallinari. 2021. QuestEval:: Summariza- tion asks for fact- evaluation. In Proceedings of the 2021 Conference on Empirical Methods in Natu- ral Language Processing, pages 6594-6604, Online and Punta Cana, Dominican Republic, Association for Computational Linguistics.
Billy Wilder, Agatha Christie. Harry Kurnitz, Alexan- dre Trauner, Tyrone Power, Marlène Dietrich, and Charles Laughton. 1957. Wimnes for the Prosecution. United Artists.
Noah Shinn, Beck Labash, and Ashwin Gopinath. 2023, Reflexion; an autonomous agent with dynamic mem- ory and self-reflection. CoRR, abs/2303.1 11366.
Dustin Wright, David Wadden, Kyle Lo, Bailey Kuehl, Arman Cohan, Isahelle Augenstein, and Lucy Lu Wang. 2022. Generating scientific claims for zero- shot scientific fact checking. In Proceedings of the 60th Annual Meeting of the Association for Compu- tational Linguistics (Volume I: Long Papers), pages 2448 2460, Dublin, Ireland. Association for Compu- tational Linguistics.
Meiqi Sun, Wilson Yan, Pieter Abbeel, and Igor Mor- datch. 2022. Quantifying uncertainty in foundation models via ensembles. In NeurlPS 2022 Workshop on Robustness in Sequence Modeling.
Derek Tam, Anisha Mascarenhas, Shiyue Zhang, Sarah Kwan, Mohit Bansal, and Colin Raffel, 2022, Eval- uating the factual consistency of large language models through summarization. arXiv preprint arXiv:2211.0844
Ori Yoran, Tomer Wolfson, Ben Bogin, Uri Katz, Daniel Deutch, and Jonathan Berant. 2023. Answering questions by meta-reasoning over multiple chains of thought. arXiv preprint arXiv:2304. 13007.

---

Hiyori Yoshikawa and Naoaki Okazaki. 2023, Selective- LAMA: Selective prediction for confidence-aware evaluation of language models. In Findings of the As- sociation for Computational Linguistics: EACL 2023, pages 2017-2028, Dubrovnik, Croatia. Association for Computational Linguistics.
Polina Zablotskaia, Du Phan, Joshua Maynez, Shashi Narayan, Jie Ren, and Jeremiah Liu. 2023. On un- certainty calibration and selective generation prob- abilistic neural summarization: A benchmark study. arXiv preprint arXiv:2304.08633
Zexuan Zhong, Dan Friedman, and Danqi Chen. 2021. Factual probing is [mask]: Learning vs. learning to recall. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistic:: Human Language Tech- nologies. pages 5017-5033.
Zexuan Zhong, Tao Lei, and Danqi Chen. 2022. Train- ing language models with memory augmentation. In Proceedings of the 2022 Conference on Empir- ical Methods in Natural Language Processing, pages 5657-5673, Dhabi, United Arab Emirates. As- sociation for Computational Linguistics.
Chunting Zhou, Junxian He. Xuezhe Ma. Taylor Berg- Kirkpatrick, and Graham Neubig. 2022. Prompt consistency for zero-shot task generalization. arXiv preprint 2205: 00009..

---

A Prompts
Algorithm 1 Cross Examination
Input: A claim C generated by EXAMINEE
Table 10 provides the prompts used in our LMVLM approach.
Output: Correct Incorrect
Report
Q - EXAMINER (P. Setapps C)
B Additional Evaluation
Reurr & EXAMINEE Q)
Report +- Report + Q + Reurr
We follow the same experimental setting as in $4, but evaluate performance with respect to accep- tance of claims rather than rejection. Specifically, here we use the following definitions:
while EXAMINER (P Pfollow up, R) is "Yes" do
Q t EXAMINER jolloo up- R)
Rpres t- Reurr
Rearr + EXAMINEL (Rrrevs Q)
Precision: the portion of correct claims, out of the claims accepted by the examiner.
Report Report + Rourr
end while
Recall: the portion of correct claims accepted by the examiner, out of all the correct claims.
return EXAMINER (P docisionn C, Report)
In addition, we introduce an ensemble AYS LMvLM: for given claim, we first run the AYS method, and if the claim is rejected by this method we then apply LMvLM (Majority) to obtain a final decision.
E Example Cross- Fross-Examinations
Full cros-exxamination examples are provided Tables 14, 15, 16, 17, 18, 19, 20, 21.
Tables 11 and shows the evaluation results for the settings CHATGPT vs. CHATGPT and GPT-3 vs. GPT-3, respectively.
In terms of precision, our method outperforms the other baselines, often by large gap (e-g., 81.6 compared to < 60 by baselines for CHATGPT on PopQA, and 68.7 compared to < 52.4 for GPT-3 on PopQA). Moreover, this while maintaining good recall with respect to the baselines, except for AYS which has the best recall scores.
C Falsehoods Data
To generate a wrong claim, given a query Q for one of the QA datasets we use, we prompt our mod- els the following way: in case Q is in a question format, we use "Please answer the following question with a wrong answer: <Q>" and further request the LM to "Please also phrase your answer an argument". In case Q is in sentence-completion format, we use "Please complete the following sentence with wrong answer: <Q>" and further concatenate Q with the model answer. Table 13 introduces a few examples of these, generated by GPT-3. We man- ually verified that all the generated claims were indeed factually wrong.
D Cross Examination Algorithm
Algorithm provides psuedo-code for our method.

---

Stage
GPT3 Prompt(s)
ChatGPT Prompt(s)
(1) Setup
Imagine trying that a claim that someone claims is true, is wrong. You have the opportunity to ask any question order prove that the <C> What would you in order to validate that?
prove Your goal following claim:<C>, based the background information you will gather. To gather this, You will provide short questions whose purpose will be verify the correctness of the claim, and I will reply you with the these. Hopefully, with the help the background questions their answers, you will be able to reach whether the claim is incorrect. Please keep asking questions as long you' re yet sure regarding the true veracity of the claim. Please start the first questions.
try verify the correctness of the
claim is wrong. The claim
conclusion
or possibly
(2) Follow-Up (i) you have any follow-up questions? Please Questions
with Yes No.
(ii) What are the follow-up questions?
(3) Factuality Based the interviewee's answers to your questions, what is conclusion
Decision
regarding the correctness of the claim? Do you think it is correct or incorrect?
Table 10: Prompts provided to EXAMINER at each stage of the examination, with respect to a claim C' by EXAMINEE.
LAMA
TriviaQA
NQ
PopQA
P
R
F1
P
F1
F1
R
F1
IDK
88.6
87.2
87.9
88.9
79.3
72.0
79.5
59.7
68.7
63.9
AYS
84.8
98.7
91.2
80.3
98.7
88.5
60.6
95.6
74.2
53.5
88.2
66.6
LMvLM
93.3
97.1
95.2
91.9
95.6
93.7
81.2
86.7
83.9
75.0
82.0
78.3
LMYLM (Majority)
94.5
97.2
95.8
94.4
95.6
85.1
88.4
86.7
81.7
85.0
83.3
AYS+ LMvLM (Ensemble)
83.3
98.9
90.4
98.1
63.2
g
Table 11: Precision (P), Recall (R), both calculated with respect acceptance, rather than rejection (see B), and Fl scores of CHATGPT as EXAMINER and EXAMINEE.
LAMA
TriviaQA
NQ
PopQA
P
R
FI
P
R
FI
R
Confidence- Baade
94.8
65.4
77.4
91.1
55.2
68.7
84.8
30.2
44.5
52.4
42.6
47.0
IDK
85.4
85.9
85.6
81.4
82.7
82.
56.6
71.0
63.0
41.2
44.9
47.4
IC-IDK
85.2
96.6
90.5
84.1
92.8
88.3
64.1
75.4
69.3
51.8
85.5
64.5
AYS
82.6
98.5
89.8
77.9
98.3
86.9
54.1
93.1
68.4
47.7
90.1
62.4
LMvLM
92.6
95.2
93.9
88.5
94.9
91.6
68.5
72.2
70.3
64.4
71.3
67.7
LMVLM (Majority)
94.5
95.3
94.9
90.4
94.9
92.6
77.6
80.0
78.8
68.7
80.2
74.0
LMYLM (Ensemble)
98.7
88.7
76.0
98.9
85.9
52.8
98.5
68.7
46.6
91.9
61.8
Table 12: Precision (P), Recall (R), both calculated with respect to acceptance, rather than rejection (see ??), and Fl scores of GPT-3 3 as EXAMINER and EXAMINEE.

---

False claim
True claim
Dataset
"Windows an operating system developed by Apple."
"Windows an operating system developed by LAMA Microsoftt"
"The Hispaniolan lizard cuckoo (Coccyzus lon- girostris) species cuckoo the Cuculidae family.l is found in the Dominican Republic and family,I is found the Dominican Republic and Haiti." "
"The Hisponiolan lizard cuckoo (Coccyzus lon- LAMA girostris) is species cuckoo the Cuculidae
Honduras."
"The first modern electric battery demon- strated by Thomas Edison, an American inventor"
"The first modern electric battery was demon- TriviaQA strated by Alessandro Volta."
"I believe that the actor who played Rockford's
"The actor who played Rockford's 's father, "Rooky," " TriviaQA in the TV series, "The Rockford Files," was Noah Beery Jr."
father, "Rocky," in the TV series, "The Rockford Files," was Tom Selleck."
"The Taurus Mountains are located in the United States, specifically in the state of California."
"The Taurus Mountains are located the south- NQ em Turkey"
heard that Taylor Swift is doing the 2018 Super "Justin Timberlake was the feutured performer NQ Bowl Half Show."
the 2018 Super Bowl Half Time Show." "
"Red Velvet type of cake"
"Red Velvet is genre of music."
PopQA
"The author of The Jacket is J.K. Rowling, be- "The author of The Jacker is Andrew Clements." PopQA cause she is the author of the Harry Potter series."
Table 13: Examples from FALSEHOOD dataset and the corresponding correct claims.

---

EXAMINEE: The song 'I'm in the Mood for Dancing' written Irish songwr ite and musician, Ben
Findon.
EXAMINER:
1. Who is the artist that recorded the song "I'm in the Mood for Dancing"?
2. when was the song "I'm in the Mood for Dancing" first released?
3. Who are the listed songwriters for "I'm in the Moad for Dancing"?
4. What is the nationality of songwriter Ben Findon?
Are there any credible sources references that credit Ben Findon the writer of "I'm in the Mood for Dancing"?
EXAMINEE: As AI language model, I can answer the following questions based on available data:
1. The song "I'm in the Hood for Dancing" was originally recorded by the Irish family band The Nolan Sisters, Later known as The Nolans.
2. The song "I'm in the Moad for Dancing" was first released in 1979, the year in which it became major international hit.
3. The listed songwriters for "I'm in the Mood for Dancing" are Ben Findon, Mike Myers, and Robert Puzey.
Ben Findon is British songwriter.
credible sources that credit Ben Findon as the co-writer of "I'm in the Mood for Dancing. " The song is registered with several music licensing agencies, such ASCAP and BMI, which list Ben Findon listed in the liner notes of The Nolans' debut album of the
Yes, there
of the song's writers. Additionally, the credits for the song are also
EXAMINER: : Can you provide any other sources or references that credit Ben Findon as the co-writer of "I'm in the Mood for Dancing"?
language model, can provide additional sources or references that credit Ben Findon as the co-writer of "I'm the Moad for Dancing." "
EXAMINEE :
1. AllMusic, a music database and online resource, lists Ben Findon as one of the songwriters for "I'm in the Hoad for Dancing."
2. BMI, a performing rights organization, lists Ben Findon as a co-writer of "I'm in the Mood for Dancing" in their repertoire of musical works.
3. The official website of The Nolans also credits Ben Findon as one of the songwriters for "I'm in the Mood for Dancing."
4. The website of Discogs, database and online marketplace for music, lists Ben Findon as credited songwriter for the song.
5. The liner notes of The Nolans' album "I'm in the Mood..." credit Ben Findon, Mike Myers, and Robert Puzey as the song's s writers. These sources all provide additional evidence that Ben Findon is a co-writer of "I'm mm in the Mood for Dancing."
EXAMINER: : Based on the available information, the claim that "The song 'I'm in the Mood for Dancing' was written by Irish songwriter and musician, Ben Findon" is incorrect. It appears that Ben Findon is only one of the co-writers of the song, alongside Mike Myers and Robert Puzey. The song was originally recorded by the Irish family band The Nolan Sisters, later known as The Nolans.
Example of successful cross-examination with CHATGPT EXAMINER and EXAMINEE, for the claim "The song IIm in the Mood for Dancing' was written by Irish songwriter and musician, Ben Findon." (which is only partly correct, and has been labeled incorrect). For brevity, prompts shown.
Table 14:

---

EXAMINER: The producer of Precious was Lee Daniels.
EXAMINER:
you provide
of the producers involved the production of Precious?
director Precious,
Lee Daniels win any awards producing Precious?
what other movies has Lee Daniels produced?
Are there any interviews or articles where Lee Daniels specifically states that was the producer Precious?
EXAMINEE:
Sure, here list producers involved in the production of "Precious": Daniels, Gary Magness, Sarah Siegel-Magness, Oprah Winfrey, Tom Heller, and Tyler Perry. The director "Precious" Lee Daniels.
Yes, Lee Daniels several awards for producing "Precious," " including the Academy Award for Best Picture 2010.
produced several other films throughout his career, including "The Paperboy," "Monster's Ball," "The Woodsman," and "The Butler."
Lee Daniels
Yes, there are several interviews and articles which Lee Daniels specifically states that often credited as one the driving forces known his efforts bring story the screen.
was one the producers "Precious." In fact, behind the film's success
EXAMINER: Based the provided answers, the claim that the producer of "Precious" was Lee Daniels is correct.
Table 15: Example of successful cross-examination with CHATGPT EXAMINER and EXAMINEE, for the claim "The producer of Precious was Lee Daniels." (which is correct). For brevity, prompts are not shown.
EXAMINEE: : There are eight vertices (corners) on an octahedron
EXAMINER: :
What octahedron?
How many faces does an octahedron have?
How many edges does an octahedron have?
How many vertices does an octahedron have?
Are all vertices of an octahedron located at the same distance from the center?
EXAMINEE:
octahedran three-dimensionn geonetric shape that made up eight equal equilateral triangles. a polyhedron with eight faces.
octahedron has eight faces, each which is an equilateral triangle. edges that connect its eight vertices.
octahedron
octahedron has eight vertices, each of which is the point where three edges meet. No, not all vertices an octahedron are located at the same distance from the center. There are two types vertices those located at the center of each face, which are further away from the
octahedron: those the corners
shape, which are equidistant
from the center, center than the corner vertices.
EXAMINER: Based on the answers, the claim that "there eight vertices octahedron" is correct.
Table 16: Example of failed cross-examination with CHATGPT EXAMINER and EXAMINEE, for the claim "There are eight vertices (corners) octahedron." (which is incorrect). For brevity, prompts not shown.

---

EXAMINEB Augustus was the first Roman Emperor to sport beard.
EXAMINER:
1. Who Augustus?
what the definition of Roman Emperor?
Was it for Roman Emperors wear beards?
who were the previous Roman Emperors before Augustus?
there any historical records paintings that depict Augustus with beard?
EXAMINER:
Augustus the first Roman Emperor who ruled from 27 BCE to 14 CE. He the successor of Julius Caesar and the founder of the Roman Empire. A Roman Emperor military, judicial, and legislative powers. The position assassination, and Augustus the first to hold the title officially. No, it wasn't clean-shaven, and wearing beard perceived
political leader who held supreme authority in the Roman Empire, including established after Julius Caesar's
beards. In fact, most Roman Emperors symbol of being barbaric uncivilized. 4. The previous rulers before Augustus were known Roman Republic's leaders, such as Julius Caesar, Pampey the Great, Marcus Licinius Crassus, and others.
for Roman Emperors
There no known historical records paintings that depict Augustus with beard. Most of the depictions show with clean-shaven
or small mustache. Addi tionally, Augustus himself was known for promoting a clean shaven look, and he even encouraged his advisors and officials same.
EXAMINER: Based on the information that has been gathered, the claim that Augustus was the first Roman Emperor to sport beard is incorrect. In fact, it was not common for Roman Emperors to wear beards, and Augustus was known for promoting clean-shaven look. Additionally, there are no known historical records paintings that depict Augustus with beard.
Table 17: Example of successful cross-examination with CHATGPT EXAMINER and EXAMINEE, for the claim " "Augustus was the first Roman Emperor to sport a beard." (which is incorrect). For brevity, prompts are not shown.
EXAMINEE: The capital of Eritrea is Asmara.
EXAMINER: What is the official record of the capital of Eritrea according to the government?
EXAMINER: The official record of the capital of Eritrea according to the government is Asmara.
EXAMINER : Does the government of Eritrea recognize any other cities as official capitals?
EXAMINEE: No, the government Eritrea only recognizes Asmara as official capital.
EXAMINER: Based the answers my questions, conclude that the claim that the capital of Eritrea Asmara is correct.
Table 18: Example successful cross-examination with GPT-3 as EXAMINER and EXAMINEE, for the claim "The capital Eritrea is Asmara" (which is correct). For brevity, prompts not shown.

---

EXAMINEB Croatian composer Igor Stravinsky was born in Zagreb.
EXAMINER : Where was Igor Stravinsky born?
EXAMINER: Igor Stravinsky was Oranienbaum, Russia Lomanosov, Russia) on June 17,
1882.
EXAMINER: What was the nationality of Igor Stravinsky's parents?
EXAMINEB: Igor Stravinsky's parents were both Russian.
EXAMINER : Based on the information provided, the claim that Croatian composer Igor Stravinsky was born in Zagreb is incorrect.
Example of successful cross-examination with GPT-3 EXAMINER and EXAMINEE, for the claim "Croatian composer Igor Stravinsky was born in Zagreb" (which is incorrect). For brevity, prompts are not shown.
Table 19:
EXAMINEE: : Cristiano Ronaldo joined Manchester United in 2003.
EXAMINER : Which club did Cristiano Ronaldo join 2003?
EXAMINEE Cristiano Ronaldo joined Sporting CP in 2003.
EXAMINER: do have proof that Cristiano Ronaldo joined Sporting CP in 2003?
EXAMINEE: : Yes, there is proof that Cristiano Ronaldo joined Sporting CP in 2003. Sporting CP has released official documents confirming the transfer, and there are also numerous news articles and interviews from the time that confirm the move.
EXAMINER Based the evidence provided, Manchester United in 2003 is false. actually joined Sporting CP in 2003.
clear that the claim that Cristiano Ronaldo joined
Table 20: Example failed cross cross-examinann examination with GPT-3 as EXAMINER and EXAMINEE, for the claim "Cristiano Ronaldo joined Manchester United 2003" (which is correct). For brevity, prompts not shown.
EXAMINEE: The United States Capitol Building is topped by the Statue of Freedom.
EXAMINER: What is the actual figure that is located top of the United States Capi Building?
The actual figure located on top of the United States Capitol Building is bronze statue of the Roman goddess Freedom, which stands 19 feet 6 inches tall.
EXAMINER: :
EXAMINER: : Is the figure known as the Statue of Freedom?
EXAMINEE: No, figure not Liberty.
the of Freedom.
known the Statue
EXAMINER: Based the answers the questions, the claim that the United States Capitol Building topped by the Statue of Freedom is incorrect. The actual figure located top of the United States Capitol Building is bronze statue of the Roman goddess Freedom, which is not known the Statue of Freedom.
Table 21: Example failed cross exosinaxaminttoon with GPT-3 as EXAMINER and EXAMINEE, for the claim "The United States Capitol Building is lopped by the Statue of Freedom" (which is correct). For brevity, prompts not shown.


Reviews:
[{"review_id": "QVxsyMGlO6", "reviewer": "Reviewer_hr8G", "summary": "", "questions": "A. It seems the authors specifically aim at designing a method that does not require any training - of any form: pre-training, fine-tuning, ICL - on either the task itself or on any of the auxiliary sub-tasks (like falsehood generation).  Why do you take this a priori decision? I understand the benefit of using a \"black box\" LLM \"out of the box\" without any additional work - but at least for baselines, which trained approaches could you consider as comparison points?  (The baselines you compare to are not trained - even the \"Are you Sure\" methods which are trained in previous work are used here without training).\n\nB. Is there a basis to believe that LLMs are globally consistent across generations wrt a specific claim? How would you assess this claim?\n\nC. You generate a set of \"falsehoods\" as part of the process - do you aim for these falsehoods to be hard to detect? How can we measure the adversarial nature of your synthesized data? How can you increase its adversarial level?\n\nD. Section 5 identifies cross-examination strategies in observed data: Table 8 - different methods used by examiner LLMs: \nrephrasing claim / question / validation of implication / logical decomposition / request for attribution / wrong intermediate answers.\n\nThese are indeed good strategies - but isn't this a form of \"confirmation bias\"? A baseline would consist of constructively pursue these specific methods for validation. Why would we rely on the LLM to come up with \"validation strategies\" instead of instructing it to use these well known methods - either through ICL or through RLHF.  \n", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their thorough review and thoughtful comments. We are encouraged that the reviewer finds our method to be effective, the experimental analysis thorough and the analysis of previous work illuminating.\n\nWe address the reviewer’s concerns raised under “reasons to reject”:\n### Response to Concern 1\nIndeed, the main incentive has been to propose a method which requires as little effort as possible, by avoiding any kind of training or dependency on annotated training examples. \nAdditionally, in order to conduct a fair comparison with the other baselines, we have used most of them without additional training, except for the IC-IDK method which indeed utilizes ICL. We also note that many models of interest (e.g., ChatGPT) do not allow fine-tuning, and we also wanted to study those (we understand that ICL could still be an option for there).\nHowever, we definitely agree that this method (and the other baselines) might have benefited from additional training, including RLHF, but decided to leave it for future work.  \n\n### Response to Concern 2:\nThe purpose of this evaluation was mostly to achieve a high-level categorization of the  “Examiner LM'' generated text. We came up with the categories based on initial exploration of conversations, and then quantified their prevalence. Perhaps “strategies'' is a too strong  term, and we can use a different wording. We do think that the fact the model generates such conversations without explicit instruction is yet another testament to the complex skills that LLMs exhibit. Finally, the reviewer suggests (Q4) using prompting to induce specific interrogation behavior and this would indeed be very interesting to explore. \n\n### Response to Concern 3:\nIndeed it would be interesting to further quantify the phenomenon of inconsistency in factual knowledge. We note that Elazar et al. 2021 mostly evaluates consistencies with respect to different paraphrases, and here our concern is with more elaborate forms of consistency. \nBut, we note that if inconsistency (of the type LMvLM evaluated) had been the same for factually correct and incorrect facts, then our method would not have been able to improve performance on factuality testing (because true and false facts would have resulted in the same result). We also note that the results of Elazar were obtained for BERT, RoBERTa and ALBERT, and it is likely that models have since improved. We do agree it would be interesting to define several notions of inconsistency (e.g., negation, aliases, logical relations etc) and evaluate how LLMs behave on these. \n\n### Response to Concern 4:\nOur goal with falsehoods data was actually not to generate adversarial examples, but rather to generate facts we know to be false, and evaluate LMvLM on these. Some of these examples may in fact be “easy” to detect. Note that even for these non-adversarial examples, we do see up to 34.3% of the cases reported as true, so there is headroom even for this simpler case.\nWe however definitely agree with the reviewer that using hard negatives could produce a more challenging evaluation approach. One way to generate those would indeed be to use a trained classifier, or alternatively take cases where generation with temperature results in conflicting results. Following your suggestion, we will also report results on hard-negatives in the camera-ready. Thanks for this note!\n\n\nWe answer the reviewer’s questions raised under “Questions For The Authors”:\n### Answer to Question 1: \nPlease see our response to concern 1 above.\n\n### Answer to Question 2: \nIndeed our motivation for devising the method is that it is harder to answer consistently when one is lying, because it requires weaving a set of facts around the lie that is self consistent. This is why asking multiple questions is a way to detect lies in humans, and our expectation was that something similar would happen with LLMs, and our results indeed support this. The reviewer asks: “Is there a basis to believe that LLMs are globally consistent across generations”. Our results suggest that for facts that are true, LLMs provide more consistent answers, at least when consistency is measured by another LLM. It would be interesting to evaluate consistency itself more robustly by having a human provide the final “verdict” as to whether generation was consistent or not. We note that there is a recent line of work that evaluates LLMs on several “logically” related facts and sees if they produce consistent results (https://aclanthology.org/2023.findings-acl.733, http://arxiv.org/abs/2305.14956, http://arxiv.org/abs/2305.14795, https://arxiv.org/abs/2307.12976). It will be interesting to combine these directions with LMvLM.\n\n### Answer to Question 3:\nPlease see our response to concern 4 above.\n\n### Answer to Question 4:\nPlease see our response to concern 2 above.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "A. The paper addresses what is probably the most acute issue with current LLM technology - the risk of generating unsupported non-factual claims.  It provides an effective technique to detect such false claims that requires no training or fine-tuning.\n\nB. Experimental analysis is thorough - across multiple datasets and baselines and shows high performance.\n\nC. Analysis of previous work is illuminating - it offers a useful synthesis of different methods and trends grouped in informative manner.", "reasons_to_reject": "A. The motivation behind the approach is that LLMs have a tendency to produce globally consistent claims across multiple turns (072-076).\nThere is, however, no basis to hold this assumption as true.  [Actually - work of Eleazar et al 2021 aims at encouraging such consistency through fine-tuning]\n\nB. 087-092 - unclear what is the point of this paragraph: \"examiner LMs introduce multiple questions throughout the examination, and employ various strategies to reveal inconsistencies, including question paraphrasing, validation of implicated arguments, claim decomposition, and requests for evidence.\"\nIs it \"emerging behavior\" attributed to the \"examiner LMs\" or is it designed by the authors through clever prompting?\nI feel uneasy attributing such \"strategic planning\" capabilities to LLMs without any task-specific training.\nThe observations reported in Section 5 are not sufficient ground to establish the presence of such \"strategies\".\n\n\"this complex reasoning is naturally performed via the strong conversational skills of modern LMs\"\nI am worried about associating \"complex reasoning\" skills with conversation - especially when conversation includes ungrounded / non-factual claims (it is the whole point of the paper to detect these).\n\nC. The survey of previous work on \"Consistency across Generations\" is very well done - and it concludes with the statement: \"we build on this... assuming inconsistencies are more likely to occur with incorrect claims, and let an examiner model search for these by introducing questions to the examinee.\"  But actually - this statement is NOT building on previous work which demonstrated that LLMs do NOT have this property of triggering more inconsistencies with incorrect claims.   \n\nI would instead expect \"building upon previous work\" to include an experiment that measures empirically specifically whether self-consistency is more prevalent when starting from a false claim or a true claim, and whether this prevalence can be improved through intervention (training, specific prompting, ICL, fine-tuning).  \n\nD. 338-352 and Appendix C describe how the dataset of \"falsehoods\" is generated.  It sounds like the aim is to generate \"adversarial falsehoods\" (that is, falsehoods that are difficult to distinguish from true claims).  But the examples in Table 2 are not at all adversarial.  It seems that the process should be validated in a more robust way - by creating a falsehood classifier using simple training.\n\n\n\n\n\n\n", "questions_to_authors": "A. It seems the authors specifically aim at designing a method that does not require any training - of any form: pre-training, fine-tuning, ICL - on either the task itself or on any of the auxiliary sub-tasks (like falsehood generation).  Why do you take this a priori decision? I understand the benefit of using a \"black box\" LLM \"out of the box\" without any additional work - but at least for baselines, which trained approaches could you consider as comparison points?  (The baselines you compare to are not trained - even the \"Are you Sure\" methods which are trained in previous work are used here without training).\n\nB. Is there a basis to believe that LLMs are globally consistent across generations wrt a specific claim? How would you assess this claim?\n\nC. You generate a set of \"falsehoods\" as part of the process - do you aim for these falsehoods to be hard to detect? How can we measure the adversarial nature of your synthesized data? How can you increase its adversarial level?\n\nD. Section 5 identifies cross-examination strategies in observed data: Table 8 - different methods used by examiner LLMs: \nrephrasing claim / question / validation of implication / logical decomposition / request for attribution / wrong intermediate answers.\n\nThese are indeed good strategies - but isn't this a form of \"confirmation bias\"? A baseline would consist of constructively pursue these specific methods for validation. Why would we rely on the LLM to come up with \"validation strategies\" instead of instructing it to use these well known methods - either through ICL or through RLHF.  \n", "paper_topic_and_main_contributions": "The paper proposes a method to verify claims made by an LLM (presumably as answer to a question) through a process of cross-examination: one LLM (possibly a different instance of the same LLM) generates questions about the claim under consideration, and through a multi-turn process reaches a conclusion whether the original claim can be trusted.\n\nThe paper includes a good analysis of previous work that puts the effort in context: \n\n* Attribution and fact checking: check whether generated text is faithful to a given source text. Difference: the authors do not assume explicit source text, but test self-consistency.\n\n* Model calibration: prob that prediction is true / turned into categorical decision (statement is true).  Could extend work to calibration with continuous prob.\n\n* Multi-agent LMs: \"social skills\" - would like to see instead a discussion of why different LMs (with different parameters) would improve each other.\n\n* Consistency across generations - self-consistency models.\n\n* COT: improves mathematical, multi-hop, commonsense reasoning abilities.  The proposed approach can be thought as Chain of Thought generation of an explanation for a claim - but without ICL or fine-tuning.\n\nExperiments assess the performance of this cross-examination method on multiple datasets (LAMA / triviaQA / NQ / PopQA).\n\nMultiple baselines are considered:\n\n1. Confidence based: product of prob of each token (372).  (The Independence assumption must be checked.)\n\n2. Are you Sure (AYS): \"LLMs can be trained to estimate their certainty\" (Kadavath et al 2023, Cohen et al 2023) - but this is not trained here - zero-shot \"AYS\".\n\n3. IDK - add in prompt option to answer \"IDK\"  - Ganguli 2023\n\n4. IC-IDK - ICL for IDK prediction - trained on generated falsehoods\n\nTests: LMvLM (1 run) and LMvLM (3 runs - majority)\n\nResults are reported in three configurations:\n* ChatGPT vs ChatGPT\n* GPT3 vs GPT3\n* ChatGPT vs Llama 7b\n\nTable 3 shows the best results when using ChatGPT for cross-examination - and obtaining factuality judgment with highest F-measure compared to baselines.  For models that provide access to token-confidence, the confidence baseline is the best (Table 5) but cross-examination still performs better.\n\nA qualitative analysis of the cross-examination process identifies \"examination strategies\" that are employed by the examiner LLM: rephrasing claim / question / validation of implication / logical decomposition / request for attribution / wrong intermediate answers.\n\n\n\n\n \n", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "kGR5oqPDuX", "reviewer": "Reviewer_eV78", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 2, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "First, thanks for the time reviewing our work. \nWe respectfully disagree with the claim that “using a set of QA is an evaluation that trivializes the proposed method”. In fact, answering questions is a key capability of LLMs, and many of its uses are precisely for this task (e.g., a large fraction of search queries are questions. We also use such queries here via the NaturalQuestions dataset). And, within this important task of QA factuality is still an unsolved problem, which significantly hinders the use of LLMs. Thus, we think QA is a very appropriate and challenging platform for working on the problem of factuality. \nWe agree that there are other tasks where factuality can be evaluated (e.g., summarization, conversation, fact-checking), but we don’t see why these are better than the QA setting we use here.\n\nHere are a few more works which use QA as a format for evaluation: https://aclanthology.org/2021.emnlp-main.619/, https://aclanthology.org/2020.findings-emnlp.117/, https://doi.org/10.18653/v1/2022.findings-acl.158, https://arxiv.org/abs/2207.05221, https://arxiv.org/abs/2205.14334, https://arxiv.org/abs/2208.11057, https://arxiv.org/pdf/2302.09664.pdf, https://aclanthology.org/2023.findings-eacl.150/.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "This paper confirms the effectiveness of the proposed method in the experiment.", "reasons_to_reject": "Although the basic concept of this paper is very reasonable and I agree with it, I am concerned that the evaluation using a set of QA is an evaluation that trivializes the proposed method.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper's contributions are as follows:\n- framing the task of factuality testing as an interaction between two LMs,\n- proposing a concrete implementation of this interaction via the use of one LM with different prompts in a zero-shot setting, and\n- demonstrating improved factuality detection accuracy across several benchmarks.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "LCHZhYT5pp", "reviewer": "Reviewer_ZiT3", "summary": "", "questions": "- I personally did not find the experiment on false synthetic claims (Table 7) to be particularly interesting. It seems likely that the LLM would generate blatantly false claims (which is confirmed in the examples of Table 2), which leads to very high performance on this set. It is not clear why this experiment is needed in the paper. What does this experiment reveal that the other main experiment on real error cases does not?\n- One major question I have is: is the # questions asked correlated with a successful outcome? If so, one could imagine artificially \"forcing\" an LLM to generate more questions (by making it believe it decided to ask more questions), and measure whether this would lead to real gains in measure.\n- Do you have thoughts on applying the protocol on other task settings that involve factuality checking, such as summarization, KG-grounded generation, retrieval-enabled QA, etc.?\n- It seems like the ultimate objective should be to obtain a Examinee that produces better answers (higher numbers in Table 1). Can the framework accomodate for the Examinee to modify its answer?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their thorough review and thoughtful comments. We are encouraged that the reviewer finds our approach to be interesting and innovative, and the experiments to be broad. \n\n### Response to Weakness 1:\nIt’s indeed a very interesting question to explore different models and their interactions. We focused on three “popular” models, but indeed others are also interesting. We note that the “Examiner LM” needs to be able to understand and follow instructions, for successfully conducting the full “Cross-Examination” (this is why we didn’t use LLama as an examiner). We will add a discussion of these interesting questions in the camera-ready version. \n\n### Response to Weakness 2:\nThanks for this valuable suggestion! Indeed it is possible to also consider retrieval augmented QA. Technically this would be simple to do by replacing the examinee and/or the examiner with a retrieval augmented model. For the examiner, we don’t expect results in this case to be qualitatively different from regular QA. Namely, we expect the examiner to still probe the examinee with questions that seek inconsistencies, as with “context-less QA”. \nNonetheless, we agree that evaluating a stronger examinee that is augmented with retrieval is interesting and important. Therefore, we followed your response and re-evaluated “GPT3 vs GPT3” on NQ, but now using a retrieval-augmented examinee – We used DPR for retrieval and took the top 3 passages it retrieved from Wikipedia, concatenated them, and instructed the examinee (in the prompt) to answer the question based on the passages.\n\nIn terms of the examinee performance, we note a substantial improvement in accuracy (i.e., the fraction of questions answered correctly) from 50.1 to 66.4 when augmented with DPR. \n**For the performance of our method compared to the baselines, we observe similar trends to the “context-less QA” case**\n(see Table 4 in the submission, columns for NQ):\n\n| **Method** | **Precision** | **Recall** |  **F1** | \n| -------- | ------- | ------- | ------- |\n| AYS (+retrieval augmented examinee) | 80.2 | 20.1 | 32.1 |\n| IDK (+retrieval augmented examinee) | 69.5 | 46.0 | 55.4 |\n| Confidence-Based (+retrieval augmented examinee) | 62.6 | 71.9 | 66.9 |\n| LMvsLM (examinee without retrieval)  | **79.3** | **76.8** | **78.0** |\n| LMvsLM (+retrieval-augmented examinee)  | **87.5** | **75.7** | **80.3** |\n\nComparing the last two rows in the table, we see that augmenting the examinee with retrieval leads to a substantial increase in precision (79.3 → 87.5) with only a small decrease in recall (76.8 → 75.7) and an overall improvement of 2.9% in F1. \n**This shows that, alongside the improvement of the examinee, “LM vs LM” still performs well in detecting factual errors when they occur.**\nWe will provide more extensive experiments and discussion about that in the camera ready version.\n\n\nWe thank the reviewer for the great and insightful questions, which we address below.\n### Answer to Question 1:\nThe  “Falsehood” dataset is meant to be a controlled test for false-claim detection, which we think is useful as a general “sanity-check” in this setting, and useful to future works on factuality. We agree that could be made more challenging, however, there’s still a room for improvement as for example GPT3 (as an Examiner) fails on 25%-35% of the claims generated based on LAMA. Yet, we will modify the camera ready version to contain a “hard-negatives” version of this dataset. This can be done in various ways, including training a separate classifier to detect falsehoods, and filter out examples detected by the classifier. Thank you for this comment!\n\n\n### Answer to Question 2:\nIndeed we have observed (Section  4.3) that using follow-up questions helps the “Examiner LM'' achieve better results. Following your suggestion, we calculated the correlation coefficient between the success of the “Examiner LM” to the number of questions it asks, and obtained a value of 0.43. This indeed suggests a dependence between these variables. Consequently, we have conducted the following experiment: we randomly sampled 200 cases (50 from each of the four datasets we use) where the examination fails, and for each of the cases we did not allow the examination to end before it gets to at least 8 ‘follow-up questions’ iterations. Indeed, for 10% of the cases the factuality decision of the examiner had switched (that is, became to be correct). Therefore, in the camera-ready version we will report on additional ways of making the examiner ask more questions. \n\n### Answer to Question 3:\nThanks again for the great question! Our approach can indeed be adapted to other tasks where factual statements may be generated. Yet, automatically evaluating the approach on those settings may be not trivial as well, and requires additional manual effort to annotate the correctness of the “Examinee LM” generations. We will definitely add a discussion of this in the camera-ready version. \n\n### Answer to Question 4:\nGreat thought again! We agree that ultimately we would like to get the “Examinee LM” to produce more accurate answers. The point is it is reasonable to think that in some cases where the “Examinee LM” fails, it is really lacking the relevant parametric knowledge. One idea could be to use retrieval after applying our approach, in cases when the “Examiner LM” concludes that the claim is wrong, and thus to reduce the calls to the retriever. We will add a discussion of this direction in the camera-ready version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- A interesting & innovative approach to factuality verification in the context of closed-book QA, which is interpretable.\n- Broad experiments on 4 datasets, confirming the generalizability of the experiments.\n- The paper is convincing on the quality of the data, with the authors carefully describing their annotation procedure (for example regarding gold label assignment, or final assignment of the Examiner score), which adds evidence that the benchmark resource could be valuable to other researchers if released.", "reasons_to_reject": "Nothing major, but here are some notes:\n- It seems limiting that the authors only included experiments with three LM-pairs. There are several interesting questions to explore that could yield insights into how to apply the procedure most successfully. Namely: does the Examiner need to be more \"proficient\" at the initial task for this to be successful, or are less \"powerful\" Examiners still valuable in uncovering errors? How does the \"performance gap\" between a model pair impact performance. How does the performance of the Examiner on each particular sample impact its ability to be a good examiner? I find it unfortunate that 2/3 pairs included involve the same LM for both roles, it seems limiting.\n- Closed-book but open-domain QA seems somewhat synthetic. Nowadays, retrieval capabilities are mainstream. It seems likely that some of the models included in the experiments would be much less likely to make mistakes given retrieved evidence, rendering the experimental setup less valuable. Perhaps a Discussion section addressing this, and giving thoughts on how to adapt the framework to a setting with retrieval would be valuable. ", "questions_to_authors": "- I personally did not find the experiment on false synthetic claims (Table 7) to be particularly interesting. It seems likely that the LLM would generate blatantly false claims (which is confirmed in the examples of Table 2), which leads to very high performance on this set. It is not clear why this experiment is needed in the paper. What does this experiment reveal that the other main experiment on real error cases does not?\n- One major question I have is: is the # questions asked correlated with a successful outcome? If so, one could imagine artificially \"forcing\" an LLM to generate more questions (by making it believe it decided to ask more questions), and measure whether this would lead to real gains in measure.\n- Do you have thoughts on applying the protocol on other task settings that involve factuality checking, such as summarization, KG-grounded generation, retrieval-enabled QA, etc.?\n- It seems like the ultimate objective should be to obtain a Examinee that produces better answers (higher numbers in Table 1). Can the framework accomodate for the Examinee to modify its answer?", "paper_topic_and_main_contributions": "The paper presents the LMvLM framework, which is a method to assess factual errors in open-domain, closed-book QA answers of LLMs. In the framework, which resembles \"cross-examination\" in the legal domain, an \"Examiner\" repeatedly questions the \"Examinee\" with regard to their claim, and finally decides whether they believe the original answer of the model is accurate/inaccurate. The paper experiments with multiple generative QA datasets, and three LLM-pair settings (GPT3-GPT3), (CGPT-CGPT) and (Llama-CGPT). The results convincingly find that this conversational method of assessing accuracy of an answer is more performant than baselines that include asking the Examinee for its assurance, or using LM probability with a threshold. Analysis reveals that successful Examiners employ diverse strategies and often do multiple rounds of questioning.", "excitement": 4, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "- Regarding the name. The title uses \"LM vs LM\" and most of the paper mentions \"LMvLM\", which both seem a bit generic. I particularly enjoyed the \"cross examination\" analogy, why not go for something more specific and eye-catching such as \"CrossExam\"?"}}]

Author rebuttal:

Meta-review: This paper proposes a method for better calibration in question answering / fact probing through cross examination: where they use a series of paraphrased questions / verification questions to the LM in order to judge whether the model’s answer is correct or not. Results demonstrate consistent improvements over 4 datasets.

The reviewers acknowledged that the problem tackled in the paper is a critical problem (hr8G), experiments and analysis are insightful  (hr8G, ZiT3) and effectively demonstrate proposed method is effective (eV78, ZiT3), and description of prior work is comprehensive (hr8G, ZiT3).

Some concerns raised by reviewers include: (1) “False” claims used in the experiments are too synthetic/simple (h48G), (2) evaluation in QA is too simple (eV78), and (3) lack of more LM pairs and discussion on the choice of the pair (ZiT3). Lack of experiments with retrieval was also pointed out (ZiT3) but was resolved in author responses. The authors are encouraged to add these additional experiments in the paper and add clarification in the experimental setting in the final version of the paper if accepted.
Decision: Accept-Main