LM vs LM : Detecting Factual Errors via Cross Examination (2023)

Abstract: A prominent weakness of modern language models (LMs) is their tendency to generate factually incorrect text, which hinders their usability. A natural question is whether such factual errors can be detected automatically. Inspired by truth-seeking mechanisms in law, we propose a factuality evaluation framework for LMs that is based on cross-examination. Our key idea is that an incorrect claim is likely to result in inconsistency with other claims that the model generates. To discover such inconsistencies, we facilitate a multi-turn interaction between the LM that generated the claim and another LM (acting as an examiner) which introduces questions to discover inconsistencies. We empirically evaluate our method on factual claims made by multiple recent LMs on four benchmarks, finding that it outperforms existing methods and baselines, often by a large gap. Our results demonstrate the potential of using interacting LMs for capturing factual errors.

## introduction modern language models (lms) often generate inconsistent @xcite , non-attributable @xcite @xcite , or factually incorrect text @xcite @xcite , thus negatively impacting the reliability of these models @xcite . this has prompted the community to develop methods that calibrate the confidence of model predictions to better align with their quality @xcite . for example, prior methods have used probabilistic approaches @xcite clustering @xcite , finetuning @xcite and in-context learning @xcite . in this work, we take a different approach to this problem, motivated by truth-seeking mechanisms in law. specifically, we consider the setting where a witness is cross-examined in order to check whether their statement is factually correct or not. in such a setting, the examiner asks questions that aim to lead towards contradictory statements by the witness, and a contradiction implies that the witness lied at least in some of the statements. hence the well known quote "were you lying then or are you lying now?" @xcite . to employ this mechanism to lm factual calibration, we propose the following setting, illustrated in figure 1 . our goal is to check whether a statement made by an lm ("augustus was the first roman emperor to sport a beard") is factually correct. we refer to the model that generated this statement as the examinee. to check whether this fact is correct, we use another lm, called examiner, to conduct a cross-examination of examinee. concretely, we craft designated prompts to facilitate a multi-turn interaction between the two lms, where examiner issues questions (e.g., "was it common for roman emperors to wear beards?") to exami-nee to check the veracity of the original statement. the examination is concluded by a decision from examiner as to whether the original claim was correct or not. 1 our problem setting is related to that of calibration @xcite , where the goal is to predict the probability at which a model will err. however, unlike previous approaches to this problem, we use text generated by lms. our approach is motivated by the intuition that calibration is actually an elaborate reasoning process where one checks the level of support that a fact has, based on other statements the model believes. we argue that such complex reasoning is naturally performed via the strong conversational skills of modern lms. we use our method to detect errors in lm generation in the context of factual question-answering. ## lm cross-examination our goal is to employ an "examiner" lm (examiner) to evaluate claims generated by another lm (examinee). to this end, we leverage the recent success of prompting @xcite , to facilitate a cross-examination setting between the two lms. in such a setting, examiner should introduce questions with the objective of revealing inconsistencies with respect to an initial claim made by examinee. such inconsistencies can be considered as a signal for uncertainty of exami-nee in its original claim, and thus can be used to assess whether its original statement was correct. given an examiner lm and a claim c generated by an examinee, our method establishes a multi-turn interaction between the lms, where at each turn the other lm is prompted with a designated prompt that incorporates the outputs from previous turns. this interaction continues until the examiner has no further questions and can provide its final decision. to establish a meaningful interaction that reveals possible inconsistencies, we define three stages for the examination, each guided by a specific prompt. as part of each prompt for ex-aminee or examiner, we provide the outputs generated in the previous rounds for context. we next describe the examination stages in detail, with the overall process illustrated in figure 2 . stage 1: setup the examination begins by "assigning" the examiner its role. namely, describing the task setting, providing it with the exam-inee's claim, and asking it to generate questions for the examinee. 2 next, we feed the questions generated by exam-iner, one at a time, to examinee, concatenated to the following instructions: please answer the following questions regarding your claim. the response from examinee yields a set of answers to the questions from examiner. stage 2: follow-up questions we next feed ex-aminer with the answers generated by examinee to its initial questions, and ask examiner whether it has any follow-up questions. notably, outputs from examiner at this stage are conditioned on the answer (1) setup …the claim is: <c>. what would you ask in order to validate that? stion q please answer the following questions regarding your claim: <@xmath0. what are the follow-up questions? (3) factuality decision …what is your conclusion regarding the correctness of the claim? yes no @xmath1. c examination questions answers follow-up questions the previous output from examinee. if the answer from examiner is "yes", we then further prompt it to obtain more questions. this phase is conducted iteratively, until either examiner declares it has no follow-up questions, or the number of turns has reached a threshold. 3 stage 3: factuality decision once no further questions are obtained from examiner, we prompt it to conclude whether the claim c is true or false. specifically, we request it to reply with either "correct" or "incorrect" as its final conclusion. in cases where the examiner does not output either of "correct" or "incorrect", we consider its final decision to be a rejection of the claim. 4 typically though, we observe that the examiner follows the instructions and indeed generates a definitive conclusion (see statistics in §5). ## related work attribution and fact checking our goal is closely related to works that check if lm-generated texts are faithful to a given source text @xcite . this problem has been addressed via several approaches, including question generation @xcite @xcite , nli @xcite @xcite @xcite , data augmentation @xcite @xcite , and planning schemes that allow the model to self-edit its own generation @xcite . unlike these works, we are not assuming any reference text or external knowledge bases. instead, we directly check if the lm's claim is likely to be correct, by probing the model for inconsistencies. our approach also uses multi-turn dialogue as a key component. model calibration a key challenge with prediction models is to provide a probability of the answer being incorrect, a problem known as model calibration @xcite . the problem of factual-error detection can be viewed as a variation of calibration, where instead of a continuous probability, we provide a binary prediction for whether the model is correct or not. this is also related to the setting of selective prediction, where models can abstain from answering a query @xcite . common approaches to calibration are to perform various transformations on model logits @xcite , and measuring uncertainty (e.g., see @xcite . more recent works have studied the use of lms for providing calibration, by training them on statements known to be factually correct or incorrect. this "supervised" approach has been explored via fine-tuning @xcite and in-context learning @xcite . we focus on zero-shot factual error detection that involves two categories: predicting whether a model's claim is correct or incorrect. while we focus on a binary setting, one could envision extensions of our approach to continuous outputs (i.e., the probability that the claim is correct). multi-agent lms using multiple lms in an interactive manner is a relatively new idea with many potential applications. it has been shown that lms can utilize additional lms or tools to enhance safety or better solve downstream tasks @xcite @xcite . @xcite showed that in a social setting, lms demonstrate certain social skills that emerge from this interaction, and @xcite proposes that an lm can use a different model to instruct it when to "reflect" on its recent action, while performing a planned sequence of actions aimed at solving a given query. intuitively, this model detects signs of hallucination or inefficient planning within the lm's trajectory. ## experiments in this section, we conduct experiments on multiple datasets and models to evaluate our approach, focusing on the task of factual question-answering. ## analysis of cross-examinations we analyze cross-examinations by gpt-3 and chatgpt to better understand the success and failure cases of our method. we find that examiner lms typically ask multiple questions in the examination, and perhaps surprisingly, seems to apply different strategies to reveal inconsistencies. we note that furhter analysis is required in order to better understand whether the examiner indeed utilizes the conducted examination in its factuality decisions. examination statistics ## conclusion we introduce lmvlm, a method for zero-shot detection of factuality errors. our method uses prompting to facilitate multi-turn interactions between an two lms, to reveal inconsistencies that imply factually incorrect claims. lmvlm builds on a fundamental connection between selfconsistency (i.e., consistency of an lm with itself) and factual consistency (i.e., consistency between claims generated by an lm and ground-truth facts). we consider the lm as the source of information, and we test whether a claim it has generated is consistent with other beliefs it has. our work can be extended in several ways. first, lmvlm provides interpretable information about related beliefs of the model, which could be analyzed to understand what makes the model commit certain mistakes. second, one may incorporate several lm instances into the factuality detection process, rather the having only a single examiner. finally, one can train the examiner to generate questions more effectively.