I nterrogate LLM : Zero-Resource Hallucination Detection in LLM -Generated Answers (2024)

Abstract: Despite the many advances of Large Language Models (LLMs) and their unprecedented rapid evolution, their impact and integration into every facet of our daily lives is limited due to various reasons. One critical factor hindering their widespread adoption is the occurrence of hallucinations, where LLMs invent answers that sound realistic, yet drift away from factual truth. In this paper, we present a novel method for detecting hallucinations in large language models, which tackles a critical issue in the adoption of these models in various real-world scenarios. Through extensive evaluations across multiple datasets and LLMs, including Llama-2, we study the hallucination levels of various recent LLMs and demonstrate the effectiveness of our method to automatically detect them. Notably, we observe up to 87% hallucinations for Llama-2 in a specific experiment, where our method achieves a Balanced Accuracy of 81%, all without relying on external knowledge.

## introduction human studies have shown that people tend to be inconsistent when they are not telling the truth @xcite . as such, a common interrogation technique consists of repeated interviews that attempt to challenge the interviewer's consistency in order to assess its credibility @xcite . truth tellers' answers are well-grounded in their memory, hence, inconsistencies in the respondent's answers are a strong indication of her not telling the truth @xcite . inspired by these studies, we present a novel method for hallucination detection in llms. our approach, which we call interrogatellm, employs a systematic evaluation of model-generated responses for potential hallucinations by repeating the process of reconstructing a query from its generated answer. repeated interviews are a very common and effective verification technique for human interrogations, however, it is not foolproof. in some cases, respondents manage to provide repeated false states that are consistent, while in other cases, truth-tellers may provide inconsistent responses due to memory errors @xcite . in a similar fashion, our method is not flawless; it represents an additional step towards addressing the yet unsolved problem of hallucination detection. nevertheless, similar to the use of consistency tests in humans, commonly employed for their effectiveness, our method also demonstrates high efficacy. in recent years, the emergence of llms such as gpt-3 @xcite , palm @xcite , and llama @xcite has revolutionized natural language processing. these models enable machines to understand and generate human-like text with unprecedented fluency and coherence. trained on vast amounts of text data, they have demonstrated remarkable capabilities in various applications, from automated content generation to virtual assistants, and beyond. however, their remarkable performance comes with a set of challenges and concerns that need to be addressed for their responsible and effective use. a major concern is the phenomenon of hallucination, whereby these language models generate misleading, potentially harmful, or erroneous text. hallucination can be characterized by the presence of false information in the output generated by the language model that lacks a factual basis. there are significant challenges associated with the deployment of large language models in real-world applications, especially in those involving critical information or decision-making processes. detecting and minimizing hallucinations in llms is crucial for ensuring their trustworthiness and reliability, especially in contexts where these models play a pivotal role in communication and decision-making. existing methods for evaluating model-generated text often rely on surface-level metrics such as fluency and coherence, which may not effectively capture the underlying issue of hallucinations. therefore, there is a pressing need for a systematic and effective method to detect and mitigate hallucinations in the outputs of these models. despite its significance, addressing this challenge remains an open issue @xcite . our method, interrogatellm, operates on the premise that language models exhibiting hallucinations produce inconsistent and incorrect responses to subsequent queries based on the hallucinated information. to identify hallucination in a generated answer, our approach involves prompting the model multiple times to reconstruct the input query using the generated answer. subsequently, interrogatellm quantifies the inconsistency level between the original query and the reconstructed queries. by leveraging the observed inconsistencies, our approach effectively identifies potential instances of hallucination. when a large-language model generates a hallucination, it struggles to consistently reconstruct the original query, leading to variations in responses. this interrogation strategy serves as the cornerstone of our approach for detecting hallucinations in generated answers. the contributions of our paper are outlined as follows: (1) introduction of the interrogatellm method designed for detecting hallucinations in textual answers generated by llms. (2) we propose an innovative evaluation approach specifically tailored to the task of hallucination detection, leveraging three datasets associated with our proposed text generation tasks. (3) we investigate the hallucination levels exhibited by recent llms, including llama2, shedding light on their fidelity levels. (4) we present comprehensive performance reports on interrogatellm and its variants, conducting a thorough comparison with alternative methods through extensive evaluations. ## related work hallucinations have been explored in various natural language generation tasks, including translation, summarization @xcite , dialogue generation @xcite , and question-answering @xcite . this is well-documented in a recent comprehensive survey conducted by @xcite , which provides an insightful overview of hallucinations in diverse natural language generation contexts. in @xcite , the authors presented a token-level reference-free hallucination detection task along with an additional dataset designed for hallucination detection in free-form text. this dataset consists of textual passages with perturbations, and the objective is to determine whether the entire passage exhibits hallucinations. it is crucial to emphasize that our task differs from their setup, as we specifically address hallucination detection within few-shot prompts involving query-answer sequences. to address inconsistencies in generated text, selfcheckgpt, introduced by @xcite , leverages multiple stochastic samples generated by llms using the same query. selfcheck-gpt evaluates the coherence between the response and the stochastic samples by querying the same llm multiple times. specifically, it incorporates an additional prompt that includes a stochastic sample and a sentence from the generated text and predicts whether the sentence is supported by the stochastic sample. the approach validates each sentence by conditioning the llm on each stochastic sample. the methodology of selfcheckgpt encompasses various approaches, including one based on bertscore @xcite , and another employing a multiple-choice question answering and generation approach (mqag) @xcite , as well as n-gram and llm-prompting. our method is benchmarked against this baseline, using the last approach in our study. in recent research, @xcite proposed a method employing a multilayer perceptron classifier that uses hidden representations from language models to predict sentence truthfulness. however, this approach necessitates labeled data for supervised training and access to the internal states of the language model, which may not always be readily available. in @xcite , the authors present a self-evaluation technique where models are trained to predict their knowledge of the answer to any given free-form question. this approach entails prompting the language model to internally assess the accuracy of its previous predictions, including estimating the likelihood that its generated response or answer is correct. it is worth noting that this method requires labeled data for model training, making it a supervised task, which differs from our settings. query 1 answer 1 .... query n answer n query q a* embedding model average few shot query-answer answer 1 query 1 .... answer n query n answer a* few shot answer-query repeat k times (1) (2) (3) cosine similarity q embedding vector 1 2 k embedding vector figure 1: an illustration of the interrogatellm method. (1) a few-shot prompt and a query are fed into f llm , which generates an answer. (2) the shots in the prompt are then reversed, forming a sequence of answer-question pairs, with the generated answer constructed on top. the b llm is then used to generate k queries that correspond to the generated answer. ideally, the generated queries should recover the original query from the forward phase. (3) the set of recovered questions is then embedded by a language model and compared with the original question, producing a final score that determines whether the generated answer suffers from hallucination. ## problem setup we assume a source domain of textual queries and a target domain of textual answers. a fewshot prompt foot_0 @xcite , a corresponding query q and a llm denoted by f llm , are provided. the query is constructed on top of the prompt and fed into the llm to generate an answer to the query. our task is to detect whether the generated answer suffers from hallucinations. the few-shot prompt is constructed as a sequence of query-answer pairs. the pairs are denoted by {(q i , a i )} n @xmath0, where q i represents a query and a i its corresponding answer. the prompt can be expressed as follows: the f llm is queried with the concatenation of the query q on top of the prompt p f orward , which retrieves a generated answer denoted by a * , signifying the response to the query q. in other words, the prompt p f orward and the query q are fed into the llm as follows: our task is to determine whether the generated answer a * exhibits hallucinatory content. ## the interrogatellm method in our approach, we introduce a backward process for reconstructing the original query q from the generated answer a * . we create a new prompt by reversing the given prompt p f orward . the reversed prompt rearranges the order of the queryanswer pairs to pairs of answer-query. the reversed prompt, denoted as p backward , can be expressed as follows: the generated answer a * is then concatenated to the end of the reversed prompt p backward , and the entire sequence is passed either by the same llm defined above, by a different llm, or by an ensemble of llms. for ease of reference and clarity, we collectively refer to the llms involved in this step as b llm . in other words, in this process, we map the generated answer to the source domain, by querying one or more llms, each trying to reconstruct the original query q. by denoting the set of reconstructed queries as q * , this "backward" process can be expressed as: algorithm 1: hallucination detection input: q, fllm , bllm , m embedding , in interrogatellm, this backward process is repeated multiple times (k times for each model in b llm , see sec. 5.2 for more details), with variable temperature values, as explained below. therefore, to determine if a * suffers from hallucination, a language embedding model is utilized to assess the similarity between the set of reconstructed queries q * and the original query q. both the generated queries and the original query are transformed into vectors within the same embedding space. for a given embedding model m embedding : @xmath1, which generates d-dimensional vectors from the input text, the embedding vector for the original query q is denoted as q @xmath2. similarly, the embedding vectors for the generated queries are denoted by: subsequently, the cosine similarity between the embedding vectors of the predicted queries q * vec and the original query q vec is calculated as follows: here, c represents the cosine similarity function: for u, @xmath3, where d is the dimension of the vectors. in other words, the cosine similarity is calculated for each q * vec in the set q * vec , and the results are then averaged to obtain the final similarity score. finally, interrogatellm predicts hallucinations if the similarity score exceeds a predetermined threshold τ . in essence, when the reconstructed queries exhibit a significant divergence from the original query, interrogatellm signifies that there is a potential hallucination in a * . more details about the selection of τ can be found in sec. 5.2. the interrogatellm method is illustrated in fig. 1 , and outlined in alg. 1. ## experiments to assess the efficacy of interrogatellm in detecting hallucinations, and due to the absence of prior datasets for hallucination detection in few-shot prompt settings, we adapted three public datasets. for each dataset, we designed a text generation task along with a verification process to ensure the accuracy of the generated answers. the verification is implemented by employing simple heuristic functions that exploit additional information that is present in the datasets. during the evaluation of hallucination detection methods, the detection predictions are compared against the verification results. importantly, the interrogatellm method operates independently of any external knowledge, making it versatile and applicable to a broad spectrum of tasks. ## conclusion in this paper, we investigate the pressing issue of hallucinations in large language models. we introduced interrogatellm, a novel method designed for detecting hallucinations in few-shot settings. our work contributes to the ongoing dialogue on the responsible use of ai-powered language models, offering a method that contributes to the reliability of llm in diverse real-world applications. as a future work, we would like to extend the method to retrieval augmented generation settings, where a query is provided with a retrieved context, and the task is to generate an answer based on the provided information in the context.