Fast and Accurate Factual Inconsistency Detection Over Long Documents (2023)

Abstract: Generative AI models exhibit remarkable potential; however, hallucinations across various tasks present a significant challenge, particularly for longer inputs that current approaches struggle to address effectively. We introduce SCALE (Source Chunking Approach for Large-scale inconsistency Evaluation), a task-agnostic model for detecting factual inconsistencies using a novel chunking strategy. Specifically, SCALE is a Natural Language Inference (NLI) based model that uses large text chunks to condition over long texts. This approach achieves state-of-the-art performance in factual inconsistency detection for diverse tasks and long inputs. Additionally, we leverage the chunking mechanism and employ a novel algorithm to explain SCALE’s decisions through relevant source sentence retrieval. Our evaluations reveal that SCALE outperforms existing methods on both standard benchmarks and a new long-form dialogue dataset ScreenEval we constructed. Moreover, SCALE surpasses competitive systems in efficiency and model explanation evaluations. We have released our code and data publicly to GitHub.

## introduction large language models (llms) have shown immense promise in various applications, but deploying them in real-time presents certain challenges such as hallucinations @xcite @xcite @xcite . hallucinations, or factual inconsistencies generated by a model relative to a source document, can mislead the user and undermine trust in llms. thus, detecting factual inconsistency in llm generations is crucial for the future of llms, especially with the growing popularity of platforms like chatgpt. prior research on inconsistency detection has predominantly dealt with short documents in offline settings @xcite @xcite and relied heavily on sentence-level text matching techniques. consequently, these methods exhibit slow performance in processing longer documents and suffer from poor calibration. such characteristics pose substantial challenges in implementing them in realworld online environments, where incorporating inconsistency detection could potentially result in a substantial increase in latency. additionally, the absence of well-calibrated scores complicates the balancing act between mitigating the risk of incorporating hallucinations and excluding pertinent information from the model output. given the exponential growth in context sizes (maximum allowed tokens of an input) of contemporary large language models (llms), foot_0 there is an increasing urgency to develop efficient and effective approaches for inconsistency detection in lengthy documents. in addressing the challenges, we introduce scale (source chunking approach for largescale inconsistency evaluation), a method designed for efficient detection of factual inconsistencies in generated sentences by identifying related source text snippets. scale consists of two crucial components. first, it builds on a natural language inference (nli) based method, integrating a novel chunking mechanism for rapid and accurate online performance in diverse natural language generation (nlg) tasks. second, model explanation is essential for real-time deployment of inconsistency detection systems, facilitating swift human inspection to determine model configurations. we show that our chunking mechanism improves calibration scores and enables the use of a binary search tree algorithm for rapidly locating relevant source text snippets for a target sentence, ultimately enhancing the explanation of model behaviors. current benchmark datasets for factual inconsistency detection predominantly feature short documents. in order to evaluate scale using a more realistic dataset with long documents, we introduce screeneval -a novel dataset designed to assess the factual inconsistency of summary sentences generated by humans, longformer, and gpt-4 in comparison to actual long-form dialogues. screeneval encompasses 52 dialogues, averaging over 6,000 tokens per dialogue. the use of dialogue in this dataset poses a considerable unique challenges such as long-distance coreference resolution and significant noise between utterances. to the best of our knowledge, screeneval is the longest dialogue based dataset for factual inconsistency detection presently available. in our experiments, we first show that scale outperforms and is better calibrated than baseline methods across various nlg tasks on the standard factual inconsistency detection benchmark true @xcite . we then assess accuracy, speed, and model explanation (via relevant text retrieval evaluation) on the new screeneval dataset for long document factual inconsistency detection. our findings indicate that scale surpasses strong competitors in the majority of tests. the key contributions of this paper are: ## related work factual inconsistency detection there are two main directions in factual inconsistency detection: natural language inference (nli) based and question answering (qa) based methods. in nli based methods, pretrained nli models can be utilized to determine whether a given "premise" factually entails a "hypothesis." although initial attempts encountered challenges @xcite , recent advancements have shown that nli models can effectively assess the factual consistency of generated text (hypothesis) with respect to a source (premise) @xcite . this progress can largely be attributed to addressing the granularity problem, which arises from the abundance of current nli datasets predominantly comprised of short, singlesentence premises and hypotheses @xcite @xcite . scale is an nli based method and our findings indicate that utilizing larger premise chunks enhances efficiency and outperforms sentence decomposition. although sentli @xcite extended nli based methods to longer documents, it adhered to the sentence decomposition assumption and focused solely on summarization tasks. summac @xcite investigated various aggregation techniques for nli scores obtained from sentence decomposition to generate overall summary scores. meanwhile, swing @xcite developed a loss function to train models for improved nli performance, yielding mixed outcomes. in qa based methods, a question is first generated based on a summary sentence, and a qa system is used to give an answer. a summary is considered factually consistent if the generated answer significantly overlaps with the original summary @xcite . prior research focused on using different question generation strategies @xcite or overlap measures @xcite . in the experiments, we consider the most competitive questeval @xcite and qafacteval @xcite . ## scale in this section we elaborate on our approach taken for our inconsistency detection model scale. firstly, we formally define the use of chunks and nli in scale, aiming at improving the accuracy and efficiency of the system. secondly, we propose to explain the model output by retrieving the relevant source sentence for a target, and show how the relevant sentence retrieval can be improved through the use of chunks. ## screeneval dataset we introduce a novel dataset for evaluating inconsistency detection on long form dialogues called screeneval. this dataset uses tv scripts and summaries pulled from the summscreen @xcite dataset. in addition to the provided human summaries, we generate summaries using longformer and gpt-4 on 52 scripts from the summscreen test set. we then hire human annotators to classify the factual inconsistency of each summary sentence and identify relevant supporting utterances for factually consistent summary sentences. screeneval is released publicly. details of how we use longformer and gpt-4 and collect human annotation can be found in the appendix a. the summscreen dataset is comprised of 2 sub datasets pulled from different sources forever-dreaming and tvmegasite. we use the forever-dreaming subset of summscreen, (summscreen-fd) to create screeneval due to its manageable summary size and diversity of shows and genres, spanning a total of 21 genres. summscreen-fd uses human-written gold summaries from wikipedia and tvmaze. screeneval our datasest screeneval compares the inconsistency detection ability of methods on long form dialogue. details can be found in sec. 4 ## conclusion in this paper, we introduce a cutting-edge nli based factual inconsistency detection method called scale. we show that scale is an nlg task agnostic factual inconsistency detection method by achieving state-of-the-art results across four distinct nlg tasks and 11 datasets within the true benchmark. we show that across nlg tasks scale is also superior in calibration, providing scores that are interpretable and enables accurate comparison of scores. we introduce a new dataset called screeneval that is the first of its kind for long form dialogue factual inconsistency evaluation. scale is shown to significantly outperform all models other than gpt-4 in factual inconsistency detection on this dataset. however, we show that scale is significantly more cost effective and faster than gpt-4 for evaluation on screeneval. moreover, we introduce a new retrieval strategy enabled by scale that significantly decreases the time to retrieve relevant utterances from a long document with increased recall accuracy.