Divide-Conquer-Reasoning for Consistency Evaluation and Automatic Improvement of Large Language Models (2024)

Abstract: Evaluating the quality and consistency of text generated by Large Language Models (LLMs) poses a significant, yet unresolved challenge for industry research. We propose , an automated framework for evaluating and improving the consistency of LLM-generated texts using a divide-conquer-reasoning approach. Unlike existing LLM-based evaluators operating at the paragraph level, our method employs a divide-and-conquer evaluator () that breaks down the paragraph-to-paragraph comparison into sentence-to-paragraph comparisons. To facilitate this approach, we also introduce an automatic metric converter () that translates the output from into an interpretable numeric score. Beyond the consistency evaluation, we further present a reason-assisted improver () that mitigates inconsistencies by leveraging the analytical reasons identified by . Through comprehensive and systematic empirical analysis, we show that our approach outperforms state-of-the-art methods by a large margin (e.g., + 16.8 % and + 32.5 % on the SummEval dataset) in consistency evaluation across multiple benchmarks. Our approach also substantially reduces nearly 90% output inconsistencies in one iteration, showing promise for effective hallucination mitigation in real-world industrial applications.

## introduction large language models (llms) such as gpt-4 and palm 2 @xcite have demonstrated impressive performance on a variety of natural language generation (nlg) tasks, including summarization @xcite , open-book question-answering (qa) @xcite , and retrieval-augmented generation (rag) @xcite . the evaluation of generated response quality often involves the assessment of the semantic equivalence between two pieces of text, e.g., between the generated response and the original text in summarization tasks or between two candidate responses in open-book qa tasks. however, conventional evaluation methods, such as bartscore @xcite and bertscore @xcite , which rely on token-level comparison, are inadequate for accurately and reliably measuring the quality of generated content, particularly in complex scenarios with long paragraphs @xcite . to address this issue, llmbased evaluators such as g-eval @xcite and @xcite have proposed a new framework that evaluates texts via paragraphlevel comparison. while these evaluators show promise for certain tasks, their scores often fail to achieve high concordance with human judgments of semantic equivalence. furthermore, as only numeric scores are provided with no explanation, it can be challenging for humans to trust or reason about these scores, particularly when using llms that are known to hallucinate @xcite @xcite . assessing the consistency of llms is more broadly connected to ai safety and has become a critical step in improving the reliability of these systems by preventing the generation of misinformation and harmful content. @xcite demonstrates that consistency checking can significantly enhance the chain of thought reasoning in llms. similarly, @xcite leverages semantic consistency for uncertainty estimation in nlg. recent studies employ consistency checking to detect hallucinations based on pre-trained llms @xcite and instruction-tuned llms @xcite . although these methods exhibit promising results on several specific tasks, including mathematical reasoning and factual assessment, the potential failures @xcite of self-consistency are often overlooked. this is essentially due to a lack of a generic, automatic, and reliable strategy that assesses the consistency of two responses, let alone remediating such inconsistency. in this work, we introduce a novel framework, divide-conquer-reasoning (abbreviated as dcr), for developing an automatic and reliable consistency evaluation method. our approach capitalizes on the intuition that human evaluators assess consistency by comparing the generated text to the reference text sentence by sentence and then combining the analysis to make a holistic judgment. unlike existing metrics that rely on either token-level or paragraph-level checks, our approach breaks down the paragraph-to-paragraph comparison into a series of sentence-to-paragraph comparisons. this approach avoids confusing llm by either providing too much information at once or zooming in too narrowly. additionally, our approach does not rely on llms to directly output verbal scores in a regression manner, which have been shown to be prone to hallucination. we note that dcr is a reference-free method, which does not rely on a golden reference written by the human expert. for example in a summary task, dcr does not need a sample summary and can compare directly between the target summary and the original paragraphs. ## preliminaries black-box llm evaluation. one of the drawbacks of current grey-box llm evaluations is that they require output token-level probabilities @xcite . however, prominent llms such as gpt-3.5, gpt-4, palm 2, and claude 2, are only available through restricted api calls. therefore, such token-level information might not be available. by contrast, we focus on the design of a black-box approach that remains applicable even when only text-based responses are available from the llm; that is, we only have access to the model output. limitation of existing methods. the conventional metrics, such as bertscore and bartscore, rely on a token-level comparison using n-gram or contextual embedding to calculate cosine similarity. however, this approach fails to capture the overall semantic meaning as it directly aggregates token-level similarities. to address this issue, leveraging the power of llms for self-evaluation has been proposed. g-eval @xcite and gpt-eval @xcite evaluate consistency at a paragraph level by prompting llms to compare two candidates as a whole. however, these ap-proaches have a major drawback as the generated verbal scores in a regression manner by llms are prone to hallucinations, resulting in abnormally higher ratings for llm-generated content that diverge from human judgment @xcite . such methods also generate no actionable insight to justify the score or mitigate inconsistencies. ## dcr framework to overcome the aforementioned limitations, we propose a divide-conquer-reasoning framework, which comprises three essential components: (1) dce disassembles the candidate paragraph, scrutinizes semantic inconsistencies using sentence-toparagraph comparison and outputs sentence-level inconsistency/consistency reasons, (2) amc converts such reasons into numeric scores for quantitative interpretation, and (3) rai conducts analytical reasoning to improve consistency through candidate regeneration. as illustrated in fig. 1 , dcr involves a combination of sentence-level analysis, semantic consistency checking, and causal analysis, making it an ideal pipeline for a diverse range of tasks such as summarization, questionanswering (qa), and retrieval-augmented generation (rag). moreover, dcr also improves the consistency of generated text through analysis and reasoning. fig. 2 provides an example of how dcr evaluates and enhances the consistency of the candidate text. ## industrial application dcr can be conveniently integrated into various industrial downstream applications, specifically in question-answering (qa), summarization, and retrieval augmented generation (rag) tasks, in both an online and offline fashion. in an online fashion, dcr enables auto-mitigation of hallucinations by reducing inconsistency before sending responses to customers. in an offline fashion, dcr empowers consistency evaluations and hallucination detection to gauge the reliability, trustworthiness, and trends of llm systems. ## conclusion we proposed a general evaluation framework based on a divide-and-conquer strategy for assessing the consistency between the llm-generated output and the reference texts across various nlg tasks. the proposed method can leverage analytical reasoning to generate revised text with improved consistency. through comprehensive and systematic empirical study across multiple benchmarks in semantic, factual, and summarization consistency tasks, we demonstrated that our approach significantly outperforms existing methods in evaluating and enhancing the consistency of llm-generated content. despite these advancements, we acknowledge several potential limitations of our proposed method, refer to appendix 8.