Large Language Models Can Self-Correct with Key Condition Verification (2024)

Abstract: Intrinsic self-correct was a method that instructed large language models (LLMs) to verify and correct their responses without external feedback. Unfortunately, the study concluded that the LLMs could not self-correct reasoning yet. We find that a simple yet effective prompting method enhances LLM performance in identifying and correcting inaccurate answers without external feedback.That is to mask a key condition in the question, add the current response to construct a verification question, and predict the condition to verify the response. The condition can be an entity in an open-domain question or a numerical value in an arithmetic question, which requires minimal effort (via prompting) to identify. We propose an iterative verify-then-correct framework to progressively identify and correct (probably) false responses, named ProCo. We conduct experiments on three reasoning tasks. On average, ProCo, with GPT-3.5-Turbo-1106 as the backend LLM, yields +6.8 exact match on four open-domain question answering datasets, +14.1 accuracy on three arithmetic reasoning datasets, and +9.6 accuracy on a commonsense reasoning dataset, compared to Self-Correct.Our implementation is made publicly available at https://wzy6642.github.io/proco.github.io/.

## introduction reasoning is a cognitive process that uses evidence, arguments, and logic to arrive at conclusions or judgements @xcite . people have been exploiting and improving the reasoning ability of large language models (llms). @xcite proposed chain-of-thought (cot) prompting and yielded promising results on several reasoning tasks, such as arithmetic reasoning @xcite , commonsense reasoning @xcite et @xcite , and open-domain question answering @xcite , using only a few or no reasoning exemplars. cot guides llms to generate intermediate reasoning paths instead of generating the final answer directly, which helps the llms simulate the human-like reasoning process. although cot enables llms to handle complex reasoning tasks, they are sensitive to mistakes in the reasoning path, as any mistake can lead to an incorrect answer. to address this issue, @xcite ; @xcite have explored verifying and correcting responses generated by llms. for example, as shown in figure 1a , for a given question and its initial llm-generated answer, @xcite first instructs the llm to criticize its generated answer using the hint: "review previous answer and find mistakes". then, self-correct instructs the llm to refine initial answers based on the critique. however, recent studies @xcite have cast doubt on the intrinsic selfcorrection capability of llms. their research indicates that without external feedback, such as input from humans, other models, or external tools to verify the correctness of previous responses, llms struggle to correct their prior outputs. since llms could not properly judge the correctness of their prior responses, the refined response might be even worse than the initial response. to improve the performance of llms in identifying and correcting inaccurate answers without external feedback, we introduce substitute verifi-(b) proco performs three steps: (1) initialization: use cot method to generate an initial answer. (2) verification: mask the key condition in the question and use the previous generated answer as a new condition to construct the verification question. solve the verification question to get the verified answer and check if the verified answer and the key condition are equivalent. if they are equivalent, the previous generated answer is adopted as the final answer, otherwise add it to the set of potentially incorrect answers. (3) correction: use the set of potentially incorrect answers as feedback to correct previous generated answer. by cycle executing step (2) and step (3), the performance of llms on various complex reasoning tasks is progressively enhanced. cation. let us look at a specific example. given an open-domain question "who plays skylar on lab rats: elite force?", we first prompt an llm to generate an initial answer for the question, e.g., "paris berelc". next, we identify a key condition in the question that is relevant to the problem-solving process, such as "skylar". by masking the key condition in the question and adding the initial answer as a new condition, we can obtain a verification question: "who plays x on lab rats: elite force? suppose the answer is paris berelc. what is the value of unknown variable x?". we use the llm to solve the verification question, and we get that x is "skylar storm". by verifying whether "skylar storm" is equivalent to "skylar", we can predict that the initial answer is likely correct. based on substitute verification, we propose a simple yet effective prompting method progressive correction (proco). figure 1 illustrates the difference between the self-correct and proco methods. compared with self-correct, our proposed proco highlights two primary distinctions: we conducted evaluations of proco using a variety of llms, including gpt-3.5-turbo-1106, gpt-4-0125-preview, and the open-source mixtral-8x7b. these evaluations spanned three distinct tasks: arithmetic reasoning, commonsense reasoning, and open-domain question answering. the experimental results reveal that proco consistently outperforms existing methods. as shown in in summary, our main contributions include: ## related work self-correct (kim et al., 2023) methods, which aim to enhance the quality of llm responses by providing feedback on initial attempts @xcite @xcite , have demonstrated effectiveness in various reasoning tasks. these tasks include arithmetic reasoning @xcite , open-domain question answering @xcite , commonsense reasoning @xcite , and others @xcite . self-correct methods vary in the source and format of feedback, and the process of verifying the correctness of llm output. source and format of feedback interscript @xcite corrected the llm's initial output by integrating natural language feedback from humans. due to the high cost of human feedback, scalar reward functions have been used as alternatives. for instance, rainer @xcite used reinforcement learning to generate contextual relevant knowledge in response to queries. self-correction @xcite trained a corrector to iteratively correct imperfect outputs. other sources, such as compilers @xcite or search engines @xcite can provide external feedback. recent research used llms to generate feedback. @xcite and self-refine @xcite utilized llms to ver- ## preliminaries given a question q, consisting of m context sentences {s j } m @xmath0. the query q ends with a question mark and is usually the last sentence of q. we can express @xmath1, where ⊕ denotes text concatenation function. we extract conditions {c i } n @xmath2, entities (open-domain question answering), and concepts (commonsense reasoning), as shown in figure 2. it is worth noting that usually @xmath3, if the question has one or multiple conditions. we denote j(i) ∈ {1, . . . , m} as the index of the context sentence containing the condition c i . among these conditions, the key condition c k is crucial for problem-solving and is used in the substitute verification process, where k is the index of the key condition within {c i } n @xmath4. we introduce two innovative approaches for identifying the key condition. ## conclusion in this study, we present a novel zero-shot prompting method for solving complex reasoning tasks. we name it progressive correction (proco), which first prompts an llm to generate an initial re-sponse, then iterates a verify-then-correct process to progressively identify and correct (probably) false responses. extensive experiments on eight complex reasoning datasets demonstrate the effectiveness and efficiency of our proposed method.