When Can LLM s Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLM s (2024)

Abstract: Self-correction is an approach to improving responses from large language models (LLMs) by refining the responses using LLMs during inference. Prior work has proposed various self-correction frameworks using different sources of feedback, including self-evaluation and external feedback. However, there is still no consensus on the question of when LLMs can correct their own mistakes, as recent studies also report negative results. In this work, we critically survey broad papers and discuss the conditions required for successful self-correction. We first find that prior studies often do not define their research questions in detail and involve impractical frameworks or unfair evaluations that over-evaluate self-correction. To tackle these issues, we categorize research questions in self-correction research and provide a checklist for designing appropriate experiments. Our critical survey based on the newly categorized research questions shows that (1) no prior work demonstrates successful self-correction with feedback from prompted LLMs, except for studies in tasks that are exceptionally suited for self-correction, (2) self-correction works well in tasks that can use reliable external feedback, and (3) large-scale fine-tuning enables self-correction.

## introduction self-correction is a popular approach to improve responses from large language models (llms) by refining them using llms during inference @xcite . extensive studies on self-correction have been conducted in various tasks, including arithmetic reasoning, code generation, and question answering @xcite . the simplest approach of self-correction prompts llms to provide feedback on their own responses and refine the responses using the feedback @xcite , under the hypothesis that recognizing errors is easier than avoiding them @xcite . as in figure 1 , self-correction has also been studied using additional information for improving feedback, including external tools such as code interpreters @xcite , external knowledge retrieved via web search @xcite , or fine-tuning @xcite . however, recent studies also report negative results indicating that llms cannot self-correct @xcite @xcite or even self-detect @xcite @xcite @xcite their own mistakes at least in certain conditions. these conflicting observations indicate that further analysis of self-correction is needed. in this work, we provide a critical survey to investigate the conditions required for successful self-correction. first, our analysis finds that prior studies often do not define their research questions in detail. as a result, many papers fail to provide appropriate experiments to evaluate the research questions they implicitly target. to address this issue, we categorize research questions in self-correction research ( §3.1) and discuss frameworks that should be used for verifying each research question ( §3.2). finally, we provide a checklist for designing appropriate experiments ( §8). next, we analyze prior work to identify when llms can self-correct their mistakes, using the new definitions of the research questions. our analysis highlights that the bottleneck is in the feedback generation ( §7). specifically, (1) no prior work shows successful self-correction with feedback from prompted llms in general tasks ( §4), (2) self-correction works well in tasks where reliable external feedback is available ( §5.1), (3) large-scale fine-tuning enables self-correction ( §5.2), and (4) some tasks have properties exceptionally suitable for self-correction ( §4). in summary, our analysis identifies the properties required for successful self-correction as follows: this survey is organized as follows. section 2 provides an overview of self-correction. section 3 introduces a new approach to classify research questions and frameworks in self-correction research. sections 4 and 5 analyze prior work in self-correction with in-context learning and external information (external tools, external knowledge, fine-tuning), respectively. section 6 explains related approaches that should be compared with self-correction as baselines. section 7 summa-rizes our findings from the analysis. section 8 provides a checklist for self-correction research. section 9 explains differences from other surveys. section 10 provides studies related to selfcorrection. section 11 provides future directions. ## self-correction of llms the term ''self-correction'' is used in a wide range of scenarios, from a strict definition in which llms refine their own responses by themselves @xcite to broader concepts that also involve feedback from external tools or knowledge @xcite . in this work, we define self-correction as a framework that refines responses from llms using llms during inference, possibly with external tools or knowledge. as in ## research questions we find that prior studies often do not define their research questions in detail and fail to use appropriate self-correction frameworks in their experiments. we propose a new approach to classify research questions and frameworks in selfcorrection. ## self-correction with external information this section analyzes self-correction frameworks that make use of external tools, external knowledge, and fine-tuning. ## strong baselines self-correction involves multiple llm calls for generating feedback and refinement. therefore, to claim that [rq3] the performance of the final outputs from self-correction frameworks is better than other approaches, it should be compared with sufficiently strong baselines, possibly relying on additional llm calls or computational cost. many self-correction studies do not compare their methods with strong baselines, although some studies pointed out this issue and compare self-correction with self-consistency @xcite or pass@k in code generation @xcite . we encourage future research to compare self-correction with strong baselines, including self-consistency and generate-and-rank, to further explore rq3. self-consistency. @xcite is an approach that generates multiple responses for the same input and takes the majority vote of the final answers in reasoning tasks. the idea of selecting good responses using the consistency between multiple responses from the same model has also been extended to other tasks such as text generation @xcite @xcite and code generation @xcite . generate-and-rank. is an approach that generates multiple responses and selects the best response using verifiers. post-hoc approach ranks responses using self-evaluation @xcite , confidence @xcite , fine-tuned verifiers @xcite @xcite , or ver-ifiers with external tools @xcite @xcite . feedback-guided decoding generates multiple responses and selects the best response for each reasoning step using generation probability @xcite , prompted self-evaluation @xcite @xcite @xcite , or fine-tuned verifiers @xcite @xcite . ## summary of our analysis bottleneck is in feedback generation. prior studies widely agree that llms can refine their responses given reliable feedback ( §5). however, generating reliable feedback on their own responses is still observed to be challenging for llms without using additional information ( §4). in other words, for the current llms, the hypothesis that recognizing errors is easier than avoiding them @xcite is only true for certain tasks whose verification is exceptionally easy, according to our analysis of the experiments in prior studies. we recommend that self-correction research analyze the quality of generated feedback in more detail, not only evaluate the downstream performance of the refined responses. tasks suitable for self-correction. our analysis identifies the properties of tasks that are suitable for self-correction under different conditions. editing human-written text. by using language models has been studied in various domains, including information update @xcite @xcite , grammatical error correction @xcite , factual error correction @xcite , and code repair @xcite @xcite @xcite . self-training. or self-improvement is an approach to train models using their own responses. some studies use self-evaluation or self-correction for creating training data @xcite or use self-evaluation as training signals @xcite . another approach improves the reasoning of llms using llm-generated reasoning by selecting highquality outputs using ground-truth final answers @xcite or self-consistency @xcite . as another direction, @xcite use sentences generated by llms with high confidence for training classifiers.