Can LLMs Learn from Previous Mistakes? Investigating LLMs’ Errors to Boost for Reasoning (2024)

Abstract: Large language models (LLMs) have demonstrated striking reasoning capability. Recent works have shown the benefits to LLMs from fine-tuning golden-standard Chain-of-Thought (CoT) rationales or using them as correct examples in few-shot prompting. While humans can indeed imitate correct examples, learning from our mistakes is another vital aspect of human cognition. Hence, a question naturally arises: can LLMs learn and benefit from their mistakes, especially for their reasoning? This study investigates this problem from both the prompting and model-tuning perspectives. We begin by introducing COTERRORSET, a new benchmark with 558,960 questions, each designed with both correct and error references, and demonstrating the types and reasons for making such mistakes. To explore the effectiveness of those mistakes, we design two methods: (1) Self-rethinking prompting guides LLMs to rethink whether they have made similar previous mistakes; and (2) Mistake tuning involves finetuning models in both correct and incorrect reasoning domains, rather than only tuning models to learn ground truth in traditional methodology. We conduct a series of experiments to prove LLMs can obtain benefits from mistakes in both directions. Our two methods offer potentially cost-effective strategies by leveraging errors to enhance reasoning capabilities, which costs significantly less than creating meticulously hand-crafted golden references. We ultimately make a thorough analysis of the reasons behind LLMs’ errors, which provides directions that future research needs to overcome. COTERRORSET will be published soon on https://github.com/YookiTong/ Learn-from-Mistakes-CotErrorSet.

heading: 1 Introduction
text: Large language models (LLMs) (Brown et al., 2020; Zhang et al., 2022; Anil et al., 2023; Tou-
∗†Corresponding author.
vron et al., 2023) have demonstrated strong capabilities across various tasks and applications (Liang et al., 2022; Chang et al., 2023). To further unleash the reasoning abilities of LLMs and align their thinking process with humans, many recent studies explored Chain-of-Thought (CoT)-based prompting (Wei et al., 2022; Wang et al., 2022; Li et al., 2023a; Tong et al., 2023; Yao et al., 2023; Besta et al., 2023) to instruct LLMs to solve the given problem with human-like logic. Besides logical step-by-step thinking, another critical learning pattern of us humans is to rethink and learn from our previous mistakes so that avoid repeating the same mistakes in the future (Mercer, 2008; Reich et al., 2023). However, few studies have focused on systematically understanding what kinds of intermediate errors occur in making CoT procedures and whether LLMs can learn from those mistakes. To address these issues, we aim to explore the potential of LLMs to effectively utilize their previous mistakes to boost reasoning.
3065
To enhance the scalability and efficiency of analyzing and learning from the mistakes of LLMs, we began by collecting a vast dataset of LLMs’ reasoning outputs and built COTERRORSET, which consists of 609,432 questions sourced from 1060 tasks across diverse domains. Each query in this set is meticulously structured, featuring both a manually curated correct reference and the incorrect rationales collected from PaLM2 (Anil et al., 2023)’s responses. Furthermore, we prompt the LLMs with the correct reference and the incorrect responses in order to make it reflect why making such mistakes. The introspective responses are also collected and subsequently utilized in our work. We employ this data for cluster analysis to identify specific details of the errors.
With our COTERRORSET, we introduce two innovative paradigms, namely mistake tuning and self-rethinking, aimed at efficiently augmenting LLMs by leveraging their historical errors during both tuning and inference stages. Diverging from the conventional approach of only relying on correct rationales in traditional supervised fine-tuning, our mistake tuning strategy incorporates combinations of both correct references and incorrect rationales. To facilitate the learning process for LLMs, we introduce the prefixes [CORRECT RATIONALE] and [INCORRECT RATIONALE] before the corresponding rationales. Intuitively, this prompt tuning facilitates LLMs to distinguish between correct and incorrect rationales while avoiding corruption from the incorrect ones with the two separated prefixes. For self-rethinking, inspired by contrastive in-context learning (Gao and Das, 2024), we expose LLMs to both correct and incorrect rationales in demonstration samples. After obtaining the initial answer output by the LLM, we iteratively prompt it to rethink and rectify the result based on the historical mistakes. To manage computational resources and prevent potential loops, we implement a threshold, limiting the number of times the model can engage in self-rethinking and corrections. Figure 1 gives an overview pipeline of our work.
To substantiate the efficacy of our proposed methodologies and to delve into the learning capabilities of LLMs from their mistakes, we undertake experiments encompassing diverse reasoning tasks and LLMs of varying sizes. The application of our methods consistently yields performance enhancements across a spectrum of tasks, underscoring the effectiveness and broad applicability
of our approaches in leveraging LLMs’ mistakes during both the tuning and inference stages. Additionally, we conduct thorough analyses of the error types exhibited by LLMs, offering comprehensive insights and guidance on mitigating the most prevalent errors in these models.
In general, our contributions are as follows:
• A large-scale error set, COTERRORSET, is constructed for scalable analysis and learning from the LLMs’ mistakes.
• We novelly designed two paradigms for LLMs to utilize and learn from their previous mistakes at both fine-tuning and inference stages.
• With extensive experiments, we validate the effectiveness of our proposed methods and provide further hints based on analysis of LLMs’ error types.

heading: 2 Related Work
text: Human-like Reasoning with LLMs. CoT (Wei et al., 2022) demonstrate the great potential of equipping LLMs with human-like reasoning capability. Following them, various logical and structural reasoning strategies (Wang et al., 2022; Zhou et al., 2022; Creswell and Shanahan, 2022; Besta et al., 2023; Li et al., 2023b; Lightman et al., 2023) are proposed to align LLMs’ thinking processes with humans. These enhanced reasoning approaches have been adopted in different tasks and areas, including commonsense reasoning (Geva et al., 2021; Ahn et al., 2022), logical reasoning (Pan et al., 2023; Lei et al., 2023) and mathematical reasoning (Cobbe et al., 2021; Hendrycks et al., 2021) and achieved promising performance. In this work, we aim to investigate whether LLMs can benefit from rethinking and learning from previous mistakes, which is one of the most important learning patterns of humans.
Refined Reasoning Errors. Several studies focus on adjusting their reasoning pathways to arrive at better solutions. Huang et al. (2022) introduce self-improve that employs CoT plus selfconsistency to obtain high-confidence solutions on a large set of unlabeled questions. The selfgenerated content is then used for fine-tuning in subsequent iterations, thereby further augmenting its reasoning capabilities. Madaan et al. (2023) propose a self-refine technique that encourages LLMs to autonomously correct their outputs without the need for external data or feedback. However, it
has been argued by some researchers that LLMs face challenges in self-correcting their responses in the absence of external feedback, and under certain conditions, such attempts might even deteriorate their performance (Huang et al., 2023). Based on that, An et al. (2023) suggest fine-tuning LLMs using pairs consisting of errors and their respective corrections generated by GPT-4 as a supervisory mechanism. Nevertheless, our work is pioneering in highlighting the impact of exposing mistake examples on in-context learning. Furthermore, our experiments reveal that in the process of model tuning, learning from mistakes can inherently enhance itself by merely being exposed to correct examples and errors, without depending on explicit corrections from teacher models.

heading: 3 A Novel Dataset: COTERRORSET

heading: 3.1 Dataset Construction
text: In order to investigate whether incorrect rationales can also contribute to LLMs’ reasoning performance, we introduce COTERRORSET, a novel benchmark based on the source of COTCOLLECTION (Kim et al., 2023), built upon various domains, including multiple-choice QA, extractive QA, closed-book QA, formal logic, natural language inference, and arithmetic reasoning. Our dataset’s question and reference are obtained from the following datasets: QASC (Khot et al., 2020), AQuA (Ling et al., 2017), GSM8K (Cobbe et al., 2021), QED (Lamm et al., 2021), StrategyQA (Geva et al., 2021), SenseMaking (Wang et al., 2019), CREAK (Onoe et al., 2021), eSNLI (Camburu et al., 2018) and ECQA (Aggarwal et al., 2021). Each task within this collection is systematically organized to include a question and a correct reference, followed by an incorrect response and the demonstrations why making such mistakes. The errors and demonstrations are both generated from PaLM2.
COTERRORSET diverges from traditional CoT datasets by employing PaLM2’s mistakes and the reasons behind them. We utilized PaLM2 to generate rationales for each question in the dataset, focusing specifically on collecting incorrect rationales. Recent research has demonstrated LLMs’ capability to provide high-quality data (Li et al., 2024a; Tong et al., 2024; Li et al., 2024b) and feedback (Pan et al., 2024; Tan et al., 2024). Following this idea, we provide PaLM2 with both correct references and its incorrect answers to demonstrate
and reflect why it makes such mistakes. The steps of the construction process are shown in Figure 2. This systematic collection of incorrect rationales can make COTERRORSET a promising benchmark in providing future improvements from a different perspective. One example is shown in Table 1.

heading: 3.2 Error Analysis with COTERRORSET
text: After collecting the COTERRORSET dataset, we observe that the error types in it are very intricate and diverse. The intricacy poses obstacles to subse-
quent enhancement efforts. In order to tackle this issue and gain a more overarching understanding of LLMs’ error types, we utilize an LLM-based unsupervised clustering approach shown in Figure 3 to match diverse error types into more general categories.
To be specific, we begin by extracting the specific error keywords from each error cause. Subsequently, we input all the extracted keywords into the LLMs and prompt them to generate more general categories that encompass the entire spectrum of error names. Following this automated clustering process, we manually review each cluster, making necessary adjustments to refine the matching results. Finally, we distill the diverse error types into several abstract categories, such as calculation error, numeric error, and logical error in domains of arithmetic reasoning and logical error, commonsense error, linguistic error, and context error in domains of commonsense reasoning. A detailed definition of each error category is shown in Appendix C. We put results and analysis in Section 8.

heading: 4 Our Methodology: Self-rethinking
text: Self-rethinking offers an innovative approach to encourage LLMs to consider if they are repeating past errors. This method starts with an initial CoT reasoning. Following this, the model uses the provided reasoning outputs and a random selection of examples from COTERRORSET. This step is designed to assess if the model’s most recent response includes similar inaccuracies. If errors are detected, it will formulate a new rationale and undergo the evaluation process again. This cycle continues until the model deems its latest answer to be correct or it reaches a set limit of evaluation rounds. The main goal is to empower the LLM to learn from its errors introspectively and minimize the recurrence of such mistakes. One example is shown in Table 2.
The core of self-rethinking lies in the backwardchecking stage. In this phase, the LLM reviews its reasoning chain, but with a specific focus on the error types it previously identified. This explicit demonstration of errors, coupled with the question,
golden reference, and incorrect rationales, is instrumental in enabling the LLM to recognize specific types of mistakes it tends to make. This targeted review helps the LLM to not just correct the random errors but to consciously avoid repeating the same types of mistakes it has made in the past. The process includes a loop for error correction and confirmation. If the LLM finds that it has repeated any of the previously identified mistakes, it revisits the reasoning process to correct them. Otherwise, the last response is adopted as the final result.
Moreover, the iterative checking process should have a crucial repeating boundary, denoted as k iterations. If the LLM’s error-checking and correction cycles surpass this predefined threshold and errors still persist, the process concludes under the assumption that the issue at hand or the error detection might exceed the LLM’s current capabilities. This constraint prevents the LLM from being caught in an endless loop of self-rethinking, ensuring the efficiency and practicality of the reasoning process.

heading: 5 Our Methodology: Mistake Tuning
text: In order to fully investigate the other potential utilization of our principles, we introduce mistake tuning, which demonstrates our motivation is a broad and pioneering framework not only in the field of in-context learning. This approach is designed to finetune LLMs on the combinations of both correct rationales and incorrect mistakes. By simply appending prefixes [CORRECT RATIONALE] and [INCORRECT RATIONALE] before corresponding rationales, mistake tuning can further improve LLMs’ abilities to distinguish between correct and incorrect rationale.
Mistake tuning is built upon the foundational motivations and conclusions of self-rethinking, where LLMs can learn from the implicit reasons and types of mistakes they made in CoT reasoning. This process can be formulated as:
p = [Q⊕ S ⊕R], (1)
L = − |p|∑
t=1
logP (pt|p<t), (2)
Where Q, S and R represent the given question, special prefix and corresponding rationale respectively. ⊕ represents the operation of concatenation.
Mistake tuning presents a cost-effective, straightforward, and efficient alternative. Previous work has proven pretraining with some controlled signals based on human feedback can result in LLMs’ better ability to generate more satisfactory contents (Korbak et al., 2023; Keskar et al., 2019). Hence, incorporating fixed prefixes in finetuning LLMs in the field of reasoning can also help models differentiate information from golden references and mistakes. Our results also demonstrate its effectiveness for promoting LLMs’ reasoning abilities without additional costs similar to annotating golden reasoning references.

heading: 6 Experiments
text: In this section, we conducted a series of experiments to compare the proposed self-rethinking methods with the existing approach on both arithmetic and commonsense reasoning benchmarks.

heading: 6.1 Experiment Setup
text: We conduct comparisons between self-rethinking and several other baselines on multiple benchmarks.
Baselines: We select the following reasoning baselines to evaluate our framework, selfrethinking’s performance.
• Standard prompting (Brown et al., 2020): the basic reasoning promptings with prefixes as question and answer.
• Chain-of-Thought (CoT) (Madaan et al., 2023): a technique that enhances large language models’ ability to perform complex and multi-step reasoning by guiding them through
a problem-solving process step by step, significantly improving their performance on tasks that require deeper cognitive processing.
• Self-refine (Madaan et al., 2023): an approach that enables LLMs to iteratively improve their initial outputs by providing feedback to themselves and refining their responses.
• Self-consistency (Wang et al., 2022): a decoding strategy that enhances CoT prompting in LLMs by sampling multiple reasoning paths and selecting the most consistent answer.
Benchmarks: We consider the following existing math problems benchmarks designed with human rationale reference.
• GSM8K benchmark of math word problems (Cobbe et al., 2021).
• AQuA dataset of algebraic math problems (Ling et al., 2017).
• MathQA benchmark of multiple-choice math problems (Amini et al., 2019).
• Openbook benchmark modeled after open book exams for assessing human understanding of a subject (Mihaylov et al., 2018).
• LogiQA dataset sourced from expert-written questions for testing human logical reasoning (Liu et al., 2020).
• Critical Reasoning in MARB benchmark of several graduate admission tests, highlighting the reasoning to assumptions, conclusions and paradoxes in arguments (Tong et al., 2023).
Models: In order to evaluate self-rethinking’s effects, we choose PaLM2 (Anil et al., 2023)
and GPT4 (OpenAI, 2023) as the baseline model. PaLM2 is a dense left-to-right, decoder-only language model. It is pre-trained on a high-quality corpus of 780 billion tokens with filtered webpages, books, Wikipedia, news articles, source code, and social media conversations. GPT4 is a large-scale multi-modal state-of-the-art model that exhibits human-level performance on various tasks. We use PaLM2’s TEXT-BISON-001 and GPT4’s GPT-4 models provided in their APIs.
For mistake tuning, we choose two differentsized Flan T5 (Chung et al., 2022), which are specifically designed for instruction tuning strategies. This model excels in understanding and generating human-like text, demonstrating remarkable performance across a wide range of natural language processing tasks.
Training Details: All of the following experiments were designed with a common setting, employing a random seed of 42, learning rate=1e-4. Considering the vast number of data in AQuA, we only randomly select 10,000 of them to represent the differences in tuning on two different domains.

heading: 6.2 Self-rethinking Results
text: Table 3 presents PaLM2’s evaluation results on chosen benchmarks. In this experiment, we set
our method, self-rethinking’s k equal to 1 to trade between the accuracy and computing resources. In order to align the commuting budget with our methods, we set the times of inference in selfconsistency to 3. Our approach involves an initial zero-shot CoT inference, then rethinking whether this rationale has made similar errors. This leads to the final answer if no errors are found. If inaccuracies are detected, it combines a demonstration and the previously suspected erroneous answer for a third inference to arrive at the final answer. Hence, the overall inference times in our methods are between 2 and 3 times per question, which is still lower than self-consistency here.
With the considered computational settings, the self-rethinking method shows superior performance with significant improvements, especially in GSM8K, AQuA, MathQA, and LogiQA, clearly outperforming self-consistency under a similar computing cost. However, while our method surpasses CoT in performance on the MathQA dataset, it falls short of achieving self-refine results. It’s important to note that this dataset is specifically tailored towards operation-based arithmetic problems rather than general questions, aiming to gauge the models’ proficiency in tackling complex issues (Amini et al., 2019). This suggests that the nature of the MathQA dataset may inherently be more suitable for self-refine. In contrast to our approach, which aims to amend responses by identifying and addressing typical errors. Table 5 compares GPT4’s performance of CoT and self-rethinking. The results demonstrate a notable improvement when using our self-rethinking method over CoT. These findings suggest that self-rethinking is a more effective approach for enhancing GPT-4’s performance.
Table 4 presents the 8-shot examples of CoT and self-rethinking, using the PaLM2 model across four different tasks: GSM8K, AQuA, MathQA, and LogiQA. A key part of the process involved collecting PaLM2’s incorrect rationales for these examples, which were then used as learning demon-
strations to rethink. The results show a clear advantage of the self-rethinking method over the standard 8-shot CoT approach. These results highlight the efficacy of the self-rethinking method in improving accuracy in few-shot learning scenarios for complex problem-solving tasks.
Notably, self-refine shares our basic motivations about self-refining or self-correcting their answers but without utilizing any mistake samples. The result shows that our self-rethinking outperformed self-refine by a considerable margin across most of the datasets. This indicates the importance of our proposal for utilizing previous mistake examples. While self-refine demonstrates improvements in three arithmetic reasoning datasets, it concurrently exhibits substantial performance drops in commonsense reasoning datasets. By contrast, our selfrethinking consistently outperforms the standard method in various domains. This further implies the introduction of previous mistakes can stabilize the refinement and rethinking process.
In conclusion, our self-rethinking method achieved remarkable accuracy improvements in most tests, particularly in scenarios that demand high logical rigor and offer the opportunity to learn from errors by identifying fixed logical patterns, especially in arithmetic reasoning tasks. It indicates self-rethinking effectiveness in tasks requiring strong logic and prone to minor errors. Additionally, the self-rethinking method proves particularly beneficial in assisting LLMs in identifying and rectifying low-level mistakes or misunderstandings that are within the model’s capabilities but have been previously overlooked. This capability indicates that self-rethinking can serve as a valuable tool in refining the accuracy and reliability of responses in LLMs, especially in complex problemsolving contexts.

heading: 6.3 Mistake Tuning Results
text: Table 6 showcases the performance of the Flan-T5 models in the context of mistake tuning, highlight-
ing the impact of combining correct and incorrect rationales. The data presented in Table 6 reveals significant insights into the performance of FlanT5 models under mistake tuning, which involves integrating both correct and incorrect rationales. This approach is evident across different model scales, whether it’s the smaller 780M version or the larger 3B variant. Notably, in the MathQA domain, Flan-T5-large(780M) tuned by our methods demonstrates superior performance compared to PaLM2, achieving an accuracy of 48.95% versus 41.37%. This phenomenon suggests that LLMs can benefit from engaging with incorrect reasoning, thereby enhancing their problem-solving and reasoning capabilities. It extends beyond merely bolstering the model’s grasp of correct CoT, to also encompassing the ability to identify and learn from incorrect rationales.
Furthermore, the expense of obtaining ground truth or hand-crafted references is significantly higher compared to generating and collecting incorrect rationales. This cost disparity underscores the practical value of our approach, offering a more cost-effective solution without compromising the quality of training data for machine learning models. All mentioned provides a direction for further work of reasoning, which involves not only enhancing the model’s understanding and learning of correct CoT but also the ability to identify and learn from incorrect rationales.

heading: 7 Further Studies

heading: 7.1 Hyperparameter Analysis of Rethinking Iteration Times
text: In this section, we conduct experiments to assess the impact of different rethinking iterations, de-
noted as k, on the performance of our framework. We evaluate it on two mainstream benchmarks in the field of mathematics and commonsense reasoning, GSM8K and LogiQA. Figure 4 represents the detailed trend under varying re-thinking times. Notably, as k increases from 1 to 24, GSM8K represents a growth of 8.11% and 12.37% in LogiQA. It is evident as k increases, both LLMs’ arithmetic and commonsense reasoning accuracy exhibit an upward trend. This trend suggests a positive correlation between the number of rethinking iterations and the overall reasoning abilities. These observations indicate self-thinking’s potential benefits with more inference time.

heading: 7.2 Ablation Study on Rethinking Process
text: In this ablation study, we examined the impact of various component combinations in promptings to guide LLMs to self-rethinking . Table 7 shows the performance of different components. The results indicate that the inclusion or exclusion of different components has varying effects on PaLM2’s accuracy in domains of GSM8K and LogiQA. However, the overall performance across various components is relatively similar. It performs similarly well regardless of the specific combination of components, indicating good generalizability of the method. This study suggests our method’s flexibility and stability in future usage.

heading: 8 Unveiling LLM’s Reasoning Errors
text: In this section, we delve into the detailed types and underlying reasons that lead to mistakes in LLMs’s inference process. We sample mistake examples from GSM8K and LogiQA to conduct an in-depth analysis of both arithmetic and commonsense reasoning. We put some examples in Appendix B.
For commonsense reasoning, we find errors like the misinterpretation of facts or concepts usually arise due to the model’s limitations in under-
standing and applying context accurately. This reveals current LLMs may still fall short of consistently recalling precise factual knowledge within a given context. Consequently, this underscores the imperative to advance toward the development of Retrieval-Augmented Generation(RAG) systems (Guu et al., 2020; Mallen et al., 2022), as they hold the promise of yielding more faithful and contextually aligned results. Additionally, errors stemming from logical fallacies or incorrect inferences reveal LLMs’ reliance on pattern recognition over logical reasoning, sometimes leading them to make logically inconsistent or unsupported connections by the given facts.
As shown in Figure 5, the most errors made by LLMs in arithmetic reasoning are about calculation. This can be attributed to the different nature of LLMs compared to other tools like calculators. To address this issue, Chen et al. (2022)’s suggestion using Program-of-Thought (PoT) is a promising approach to instruct LLMs to generate a segment of code to solve the given problem, resulting in more accurate calculation results. Furthermore, it’s important to note that logical error is also a type of error that LLMs always suffer from. Compared with calculation errors and numeric errors, the causes of logical errors are more complicated and nuanced. For instance, errors like misinterpreting given data or misapplying arithmetic operations reveal a lack of depth in understanding mathematical relationships. This can result from the model’s limitations in comprehending the nuances of mathematical concepts or its inability to correctly infer the needed function from the context of the question. In the future, more fine-grained analysis and methods are needed to address such complex logical errors in arithmetic reasoning.

heading: 9 Conclusions and Future Work
text: In this work, we explore whether LLMs can learn from their mistakes. In order to investigate LLMs’ abilities to differentiate and learn from mistakes, we introduce COTERRORSET, a novel benchmark collecting both correct and incorrect CoT rationales across various domains and designed with demonstrations for making errors. We propose two possible solutions to expose the effects of mistakes from different perspectives: self-rethinking and mistake tuning. Both of them have achieved consistent and significant improvements, which demonstrates the potential benefits of learning from reasoning errors. In the last, we conduct a comprehensive and detailed analysis of LLMs’ common mistakes in both arithmetic and commonsense reasoning. The findings will provide a clear direction for future improvements.
For future work, we envision proposing corresponding algorithms or loss functions to learn implicit information from mistakes. The primary intent of this work is to provide a new paradigm so there are still a lot of improvements that can be down following this work. For example, incorporating contrastive learning to differentiate correct references and errors is intuitive to make more improvements. Also, some memorization and retrieval-augmented skills can help models benefit from mistakes similar to each question.
Limitations
In addition to the noted challenge of fine-tuning commercial LLMs, we recognize several other specific limitations in our study that require attention. Primarily, our self-rethinking methodology may not be entirely suitable for tasks where a distinct, objective label is not readily available, such as in machine translation or dialogue generation. These areas pose a unique challenge as the correctness of outputs can often be subjective or context-dependent, making it difficult to apply our approach effectively. Moreover, our utilization of the COTERRORSET collection for mistake tuning necessitates a ground truth label for each sample, posing a potential impediment to the applicability of our method in low-resource scenarios. In the future, we will continually improve our method and bring the concept of learning from mistakes to wider scenarios and applications. Thanks again for your thoughtful insights and informative comments.

heading: Acknowledgements
text: Our work is sponsored in part by NSF CAREER Award 2239440, NSF Proto-OKN Award 2333790, as well as generous gifts from Google, Adobe, and Teradata. Any opinions, findings, and conclusions or recommendations expressed herein are those of the authors and should not be interpreted as necessarily representing the views, either expressed or implied, of the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for government purposes not withstanding any copyright annotation hereon.

heading: A Algorithm for self-rethinking
text: Algorithm 1 self-rethinking Mistakes = {...} Correct & Incorrect Examples = {...} ErrorCounter← 0 Prompt: Why you made the mistakes? Mistakes← Error Type, Demonstrations, Examples. Stage1 Prompt: Let’s think step by step. Stage2 Prompt: Do you make the same mistakes in Mistakes? while ErrorCounter < k do
if Yes then go to Step2 ErrorCounter← ErrorCounter + 1 else if No then get the answer break
end if end while if ErrorCounter == k then
Assume: Problem or error detection exceeds the model’s capabilities. end if Prompt: So the final answer is:
B Reasoning Mistake Examples
Error name: Logical Fallacy or Incorrect Inference Error type: Logical
– Example: "When standing miles away from Mount Rushmore" – Correct Rationale: Objects appear smaller when viewed from a greater distance. – Incorrect Rationale: "The mountains do not look smaller when standing miles away from Mount Rushmore. They look
larger." (Logical fallacy) – Demonstration: 1. The correct rationale is that objects appear smaller when viewed from a greater distance, whereas the
incorrect rationale states the opposite. 2. This is a logical fallacy as it contradicts a basic principle of perception. 3. The type name of the incorrect rationale is logical.
Error name: Incorrect Assumptions or Generalizations Error type: Logical

heading: C More Details about LLM-based Clustering Approach

heading: Future Work
text: The findings will provide a clear direction for future improvements. For future work, we envision proposing corresponding algorithms or loss functions to learn implicit information from mistakes. The primary intent of this work is to provide a new paradigm so there are still a lot of improvements that can be down following this work. For example, incorporating contrastive learning to differentiate correct references and errors is intuitive to make more improvements. Also, some memorization and retrieval-augmented skills can help models benefit from mistakes similar to each question. Limitations
In addition to the noted challenge of fine-tuning commercial LLMs, we recognize several other specific limitations in our study that require attention. Primarily, our self-rethinking methodology may not be entirely suitable for tasks where a distinct, objective label is not readily available, such as in machine translation or dialogue generation. These areas pose a unique challenge as the correctness of outputs can often be subjective or context-dependent, making it difficult to apply our approach effectively. Moreover, our utilization of the COTERRORSET collection for mistake tuning necessitates a ground truth label for each sample, posing a potential impediment to the applicability of our method in low-resource scenarios. In the future, we will continually improve our method and bring the concept of learning from mistakes to wider scenarios and applications. Thanks again for your thoughtful insights and informative comments