Can LLM s Learn From Mistakes? An Empirical Study on Reasoning Tasks (2024)

Abstract: Towards enhancing the chain-of-thought (CoT) reasoning of large language models (LLMs), much existing work has revealed the effectiveness of straightforward learning on annotated/generated CoT paths. However, there is less evidence yet that reasoning capabilities can be enhanced through a reverse learning process, i.e., learning from potential mistakes in reasoning. To investigate whether LLMs can learn from mistakes, we construct mistake-correction datasets, using GPT-4 to identify and correct the mistakes in inaccurate CoTs. With these mistake-correction datasets, we fine-tune open-source LLMs and arrive at the following conclusions. (1) LLMs can indeed learn from mistakes to enhance their CoT reasoning performances. (2) Compared to CoT data, the mistake-correction data provides additional knowledge on the explanations and reasons for the potential mistakes in CoTs, which consistently contributes to the effectiveness of learning from mistakes. (3) Evolution techniques, especially the correction-centric evolution we introduced, can further enhance the effectiveness of learning from mistakes.

## introduction mistakes are the portals of discovery. ## methodology our exploration consists of three primary stages: constructing the mistake-correction dataset, expanding the dataset with correction-centric evolution, and fine-tuning llms. ## results and analysis we mainly focus on three main research questions in this section. more results and analysis are contained in appendix d. ## related work llms with cot reasoning. @xcite uncovered the emergence of cot reasoning capability for extremely large language models, and this reasoning capability was then examined in various reasoning-related domains including logical reasoning @xcite @xcite , commonsense reasoning @xcite @xcite , and math reasoning @xcite @xcite @xcite . the impressive performance of llms in these domains has spurred the research community to further investigate methods for effectively harnessing and enhancing cot reasoning for llms @xcite @xcite @xcite . ## conclusion this work provides an empirical study on exploring whether llms can learn from mistakes to improve their cot reasoning performances. our experiments on mistake-correction data reveal that llms can indeed learn from mistakes, especially benefit from the explanations and reasons to mistakes.