LEMMA : Learning from Errors for M athe M atical Advancement in LLM s (2025)

Abstract: Large language models (LLMs) have demonstrated remarkable reasoning capability in solving mathematical problems. However, existing approaches primarily focus on improving the quality of correct training data, e.g., distilling high-quality correct solutions from advanced models, neglecting the value contained in error data, potentially hindering the model’s reflective ability. Though some studies attempted to leverage error data, they often involve complex mechanisms, such as Monte Carlo Tree Search (MCTS) to explore error nodes.In this work, we propose to enhance LLM’s reasoning ability by Learning from Errors for MatheMatical Advancement (LEMMA). LEMMA constructs data consists of an incorrect solution with an erroneous step and a reflection connection to a correct solution for fine-tuning. Specifically, we systematically analyze the model-generated error types and introduce an _error-type grounded mistake augmentation_ method to collect diverse and representative errors. Correct solutions are either from fixing the errors or generating a fresh start. By fine-tuning on the constructed dataset, the model is able to _self-correct errors autonomously_ within the generation process _without relying on external critique models_. Experimental results demonstrate that LEMMA achieves significant performance improvements over other strong models with less than 90k data.

## introduction recently, large language models (llms) have significantly improved their ability to solve mathematical problems through supervised fine-tuning (sft). a common strategy involves refining the quality of chain-of-thought (cot) reasoning data, such as distilling high-quality solutions from advanced models @xcite . while these methods enhance the model's capacity to generate step-by-step solutions, they predominantly focus on optimizing correct reasoning trajectories while overlooking the potential of error data. this omission limits the model's ability to learn from mistakes, thereby constraining its reflective reasoning capability. reflection-the process of identifying, analyzing, and correcting errors-is a critical component of human problemsolving @xcite . given the failure to integrate this ability into llms, models remain vulnerable to propagating errors during inference without autonomous correction mechanisms. to address this gap, recent studies have begun exploring methods to cultivate reflection in llms by leveraging error data. for instance, some works @xcite @xcite employ external critical models to critique intermediate reasoning steps or use monte carlo tree search (mcts) to navigate complex reasoning paths and prune error branches. others @xcite @xcite propose self-correction frameworks that construct incorrect-correct data for fine-tuning, enabling the model to iteratively revise its outputs. however, these approaches suffer from significant limitations. mcts-based methods introduce substantial computational overhead and complexity, while self-correction methods rely on naive and inefficient techniques to collect incorrect and correct reasoning trajectories. in this work, we propose lemma to learn from errors for mathematical advancement, a novel method to systematically enhance llms' reflective reasoning by constructing and learning from error-corrective trajectories. our approach begins with a fine-grained categorization of error types in model-generated solutions, ranging from "question misinterpretation" to "calculation error". building on this taxonomy, we design an error-type grounded error augmentation strategy that diversifies error data by (1) harvesting mistakes from the target model's own reasoning traces and (2) guiding advanced models to generate representative errors according to the analyzed error type distributions. for each erroneous solution, we then construct paired reflection data through two complementary mechanisms: fix & continue trajectories, where the mistake is directly corrected within its original context, and fresh & restart trajectories, where a new correct solution is generated from scratch. these trajectories are seamlessly connected via model-aware reflection links-annotations that explain the error's origin and justify the correction-resulting in coherent training examples. experiments across mathematical reasoning benchmarks (e.g., gsm8k, math) demonstrate lemma's effectiveness. models fine-tuned with lemma outperform both standard sft baselines and prior error-aware methods (up to 13.3% average accuracy improvement on llama3-8b). lemma-trained models also achieve strong generalization ability through evaluation on out-ofdistribution (ood) benchmarks. further analysis reveals lemma can consistently reduce the occurrence of representative error types. in contrast, while fine-tuning on the original training set (sft) improves overall accuracy, it leads to an increase in certain error types. these results validate that structured learning from errors, guided by systematic analysis, is a powerful yet underutilized lever for advancing mathematical reasoning in llms. recently, some works @xcite @xcite @xcite @xcite have begun using monte carlo tree search (mcts), process reward models (prm) @xcite or critique models to further enhance selfimprovement. however, these methods only utilize the correct solutions, neglecting the generated errors. since models only learn from correct solutions, they struggle to reflect on and selfcorrect errors they made, leading to error accumulation @xcite @xcite . @xcite introduces a difficulty-aware answer augmentation strategy, where more solutions are generated for harder problems. to further boost model performance, several works @xcite @xcite @xcite @xcite @xcite @xcite synthesize more training data by creating new mathematical problems and solutions. for example, llm2llm @xcite iteratively synthesizes more data based on the data points that the student model fails to answer. metamath @xcite combines answer augmentation and question augmentation, as well as two backward reasoning methods @xcite , to further augment training data. although these works have acknowledged the value of erroneous data, their focus is on sample-level augmentation through the synthesis of additional data points. in contrast, our method implements sequence-level augmentation by enriching each data point with common errors and subsequent reflections, thereby fostering a more profound capacity for self-reflection and correction. furthermore, our method is orthogonal to these question augmentation methods and can be directly integrated with them. ## methodology to better leverage generated reasoning errors for enhancing the self-reflection and correction capabilities of llms, we begin by conducting a systematic analysis of common error types in widely-used models. building on this analysis, we introduce lemma, a novel approach that strategically constructs self-correction data to improve the mathematical reasoning abilities of llms. fig. 2 provides an overview of the lemma framework. ## experiments we evaluate our method through comprehensive experiments from three key aspects: (1) in-distribution mathematical tasks, (2) out-ofdistribution mathematical tasks, and (3) reflective mathematical reasoning tasks. ## conclusion in this work, we introduce lemma, a novel framework designed to enhance the mathematical reasoning capabilities of llms by systematically learning from errors. based on a comprehensive analysis of error types, lemma employs an errortype grounded mistake augmentation strategy and constructs diverse revision pathways using both the fix & continue and fresh & restart correction strategies. this framework allows models to autonomously detect and correct errors during the generation process, thereby improving their mathematical reasoning abilities. extensive experiments demonstrate that lemma significantly outperforms strong baselines.