Exposing the Achilles' Heel: Evaluating LLMs Ability to Handle Mistakes in Mathematical Reasoning

## Abstract

Abstract Large Language Models (LLMs) have been applied to Math Word Problems (MWPs) with transformative impacts, revolutionizing how these complex problems are approached and solved in various domains including educational settings. However, the evaluation of these models often prioritizes final accuracy, overlooking the crucial aspect of reasoning capabilities. This work addresses this gap by focusing on the ability of LLMs to detect and correct reasoning mistakes. We introduce a novel dataset MWP-MISTAKE , incorporating MWPs with both correct and incorrect reasoning steps generated through rule-based methods and smaller language models. Our comprehensive benchmarking reveals significant insights into the strengths and weaknesses of state-of-the-art models, such as GPT-4o , GPT-4 , GPT-3.5Turbo , and others. We highlight GPT-4o ’s superior performance in mistake detection and rectification and the persistent challenges faced by smaller models. Additionally, we identify issues related to data contamination and memorization, impacting the reliability of LLMs in real-world applications. Our findings emphasize the importance of rigorous evaluation of reasoning processes and propose future directions to enhance the generalization and robustness of LLMs in mathematical problem-solving.

## 1 Introduction

Large Language Models (LLMs) have transformed artificial intelligence applications across diverse domains, including healthcare, agriculture, and education [, ]. Their remarkable capabilities in natural language understanding, question answering, and mathematical problem-have shown potential to revolutionize various human endeavors []. Recent advancements have fueled extensive research into applying LLMs to interpret and solve a wide array of mathematical tasks, from basic arithmetic to complex algebraic equations and calculus problems [, ].

Math Word Problems (MWPs) convey mathematical concepts and calculations through written descriptions, typically involving narrative scenarios []. Solvers must extract relevant mathematical information from these narratives and apply appropriate principles to arrive at solutions. Studies [, , ] have demonstrated that LLMs are proficient at understanding the contextual subtleties of MWPs, translating textual descriptions into mathematical expressions, and delivering precise solutions. Central to this process is mathematical reasoning, which enables models to adeptly manage complex, multi-step problems, draw logical inferences, and provide accurate solutions.

Despite achieving remarkable accuracy rates exceeding 90% on datasets like GSM-8K (Grade School Math dataset with linguistically diverse word problems) [], foundational LLMs such as Claude-3-Opus [], Gemini Ultra [], and OpenAI GPT-4 [] reveal a significant gap in our understanding of their capabilities in mathematical reasoning []. Current research predominantly focuses on evaluating the final accuracy of MWPs [, ], neglecting the intricate reasoning processes necessary to derive solutions. We argue that the reasoning steps play a pivotal role, and it is imperative to assess them to comprehensively analyze the foundational capabilities of these models. This necessity is further underscored by the increasing utilization of LLMs in domains such as education [], where they serve as personalized tutors for students, aiding in teaching concepts and solving mathematical problems. Simply deriving the final answer is insufficient; the ability to guide students through correct steps, identify errors in their reasoning, and provide corrective guidance is paramount for such applications.

Figure: Figure 1: Model is prompted with a question along with incorrect reasoning steps to detect any mistake and correct the reasoning step to get to the correct final answer. GPT-4o generates the correct output, while GPT-3.5Turbo fails to identify any mistake in the reasoning step. (Task - T1)
Refer to caption: x1.png

This paper aims to bridge this gap by providing a comprehensive benchmark and evaluation of LLMs’ performance on math word problems, including their capabilities in mistake detection and correction within the reasoning steps (Figure 1). Analyzing LLMs’ ability to detect and rectify errors along the reasoning steps yields valuable insights into their overall problem-solving capabilities. Our objectives are threefold: firstly, to comprehensively evaluate LLMs’ capabilities in mathematical reasoning, with a particular emphasis on mistake detection and correction; secondly, to identify the specific strengths and weaknesses of these models in handling various types of mathematical challenges; and thirdly, to propose potential directions for enhancing LLM capabilities in this domain.

To achieve this comprehensive evaluation, we have developed our own mistake dataset, designed to include errors in the reasoning steps. This dataset allows the assessment of models’ proficiency not only in providing correct solutions but also in detecting and correcting mistakes within the reasoning steps.
We evaluate eight different models including both large and smaller language models on our curated dataset MWP-MISTAKE.

Our analysis reveals several key insights into the performance of LLMs on MWPs. Firstly, detecting mistakes, even trivial ones remains a significant challenge for these models. Secondly, LLMs often derive correct answers despite this difficulty in mistake detection. This can be attributed to data memorization and potential contamination in training datasets, where models may have encountered similar/same problems before. However, the ability to recover from or correct errors in the reasoning process is generally poor across most models.
Our contributions to this paper are as follows:

- 1.
We collect and release to the research community MWP-MISTAKE, a dataset containing MWPs with both correct and incorrect reasoning obtained from state-of-the-art MWP datasets such as GSM-8K [], MATH [], MATHBENCH [], and JEEBENCH []. Incorrect reasoning is derived through meticulously crafted rules to alter the reasoning steps and using smaller models, leveraging their inherent limitations in solving MWPs.
- 2.
We provide benchmark results for our dataset to evaluate the reasoning capabilities of state-of-the-art LLMs such as GPT-4o [], GPT-4 [], GPT-3.5Turbo [], Claude [], as well as smaller language models like Llama [], Phi [], and Mixtral []. Our analysis demonstrates that most state-of-the-art LLMs, excluding GPT-4o, struggle with mistake detection and correction.
- 3.
Through meticulous evaluation and comparison of different LLMs, we offer a detailed analysis of their strengths and weaknesses in handling mathematical reasoning tasks.

## 2 MWP-Mistake Dataset

Most MWP datasets include a math problem and the final answer, with some optionally providing reasoning steps (i.e., steps to solve the math problem) (See Figure. 2). Our objective in this work is to evaluate the LLMs’ ability to detect and rectify errors to derive the correct final answer. However, no existing datasets include incorrect reasoning steps for MWPs. To address this, we curated our own dataset, MWP-MISTAKE, by leveraging state-of-the-art MWP datasets such as GSM-8K [], MATH [], MATHBENCH [], and JEEBENCH []. MATHBENCH and JEEBENCH are relatively newer datasets as compared to GSM-8K and MATH (Additional details in Appendix 8).

Figure: Figure 2: Examples of MWPs with correct reasoning, rule-based incorrect and smaller model based incorrect reasoning from MATH.
Refer to caption: x2.png

Each dataset contains an MWP question and a final solution. While GSM-8K and MATH have ground truth reasoning steps, MATHBENCH and JEEBENCH do not. For these datasets, we used GPT-4 to curate chain-of-thought reasoning steps. Thus, for all four datasets, we have an MWP question, a final answer, and associated correct reasoning steps. Also, note that in GSM-8K and MATH, the reasoning steps might include the final answer, however in our COT-generated steps, we ensure the answer is not present in the reasoning steps (Appendix 8 for additional details).

To create incorrect reasoning steps, we follow two approaches: (i) meticulously crafted rules, and (ii) using smaller models as bad reasoners, which we describe next.

### 2.1 Meticulously Crafted Rules to Programmatically Inject Errors

Given our focus on MWPs and based on extensive interactions with math teachers, the rules are derived from common mistakes observed in educational settings, ensuring the errors introduced are realistic and representative of actual student errors.

- 1.
Shuffle reasoning steps: The reasoning steps are shuffled to introduce ambiguity in the thought process. This tests whether the model can identify changes in reasoning order.
- 2.
Delete reasoning steps: One reasoning step is deleted in solutions that have two or more steps. This helps to identify if the model can spot omissions in the reasoning process.
- 3.
Shuffle numerical values: Numerical values are shuffled among themselves to verify if models can correctly understand the question and select appropriate numerical values from the question.
- 4.
Replace numerical values: Numerical values are replaced with random numbers ranging from 0 to 100. It identifies if the model can correctly pick the numerical values present in the question.
- 5.
Shuffle operations: We randomly swap operators with other operators to test the model’s ability to perform numerical operations.
- 6.
Insert random reasoning steps: A random reasoning step is added at a random position to test the model’s ability to identify incorrect reasoning.

**Table 1: MWP-MISTAKE Dataset details with the total number of questions and reasoning steps.**
| Dataset | Default reasoning | Smaller model reasoning |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
| # Questions<br>with correct<br>reason (GT) | # Questions<br>with incorrect<br>reason (Rules) | # Questions<br>with incorrect reasoning | Total |  |  |  |
| Llama-2-7b-chat | Mixtral-8x7B | Phi-3-mini |  |  |  |  |
| GSM-8K | 93 | 558 | 100 | 100 | 100 | 951 |
| MATH | 150 | 900 | 150 | 150 | 150 | 1500 |
| MATHBENCH | 100 | 600 | 100 | 100 | 100 | 1000 |
| JEEBENCH | 38 | 228 | 12 | 19 | 35 | 332 |

These rules mimic real-world student behavior by reflecting tendencies to get the order of steps wrong, skip steps, misinterpret numerical values, use incorrect numbers, apply the wrong mathematical operations, and add irrelevant steps in problem-solving. While rules #1 and #2 do not introduce explicit errors in reasoning, they are considered mistakes in our dataset to prompt the model to identify scenarios lacking clarity. Such scenarios, whether due to an incorrect thought process or missing steps, are common in real-life situations. Table 1 shows the number of questions selected from each of the four datasets to which these six rules are applied to curate incorrect reasoning. Thus, for every question selected, we created seven variations of reasoning steps(one correct + six incorrect).

### 2.2 Smaller Models as Bad Reasoners

Recently, numerous small language models (SLMs) have been developed that are smaller in size and trained on smaller amounts of data. These models are highly efficient, effective, and capable of solving numerous tasks, including MWPs. However, they still lack several capabilities, including advanced mathematical reasoning, resulting in poorer performance on MWPs.

To curate incorrect reasoning steps, we use SLMs to generate Chain-of-Thought (COT) reasoning and final answers for all dataset questions. We filter out questions with incorrect final answers (comparing with the ground truth final answer from the dataset), assuming incorrect answers stem from incorrect reasoning. Thus, the reasoning steps for all incorrect answers are used as incorrect reasoning steps.
We employ state-of-the-art SLMs, such as Llama-2-7b-chat, Phi-3-mini, and Mixtral-8x7B, to generate COT reasoning steps without a final answer (Appendix 9 for additional details). Table 1 shows dataset stats for each of the three models across all datasets.

Thus, our dataset includes questions with original correct reasoning steps, rule-based incorrect reasoning, and smaller model (SLM) generated incorrect reasoning. For detailed evaluation, we split this data into two parts: (1) Default: containing questions with correct reasoning from the dataset and rule-based incorrect reasoning, and (2) SLM reason: containing questions with incorrect steps generated by SLMs.
Table 1 provides the complete details of the curated MWP-MISTAKE dataset with the above two splits. We are releasing this dataset for further evaluation and benchmarking.

## 3 Experimental Setup

Task Details. Our aim is to assess the performance of LLMs on MWPs, focusing on their ability to detect and correct mistakes within the reasoning steps. We have two task variants to accomplish this:

Task-1 (T1): Here, given a question and its reasoning steps, we ask the model to identify if the steps are correct or incorrect. If incorrect, the model must rectify the mistake and calculate the final answer. The final answer or corrected reasoning step can be either correct or incorrect (Figure 1).
Task-2 (T2): In this scenario, the model only needs to identify whether the reasoning steps provided are correct or incorrect and provide the final answer. No correction of reasoning steps is required.

In essence, T1 evaluates the model’s ability to detect mistakes, rectify them, and derive the correct answer, while T2 focuses solely on detecting mistakes and solving MWP correctly. Both tasks operate under few-shot settings, with specific prompt details provided in Appendix 10.

Models. To evaluate LLMs’ mathematical reasoning capabilities, we utilize state-of-the-art LLMs and Small Language Models (SLMs).

LLMs: We utilize LLMs that have shown tremendous performance in MWPs such as GPT-4o, GPT-4, GPT-3.5Turbo, Claude-3-Opus. These models are accessed through their respective APIs.
SLMs. Additionally, we assess SLMs trained with high-quality data and reasoning capabilities and explored three popular SLMs with diverse capabilities: Phi-3-mini, Mixtral-8x7B, and Llama-2-7b-chat.
Appendix 12 provides the details of the models, including their last training date.

## 4 Results and Analysis

We rigorously evaluate various SOTA LLMs and SLMs on our MWP-MISTAKE dataset to analyze their mathematical reasoning capabilities, focusing on mistake detection and recovery.

**Table 2: Mistake Detection Performance (F1 score) on MWP-MISTAKE dataset for Task T1. (D-Default reasoning steps, SM-Smaller model reasoning steps) (Bold: Best, Underline:Second best)**
|  | GSM-8K | MATH | MATHBENCH | JEEBENCH | Average |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Model | D | SM | D | SM | D | SM | D | SM | D | SM | Overall |
| GPT-4o | 0.85 | 0.84 | 0.83 | 0.86 | 0.80 | 0.99 | 0.80 | 0.99 | 0.82 | 0.92 | 0.87 |
| GPT-4 | 0.72 | 0.68 | 0.78 | 0.80 | 0.51 | 0.90 | 0.74 | 0.87 | 0.69 | 0.81 | 0.75 |
| GPT-3.5Turbo | 0.80 | 0.69 | 0.80 | 0.54 | 0.50 | 0.34 | 0.54 | 0.46 | 0.66 | 0.51 | 0.58 |
| Llama-2-7b-chat | 0.07 | NA | 0.16 | NA | 0.08 | NA | 0.41 | NA | 0.18 | NA | 0.18 |
| Mixtral-8x7B | 0.73 | NA | 0.79 | NA | 0.62 | NA | 0.70 | NA | 0.71 | NA | 0.71 |
| Phi-3-mini | 0.70 | NA | 0.65 | NA | 0.54 | NA | 0.67 | NA | 0.64 | NA | 0.64 |
| Claude-3-Opus | 0.79 | 0.87 | 0.73 | 0.76 | 0.68 | 0.91 | 0.69 | 0.88 | 0.72 | 0.85 | 0.79 |

### 4.1 Question 1: Can LLMs Effectively Identify Mistakes in Reasoning Steps?

We first analyze the capability of various models to detect mistakes in MWP reasoning steps. Table 2 presents the mistake detection performance (F1 score) of all the models with Task T1 on our dataset, which includes reasoning steps derived from default and smaller models across four datasets.

- •
GPT-4o’s Dominance: GPT-4o demonstrates a substantial advantage, with a 10% improvement over GPT-4, a 25% improvement over GPT-3.5Turbo, and over 20% improvement over SLMs in detecting mistakes. It is uniquely capable of consistently identifying mistakes created using both rule-based methods and smaller models, underscoring its robust capabilities in mistake detection.
- •
GPT-3.5Turbo’s Performance: Interestingly, GPT-3.5Turbo outperforms GPT-4 in mistake detection specifically for the GSM-8K dataset. We hypothesize that this could be due to potential overfitting or data contamination in GPT-4’s training data. Despite this anomaly, GPT-4 maintains its position as the second-best model overall, following closely behind GPT-4o in terms of mistake detection abilities on other datasets.
- •
Performance of SLMs:
SLMs show significantly lower mistake detection abilities compared to GPT-4o and GPT-4. This stark contrast highlights the need to enhance reasoning capabilities in smaller models to match advanced LLMs.
- •
Performance on Newer Datasets: The performance of most models, including GPT-4 and GPT-3.5Turbo, drops drastically on newer datasets such as MATHBENCH and JEEBENCH. This decline indicates that the reasoning abilities of these models are not yet generalized to newer datasets and problems. Furthermore, JEEBENCH is more challenging dataset compared to others. GPT-4o, however, maintains a significant lead even on these newer datasets, reinforcing its superior capability in mistake detection across diverse and unseen problems.

Similar results are seen also for Task T2 as both T1 and T2 probes the model to detect mistakes, however, in the former case it goes further by asking the model to correct the reasoning step.

### 4.2 Can LLMs Accurately Derive Correct Answers Despite Mistakes?

We now assess the models’ ability to accurately derive the correct answer for the given question despite mistakes in the reasoning steps. Table 3 shows the performance of all the models in deriving correct answers (F1 score) on our dataset.

**Table 3: Performance in deriving correct answers (F1 score) on MWP-MISTAKE dataset for Task T1. (D-Default reasoning steps, SM-Smaller model reasoning steps) (Bold: Best, Underline:Second best)**
|  | GSM-8K | MATH | MATHBENCH | JEEBENCH | Average |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Model | D | SM | D | SM | D | SM | D | SM | D | SM | Overall |
| GPT-4o | 0.99 | 0.88 | 0.90 | 0.79 | 0.90 | 0.69 | 0.42 | 0.47 | 0.80 | 0.71 | 0.76 |
| GPT-4 | 0.97 | 0.79 | 0.80 | 0.69 | 0.88 | 0.46 | 0.35 | 0.27 | 0.75 | 0.55 | 0.65 |
| GPT-3.5Turbo | 0.89 | 0.48 | 0.69 | 0.35 | 0.75 | 0.20 | 0.26 | 0.14 | 0.65 | 0.29 | 0.47 |
| Llama-2-7b-chat | 0.80 | NA | 0.27 | NA | 0.40 | NA | 0.06 | NA | 0.38 | NA | 0.38 |
| Mixtral-8x7B | 0.87 | NA | 0.67 | NA | 0.70 | NA | 0.16 | NA | 0.60 | NA | 0.60 |
| Phi-3-mini | 0.88 | NA | 0.51 | NA | 0.63 | NA | 0.25 | NA | 0.57 | NA | 0.57 |
| Claude-3-Opus | 0.98 | 0.88 | 0.89 | 0.93 | 0.92 | 0.51 | 0.46 | 0.26 | 0.80 | 0.64 | 0.73 |

- 1.
GPT-4o’s Superior Accuracy: GPT-4o’s ability to derive correct answers is notably higher LMs. GPT-4o outperforms GPT-4 by 10%, GPT-3.5Turbo Turbo by 30%, and SLMs by a similar margin. We suspect the very high accuracy in GSM-8K may be due to data contamination, however on newer and complex datasets GPT-4o still outperforms other models. This indicates a strong capability to produce correct answers even when intermediate steps contain errors.
- 2.
GPT-4’s Performance: Interestingly, GPT-4’s ability to derive correct answers despite mistakes (F1 score of 0.97) is significantly better than GPT-3.5Turbo Turbo (F1 score of 0.89). Yet, GPT-4 performs poorly in mistake detection (0.72 vs. 0.80). This improvement in deriving correct answers may potentially be due to data contamination, resulting in the memorization of problems in the GSM-8K dataset during GPT-4’s training.
- 3.
SLMs’ Performance: SLMs, particularly Mixtral-8x7B, show performance very close to GPT-4 in deriving correct answers. This might again be due to its strong ability to produce correct answers in the presence of mistakes or data contamination during the training of SLMs, which allows them to recall correct answers despite reasoning mistakes.
- 4.
Performance on Newer and Complex Datasets: On newer and more complex datasets such as MATHBENCH and JEEBENCH, the performance significantly drops even for GPT-4o and more drastically for all other LLMs and SLMs. This highlights a critical limitation in the generalization of these models to newer and unseen problem sets.

Figure: Figure 3: Performance in deriving final answer between T1 and T2. A significant drop in performance when the model does not rectify the incorrect reasoning steps.
Refer to caption: x3.png

Figure 3 shows the performance difference between T1 and T2. For T2, we observe a significant performance drop in deriving correct answers despite mistakes. This is primarily because, in T1, we instruct the model to not only detect mistakes but also correct them before deriving the final answer, whereas in T2, the model is only asked to detect the mistake and then directly derive the final answer without correcting the reasoning (Appendix 11 for further details).

### 4.3 Exploring Data Contamination and Memorization Effects in Math Reasoning Tasks

In our analysis of LLMs’ mathematical reasoning performance, we’ve identified potential instances of data contamination and memorization, both of which can significantly impact the effectiveness of these models. Data contamination, characterized by the presence of test data from downstream tasks in LLMs’ training data, poses a major challenge in accurately assessing their real-world performance. Meanwhile, memorization occurs when models replicate solutions from training data without grasping the underlying principles, thereby hindering their ability to generalize to new problems.

The presence of data contamination is evident in instances of unexpectedly high performance on certain datasets. For example, GPT-3.5Turbo’s superior performance over GPT-4 on the GSM-8K dataset raises concerns about biases in GPT-4’s training data. Similarly, the comparable performance between smaller and larger models suggests the potential presence of memorization. These findings underscore the critical need for rigorous evaluation to mitigate the impacts of memorization, ensuring the reliability and effectiveness of LLMs in real-world applications.

Investigating data contamination and memorization poses challenges due to restricted pre-training data access and computational limitations. To tackle this, we employ an approach outlined in [], utilizing an LLM to replicate individual instances of the dataset. This involves guiding the LLM with instructions containing unique identifiers from the source dataset, like dataset name, partition (e.g., train, test, or validation), and a fragment of the reference instance. By instructing the LLM to complete these partial instances, we can evaluate contamination and memorization.

To detect contamination, a heuristic is applied comparing the average overlap score between generated completions and reference instances using ROUGE-L []. This comparison is made between guided instructions (including dataset and partition identifiers) and general instructions (lacking such identifiers). If the overlap score is significantly larger with guided instructions, it suggests contamination. This method relies on the premise that the only distinction between the two instructions is the inclusion of dataset and partition names in guided instructions, implying any improvement can be attributed to contamination (Appendix 15 for more details).
Figure 4 shows the difference between guided and general instructions ROUGE-L score across all models and datasets.

Figure: Figure 4: Difference between guided and general instructions rouge-L score across all models and datasets. A high positive difference indicates high contamination and a low positive or negative difference indicates, little to no contamination.
Refer to caption: x4.png

- •
GPT-4 Models: Across all datasets for default reasoning steps, the guided scores are higher than the general scores, indicating contamination for all LLMs such as GPT-4o, GPT-4, and GPT-3.5Turbo.
- •
Smaller Models’ Reasoning Mistakes: For reasoning mistakes from smaller models (SM), guided scores are closer to general scores, indicating little to no contamination across all models. This is intuitive as the reasoning steps are created anew by smaller models, and due to their probabilistic nature, variations are expected.
- •
Smaller Models like Llama-2-7b-chat and Phi-3-mini: These models show closer guided and general scores, indicating no contamination.
- •
Mixtral-8x7B Model: Mixtral-8x7B shows greater contamination as compared to the rest of SLMs, explaining high performance when deriving correct answers.
- •
GPT-4o: For datasets like GSM-8K and MATH, GPT-4o shows a higher guided score than the general scores, indicating contamination, which decreases in contamination as the dataset becomes newer and more complex.

### 4.4 Can LLMs Correctly Rectify Mistakes in Reasoning Steps?

In Task 1, LLMs detect and rectify mistakes in reasoning to find the correct final answer. To evaluate the model’s ability in this regard, we introduce the ’rectify metric’ to quantify instances where the model identifies a mistake, corrects it, and reaches the accurate final answer. Reasoning steps are considered correct only if they lead to the accurate final answer.
Table 4 shows the ability of different models to rectify reasoning steps and derive the correct final answer across various datasets.

**Table 4: Ability to Rectify mistakes and derive correct final answer on MWP-MISTAKE dataset for Task T1. (D-Default reasoning steps, SM-Smaller model reasoning steps) (Bold: Best, Underline:Second best)**
|  | GSM-8K | MATH | MATHBENCH | JEEBENCH | Average |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Model | D | SM | D | SM | D | SM | D | SM | D | SM | Overall |
| GPT-4o | 0.98 | 0.92 | 0.87 | 0.83 | 0.90 | 0.65 | 0.39 | 0.42 | 0.79 | 0.70 | 0.74 |
| GPT-4 | 0.96 | 0.89 | 0.72 | 0.68 | 0.83 | 0.46 | 0.23 | 0.24 | 0.69 | 0.57 | 0.63 |
| GPT-3.5Turbo | 0.81 | 0.58 | 0.54 | 0.40 | 0.62 | 0.35 | 0.05 | 0.05 | 0.51 | 0.35 | 0.43 |
| Llama-2-7b-chat | 0.73 | NA | 0.21 | NA | 0.11 | NA | 0.04 | NA | 0.27 | NA | 0.27 |
| Mixtral-8x7B | 0.77 | NA | 0.56 | NA | 0.57 | NA | 0.17 | NA | 0.52 | NA | 0.52 |
| Phi-3-mini | 0.79 | NA | 0.37 | NA | 0.41 | NA | 0.03 | NA | 0.40 | NA | 0.40 |
| Claude-3-Opus | 0.97 | 0.94 | 0.84 | 0.90 | 0.87 | 0.57 | 0.26 | 0.27 | 0.73 | 0.67 | 0.70 |

- •
GPT-4o’s Remarkable Capabilities: GPT-4o exhibits outstanding abilities in rectifying incorrect reasoning steps to derive the correct final answer. It outperforms GPT-4 by 11% and surpasses other models, including SLMs, by over 35%. Across all datasets, GPT-4o achieves high rectification scores, with an average of 85% across all datasets except JEEBENCH.
- •
Limitations of SLMs: SLMs perform notably worse than larger models like GPT-4o in rectifying errors, with an average score of only 40% across all datasets. This suggests significant challenges in effectively handling complex reasoning tasks.
- •
Performance on Newer and Complex Datasets: Despite its overall superiority, GPT-4o’s performance on newer and more complex datasets like MATHBENCH and JEEBENCH is lower, raising concerns about the generalization of its capabilities.
- •
Ability to Rectify Mistakes from Both Rules and Smaller Models: GPT-4o demonstrates tremendous capabilities in rectifying mistakes from both rule-based and smaller models. While potential contamination exists, GPT-4o ’s ability to correct mistakes in the reasoning steps generated by SLMs underscores its robustness in detecting and rectifying errors.

We now dig deeper into the rectification process. While Table 4 showed the models’ ability to detect and rectify mistakes, we compute the percentage of questions where the model rectified the reasoning but still resulted in incorrect answers. Across the MWP-MISTAKE dataset, after correcting the reasoning steps, GPT-4o failed to derive correct answers in 17% of the questions, whereas other models like GPT-4, GPT-4, Llama-2-7b-chat, Mixtral-8x7B, and Phi-3-mini resulted in 30%, 43.5%, 80.9%, 40.2%, and 55.6% incorrect answers, respectively. This showcases GPT-4o’s ability to detect mistakes and rectify them correctly, resulting in very few questions it could not answer correctly.

Furthermore, we noticed that the average word length of rectified reasoning for correct and incorrect answers for GPT-4o was significantly higher than GPT-4 and other models. This is mainly because GPT-4o generates its own reasoning steps to rectify the mistakes, unlike other models that perform poorly. This also adds challenges to evaluating mistake rectification as the new rectified reasoning is significantly different from ground truth reasoning steps. There could be multiple ways to solve the same problems, complicating the evaluation.

We also evaluated the rectified reasoning steps and compared them with ground truth reasoning steps to see the effectiveness and alignment of the rectification process across models. We computed BERTScore [] that computes a similarity score for each token in the candidate sentence with each token in the reference sentence, using BERT embeddings.
We found that BERTScore is similar across all models. This is because the BERTScore metric focuses on word-level matches and misses out on numerical and other logical aspects of reasoning which are crucial for correctness. We also evaluated the alignment with METEOR [] score (see Appendix 13 for BERTScore and METEOR Score), which similarly resulted in an inadequate analysis. Thus, it becomes evident that the current evaluation methodologies may not fully capture the nuanced capabilities of LLMs in rectifying mistakes within reasoning steps.

## 5 Key Insights, Takeaways, and Potential Directions for Improving Mathematical Reasoning

We now present an overview of key insights and takeaways obtained from our detailed benchmarking and evaluation of LLMs on our MWP-MISTAKE dataset. Further, we provide potential directions for improving mathematical reasoning abilities in LLMs.

- 1.
GPT-4o’s Superior Performance: Despite potential data contamination, as observed in GPT-4o’s performance, its superior foundational capabilities enable it to excel consistently across all datasets for mistake detection, rectification, and correct answer derivation. GPT-4o’s remarkable performance positions it as a leading model for complex mathematical reasoning tasks, underscoring the robustness of its fundamental capabilities despite challenges such as data contamination.
- 2.
Challenges with SLMs: The considerable performance gap between smaller language models (SLMs) and larger models like GPT-4 and GPT-4o emphasizes the necessity for advancements in the reasoning capabilities of smaller models. Enhancing these models could make them more competitive and useful in applications where resource constraints are significant.
- 3.
Overfitting and Data Contamination Concerns: The unexpected performance of GPT-3.5Turbo over GPT-4 in certain datasets suggests issues related to overfitting and data contamination. This is evident in the performance disparity, particularly in the GSM-8K dataset, indicating potential memorization of problems during training. Addressing these concerns requires cleaner training datasets and more robust methodologies to avoid overfitting and ensure genuine reasoning skills.
- 4.
Generalization Challenges: The notable performance drop on newer datasets like MATHBENCH and JEEBENCH underscores a critical challenge in generalizing LLMs’ reasoning abilities to novel problems. Addressing this issue is crucial for enhancing the applicability and reliability of LLMs across a broader spectrum of mathematical problems and datasets.
- 5.
SLMs’ Unexpected Performance: The close performance of some SLMs, like Mixtral-8x7B, to larger models such as GPT-4 suggests that these smaller models might also benefit from data contamination. This indicates a need for further investigation into training processes and dataset integrity to ensure fair and accurate performance assessments.

These insights underscore the ongoing necessity to refine LLM training processes, enhance reasoning capabilities, and improve generalization to ensure models can reliably and accurately solve a wide range of mathematical problems. Future research should prioritize addressing overfitting, data contamination, and generalization challenges to advance LLMs in the field of mathematical reasoning.

## 6 Related Work

Recent studies [] indicate that Large Language Models (LLMs) can handle intricate tasks using the Chain of Thought (COT) mechanism []. LLMs have gained significance in solving math word problems (MWPs) [], with MathPrompter [] showcasing excellent results, not only generating correct answers but also complex reasoning steps. Various approaches aim to enhance LLMs’ mathematical capabilities and address challenges []. [] investigates factors like pre-training loss, supervised data, and augmented data, proposing rejection sampling fine-tuning (RFT) to improve mathematical reasoning. WizardMath [] introduces a reinforced Evol-Instruct Feedback (RLEIF) method to enhance reasoning abilities through supervised fine-tuning and PPO training []. MAmmoTH [] combines Chain of Thought (CoT) and Program-of-Thought [] rationales to teach LLMs to use external tools like Python interpreters for mathematical problem-solving.

To assess the correctness of reasoning steps, most existing work [, ] evaluates the quality by directly comparing the final answer. However, some early studies explore reasoning step quality differently. [] measures reasoning step quality by comparing the similarity between generated and reference reasoning. [] treats powerful LLMs as verifiers, asking them to generate judgments for the reasoning steps. [] introduces a new methodology employing validity and redundancy to characterize reasoning quality, along with accompanying LLMs to assess them automatically.

Various methods extend LLMs as verifiers and demonstrate their usage for self-correction []. [] shows that models like GPT-4 align with human preferences, indicating their potential as tools for accessing LLM-generated responses. [] finds that LLMs struggle to find their own reasoning errors in code generation but can correct them with adequate feedback. However, there’s still a lack of clarity in math reasoning and using LLMs for mistake detection and rectification in foreign reasoning steps, not just their own self-generated reasoning steps. Our work focuses on LLMs’ ability to correct MWPs reasoning steps and rectify them to reach the correct answer, as well as whether LLMs generalize to newer and complex datasets.

## 7 Conclusions

This study evaluates large language models (LLMs) like GPT-4o, GPT-4 4, GPT-3.5Turbo, and smaller models (Llama-2-7b-chat, Mixtral-8x7B, Phi-3-mini) on their ability to detect and correct errors in mathematical reasoning. Our MWP-MISTAKE dataset is meticulously curated with incorrect reasoning steps generated using both rule-based methods and smaller language models, ensuring a comprehensive evaluation of LLMs’ error detection and correction capabilities. GPT-4o stands out, demonstrating superior performance in handling complex tasks and correcting mistakes. However, smaller models lag significantly, highlighting the need for advancements in their reasoning capabilities. The analysis also reveals concerns about data contamination and overfitting, particularly in GPT-4’s performance on GSM8K. A notable drop in performance on newer datasets like MATHBENCH and JEEBENCH indicates challenges in generalizing to novel problems. Addressing these issues is crucial for improving LLMs’ reliability and applicability in real-world mathematical problem-solving. Future research should focus on refining training processes, enhancing generalization, and mitigating data contamination to advance the field.

## Appendix

The dataset and code to run all experiments will be made available soon.

## 8 MWP-MISTAKE Dataset

MWP-MISTAKE dataset is curated using 4 different types of well-known datasets. Below are the details of each of the datasets.

- •
GSM-8K []:GSM-8K is a dataset of diverse grade school math word problems created by human writers, involving basic arithmetic operations. Released in November 2021.
- •
MATH []: The MATH dataset is divided into seven categories, each with five difficulty levels. For our study, we used levels 1, 2, and 3 from the algebra and counting and probability categories. Released in November 2021.
- •
MATHBENCH []: MATHBENCH is a recent dataset with questions divided by educational stages, from basic arithmetic to college levels. For our experiment, we chose middle and high-school-level single-choice multiple-choice questions. Released in May 2024.
- •
JEEBENCH []: JEEBENCH is a challenging benchmark dataset for evaluating LLM problem-solving abilities, containing 515 pre-engineering math, physics, and chemistry problems from the IIT JEE-Advanced Exam. For our experiment, we chose mathematics single-choice questions. Released in October 2023.

### 8.1 Prompts to curate reasoning steps in MWP-MISTAKE dataset

GSM-8K and MATH already contain MWP questions, a chain of thought reasoning steps and a final answer. To curate chain of thought reasoning step for MATHBENCH and JEEBENCH we made use of GPT-4. While prompting GPT-4 we made sure that reasoning steps did not contain the final answer, so that final answer is not picked directly from the reasoning step. 1 prompt is used to curate the reasoning steps.
[⬇](data:text/plain;base64,U3RyaWN0bHkgZm9sbG93IHRoZSBiZWxvdyBjb25kaXRpb25zLgoxLiBPdXRwdXQgZm9ybWF0OiBcblJlYXNvbmluZyBDaGFpbjogXG5GaW5hbCBBbnN3ZXI6CjIuIFJlYXNvbmluZyBDaGFpbiBzaG91bGQgYmUgc2VwYXJhdGVkIGJ5IGEgbmV3IGxpbmUgb25seS4KMy4gUmVhc29uaW5nIGNoYWluIGNhbm5vdCBoYXZlIHRoZSBmaW5hbCBhbnN3ZXIuIChSZXBsYWNlIHRoZSBmaW5hbCBhbnN3ZXIgIGluIHRoZSByZWFzb25pbmcgY2hhaW4gd2l0aCBpdHMgY2FsY3VsYXRpb24gb3IgIyMjIykKNC4gRG8gbm90IGluY2x1ZGUgYW55IGFkZGl0aW9uYWwgaW5mb3JtYXRpb24gaW4gdGhlIGZpbmFsIGFuc3dlciAob25seSB0aGUgYW5zd2VyKS4=)
1Strictly follow the below conditions.
21. Output format: \nReasoning Chain: \nFinal Answer:
32. Reasoning Chain should be separated by a new line only.
43. Reasoning chain cannot have the final answer. (Replace the final answer in the reasoning chain with its calculation or ####)
54. Do not include any additional information in the final answer (only the answer).
Listing 1: Prompt to curate reasoning chain without answers.
Table 5 shows examples of default reasoning steps from GSM-8K dataset.

**Table 5: Example of rule based incorrect reasoning step (GSM-8K dataset)**
| Question | Gerald spends $100 a month on baseball supplies.<br>His season is 4 months long.<br>He wants to use the months he’s not playing baseball<br>to save up by raking, shoveling, and mowing lawns.<br>He charges $10 for each. How many chores does he need to average a month<br>to save up for his supplies? |
| --- | --- |
| Final Answer | 5 |
| Gold Reasoning step | He needs to save up $400 because 4 x 100 = 400<br>He has 8 months to earn this money because 12 - 4 = 8<br>He needs to earn $50 a month because 400 / 8 = 50<br>He needs to do 5 tasks a month because 50 / 10 = 5 |
| Shuffle reasoning step | He needs to earn $50 a month because 400 / 8 = 50<br>He needs to save up $400 because 4 x 100 = 400<br>He needs to do 5 tasks a month because 50 / 10 = 5<br>He has 8 months to earn this money because 12 - 4 = 8 |
| Delete reasoning step | He needs to save up $400 because 4 x 100 = 400<br>He needs to earn $50 a month because 400 / 8 = 50<br>He needs to do 5 tasks a month because 50 / 10 = 5 |
| Shuffle numerical values | He needs to save up $400 because 4 x 100 = 400<br>He has 50 months to earn this money because 8 - 8 = 4<br>He needs to earn $12 a month because 400 / 8 = 50<br>He needs to do 5 tasks a month because 50 / 10 = 5 |
| Replace numerical values | He needs to save up $400 because 4 x 100 = 400<br>He has 8 months to earn this money because 12 - 4 = 8<br>He needs to earn $6 a month because 32 / 8 = 50<br>He needs to do 76 tasks a month because 50 / 10 = 5 |
| Shuffle Operations | He needs to save up $400 because 4 x 100 = 400<br>He has 8 months to earn this money because 12 * 4 = 8<br>He needs to earn $50 a month because 400 - 8 = 50<br>He needs to do 5 tasks a month because 50 / 10 = 5 |
| Insert Random Reasoning step | He needs to save up $400 because 4 x 100 = 400<br>Therefore, Faye has $60 - $30 = $30 left.<br>He has 8 months to earn this money because 12 - 4 = 8<br>He needs to earn $50 a month because 400 / 8 = 50<br>He needs to do 5 tasks a month because 50 / 10 = 5 |

## 9 SLMs reasoning steps

SLMs were used to generate chain of thought (COT) reasoning step and final answers for all dataset questions. Each model Llama-2-7b-chat, Mixtral-8x7B, Phi-3-mini where prompted using 1 to curate the reasoning step without an answer. If the final answer was incorrect we filtered out the reasoning steps as incorrect.

Table 6 shows examples of SLM incorrect reasoning steps from GSM-8K dataset.

**Table 6: Example of incorrect reasoning steps generated using SLM’s (GSM-8K dataset)**
| Question | Gerald spends $100 a month on baseball supplies.<br>His season is 4 months long.<br>He wants to use the months he’s not playing baseball to save up by raking, shoveling, and mowing lawns.<br>He charges $10 for each. How many chores does he need to average a month to save up for his supplies? |
| --- | --- |
| Final Answer | 5 |
| Gold Reasoning step | He needs to save up $400 because 4 x 100 = 400.<br>He has 8 months to earn this money because 12 - 4 = 8<br>He needs to earn $50 a month because 400 / 8 = 50<br>He needs to do 5 tasks a month because 50 / 10 = 5<br>#### 5 |
| Llama-2-7b-chat Incorrect reasoning step | Gerald spends $100 on baseball supplies per month.<br>Season is 4 months long.<br>To save up for baseball supplies, Gerald needs to earn more than the cost of supplies per month.<br>Gerald charges $10 for each chore he does.<br>To calculate the number of chores Gerald needs to do per month,<br>divide the cost of supplies by the amount he charges per chore. |
| Llama-2-7b-chat Predicted Answer | "" |
| Mixtral-8x7B Incorrect reasoning step | Gerald spends $100 a month for 4 months, so his total spending for the season is 100*4 = $400<br>Now, we need to find out how many chores he needs to do to earn this amount<br>He charges $10 per chore, so we divide the total amount by the cost per chore: 400 / 10 = #### |
| Mixtral-8x7B Predicted Answer | 40 |
| Phi-3-mini Incorrect reasoning step | Gerald’s total expenditure for baseball supplies = $100 per month * 4 months = $400<br>Gerald earns $10 per chore.<br>To find out how many chores he needs to do to save up $400,<br>we divide his total expenditure by the amount he earns per chore.= $400 / $10= #### |
| Phi-3-mini Predicted Answer | 40 |

## 10 Task T1 and T2

Task T1 evaluates the model’s ability to detect mistakes rectify them and derive the correct answer. 2 was used in a few shot settings for task T1.

Figure: Listing 2: Prompt for Task T1

Task T2 evaulates the model’s ability to detect mistake and solve MWP based on the provided reasoning step. 3 was used in a few shot setting for task T2. Here we insure that final answer is generated with the help of the reasoning steps provided, which may or may not be correct.

Figure: Listing 3: Prompt for Task T2

## 11 T2 Results

Task T2 evaluates the performance in deriving the final answer based on the reasoning step which may or may not be correct. In task T2 we do not instruct the model to correct the reasoning step, and calcualate the final answer based on the provided reasoning step. Due to which we see a signifant drop in performance between Task T1 and Task T2. Table 7 presents the mistake detection performance (F1 score) of all the models with Task T2 and Table 8 presents the performance in deriving the final answer (F1 Score) of all the models.

**Table 7: Mistake Detection Performance (F1 score) on MWP-MISTAKE dataset for Task T2. (D-Default reasoning steps, SM-Smaller model reasoning steps) (Bold: Best, Underline:Second best)**
|  | GSM-8K | MATH | MATHBENCH | JEEBENCH | Average |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Model | D | SM | D | SM | D | SM | D | SM | D | SM | Overall |
| GPT-4o | —- | —- | —- | —- | —- | —- | —- | —- | —- | —- | —- |
| GPT-4 | 0.67 | 0.61 | 0.75 | 0.76 | 0.48 | 0.88 | 0.76 | 0.85 | 0.66 | 0.78 | 0.72 |
| GPT-3.5Turbo | 0.58 | 0.40 | 0.69 | 0.42 | 0.33 | 0.24 | 0.51 | 0.41 | 0.53 | 0.36 | 0.45 |
| Llama-2-7b-chat | 0.11 | NA | 0.22 | NA | 0.11 | NA | 0.75 | NA | 0.30 | NA | 0.30 |
| Mixtral-8x7B | 0.69 | NA | 0.75 | NA | 0.60 | NA | 0.76 | NA | 0.70 | NA | 0.70 |
| Phi-3-mini | 0.56 | NA | 0.52 | NA | 0.46 | NA | 0.54 | NA | 0.52 | NA | 0.52 |
| Claude-3-Opus | —- | —- | —- | —- | —- | —- | —- | —- | —- | —- | —- |

**Table 8: Performance in deriving correct answers (F1 score) on MWP-MISTAKE dataset for Task T2. (D-Default reasoning steps, SM-Smaller model reasoning steps) (Bold: Best, Underline:Second best)**
|  | GSM-8K | MATH | MATHBENCH | JEEBENCH | Average |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Model | D | SM | D | SM | D | SM | D | SM | D | SM | Overall |
| GPT-4o | —- | —- | —- | —- | —- | —- | —- | —- | —- | —- | —- |
| GPT-4 | 0.99 | 0.65 | 0.72 | 0.48 | 0.82 | 0.27 | 0.39 | 0.29 | 0.73 | 0.42 | 0.57 |
| GPT-3.5Turbo | 0.85 | 0.26 | 0.66 | 0.31 | 0.67 | 0.16 | 0.48 | 0.20 | 0.67 | 0.23 | 0.45 |
| Llama-2-7b-chat | 0.84 | NA | 0.33 | NA | 0.44 | NA | 0.36 | NA | 0.49 | NA | 0.49 |
| Mixtral-8x7B | 0.91 | NA | 0.64 | NA | 0.68 | NA | 0.11 | NA | 0.58 | NA | 0.58 |
| Phi-3-mini | 0.92 | NA | 0.62 | NA | 0.65 | NA | 0.49 | NA | 0.67 | NA | 0.67 |
| Claude-3-Opus | —- | —- | —- | —- | —- | —- | —- | —- | —- | —- | —- |

## 12 Model Used

Below are brief details of the models we have used for benchmarking our MWP-MISTAKE dataset.

- 1.
GPT-4o: GPT-4o is a multimodal model by OpenAI, and it has the same high intelligence as GPT-4 Turbo but is much more efficient—it generates text 2x faster and is 50% cheaper. Additionally, GPT-4o has the best vision and performance across non-English languages of any OpenAI model. Last training data: October 2023.
- 2.
GPT-4: GPT-4 is a large multimodal model by OpenAI that can solve difficult problems with greater accuracy than any of OpenAI previous models, thanks to its broader general knowledge and advanced reasoning capabilities. Last training data: September 2021.
- 3.
GPT-3.5Turbo: GPT-3.5Turbo is a large language model by OpenAI GPT-3.5 that can understand and generate natural language or code and has been optimized for chat using the Chat Completions API but work well for non-chat tasks as well. Last training date: September 2021.
- 4.
Claude-3-Opus: Claude-3-Opus is Anthropic’s most capable and intelligent model yet, ideal for navigating complex tasks like in-depth analysis, research, and task automation. Last training data: August 2023.
- 5.
Llama-2-7b-chat: Llama 2 is a collection of pretrained and fine-tuned generative text models ranging in scale from 7 billion to 70 billion parameters from meta. This is the 7B fine-tuned model, optimized for dialogue use cases. Training date: September 2022.
- 6.
Mixtral-8x7B: Mixtral is a Mixture of Experts (MoE) model with 8 experts per MLP, with a total of 45 billion parameters. Despite the model having 45 billion parameters, the compute required for a single forward pass is the same as that of a 14 billion parameter model. This is because even though each of the experts have to be loaded in RAM (70B like ram requirement) each token from the hidden states are dispatched twice (top 2 routing) and thus the compute (the operation required at each forward computation) is just 2 X sequence_length.
- 7.
Phi-3-mini: The Phi-3-Mini-128K-Instruct is a 3.8 billion-parameter by microsoft, lightweight, state-of-the-art open model trained using the Phi-3 datasets. This dataset includes both synthetic data and filtered publicly available website data, with an emphasis on high-quality and reasoning-dense properties. Last training data: October 2023.

## 13 METEOR and BertScore results

BertScore computes a similarity score for each token in the candidate sentence with each token in the reference sentence using the BERT embeddings.
Metric for Evaluation of Translation with Explicit Ordering (METEOR) score is a metric that measures the quality of generated text based on the alignment between the generated text and the reference text. The metric is based on the harmonic mean of unigram precision and recall, with recall weighted higher than precision.

Table 9 and Table 10 present the BertScore and Meteor Score respectively for all the datasets across all models. We observed that these two metric evaluations where not fully able to capture the nuance capabilities of LLMs in rectifying the mistakes within reasoning steps. This can be seen in the results. GPT-4o has a consistently high performance across all the dataset, but when you compare the BERTScore between the corrected reasoning step and ground truth reasoning step you see the rest of the models clearly performing better than GPT-4o. GPT-4 has performed better than GPT-3.5Turbo in most datasets.

**Table 9: BERTscores for correct and incorrect final answers derived after mistake rectification across all models and datasets.**
| Datasets | Models | GPT-4o | GPT-4 | GPT-3.5Turbo | Llama-2-7b-chat | Mixtral-8x7B | Phi-3-mini |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  |  | Correct | Incorrect | Correct | Incorrect | Correct | Incorrect | Correct | Incorrect | Correct | Incorrect | Correct | Incorrect |
| GSM-8K | D | 0.95 | 0.91 | 0.98 | 0.93 | 0.97 | 0.95 | 0.96 | 0.98 | 0.97 | 0.94 | 0.94 | 0.91 |
| SM | 0.83 | 0.82 | 0.84 | 0.82 | 0.84 | 0.82 | NA | NA | NA | NA | NA | NA |  |
| MATH | D | 0.88 | 0.90 | 0.96 | 0.93 | 0.95 | 0.93 | 0.96 | 0.88 | 0.95 | 0.92 | 0.90 | 0.87 |
| SM | 0.84 | 0.80 | 0.83 | 0.81 | 0.84 | 0.81 | NA | NA | NA | NA | NA | NA |  |
| MATHBENCH | D | 0.88 | 0.83 | 0.97 | 0.95 | 0.97 | 0.94 | 0.90 | 0.89 | 0.96 | 0.95 | 0.93 | 0.90 |
|  | SM | 0.82 | 0.82 | 0.85 | 0.82 | 0.84 | 0.83 | NA | NA | NA | NA | NA | NA |
| JEEBENCH | D | 0.89 | 0.89 | 0.88 | 0.87 | 0.94 | 0.95 | 0.86 | 0.82 | 0.85 | 0.87 | 0.70 | 0.85 |
|  | SM | 0.86 | 0.87 | 0.85 | 0.86 | 0.78 | 0.86 | NA | NA | NA | NA | NA | NA |

**Table 10: Meteor Score for correct and incorrect final answers derived after mistake rectification across all models and datasets.**
| Datasets | Models | GPT-4o | GPT-4 | GPT-3.5Turbo | Llama-2-7b-chat | Mixtral-8x7B | Phi-3-mini |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  |  | Correct | Incorrect | Correct | Incorrect | Correct | Incorrect | Correct | Incorrect | Correct | Incorrect | Correct | Incorrect |
| GSM-8K | D | 0.81 | 0.54 | 0.92 | 0.62 | 0.88 | 0.77 | 0.87 | 0.83 | 0.85 | 0.74 | 0.77 | 0.66 |
| SM | 0.33 | 0.27 | 0.37 | 0.31 | 0.37 | 0.32 | NA | NA | NA | NA | NA | NA |  |
| MATH | D | 0.48 | 0.54 | 0.76 | 0.70 | 0.76 | 0.67 | 0.78 | 0.59 | 0.73 | 0.66 | 0.55 | 0.48 |
| SM | 0.32 | 0.28 | 0.30 | 0.26 | 0.33 | 0.28 | NA | NA | NA | NA | NA | NA |  |
| MATHBENCH | D | 0.55 | 0.35 | 0.82 | 0.63 | 0.82 | 0.68 | 0.49 | 0.57 | 0.81 | 0.68 | 0.67 | 0.53 |
|  | SM | 0.33 | 0.30 | 0.32 | 0.25 | 0.32 | 0.29 | NA | NA | NA | NA | NA | NA |
| JEEBENCH | D | 0.37 | 0.31 | 0.30 | 0.22 | 0.49 | 0.54 | 0.15 | 0.13 | 0.53 | 0.46 | 0.20 | 0.25 |
|  | SM | 0.28 | 0.26 | 0.21 | 0.21 | 0.08 | 0.25 | NA | NA | NA | NA | NA | NA |

## 14 Average reasoning Step Length

We noticed that the average word length of rectified reasoning for correct and incorrect for GPT-4o was higher than other models. Table 11 presents the average word length of the rectified reasoning step for all datasets across the models.

**Table 11: Average length of rectified reasoning steps on MWP-MISTAKE dataset**
|  | GSM-8K | MATH | MATHBENCH | JEEBENCH | Average |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Model | D | SM | D | SM | D | SM | D | SM | D | SM | Overall |
| GPT-4o | 100.14 | 131.47 | 147.50 | 182.69 | 312.11 | 323.45 | 647.66 | 619.09 | 301.85 | 314.18 | 308.01 |
| GPT-4 | 66.59 | 122.24 | 79.32 | 121.59 | 146.54 | 140.43 | 356.71 | 322.53 | 162.29 | 176.69 | 169.49 |
| GPT-3.5Turbo | 66.58 | 126.30 | 94.17 | 124.56 | 140.50 | 177.36 | 670.34 | 338.53 | 242.90 | 191.69 | 217.29 |
| Llama-2-7b-chat | 44.73 | NA | 113.35 | NA | 177.67 | NA | 137.05 | NA | 118.20 | NA | 118.20 |
| Mixtral-8x7B | 63.04 | NA | 88.26 | NA | 140.57 | NA | 402.79 | NA | 173.67 | NA | 173.67 |
| Phi-3-mini | 84.92 | NA | 115.10 | NA | 172.57 | NA | 293.90 | NA | 166.62 | NA | 166.62 |
| Claude-3-Opus | 62.18 | 138.91 | 70.60 | 134.05 | 144.85 | 192.84 | 561.88 | 438.44 | 209.88 | 226.06 | 217.97 |

## 15 Data contamination and Memorization

Two different types of prompts were used to test out memorization of individual instances of the dataset. The reasoning step was split into two halves, and models were prompted in two different ways (general and guided) to complete the second half of the reasoning step. To detect contamination we evaluated the overlap score between generated and reference instances using ROUGE-L Score. 4 and 5 shows the prompt used for guided and generation instructions respectively.

Figure: Listing 4: Guided instruction for dataset GSM8K

Figure: Listing 5: General instruction for dataset GSM8K

Here GSM-8K and test are the extra information provided for the model to uniquely identify instances from the source dataset and complete the reasoning step.

Table 12 presents the complete result for the average ROUGE-L score of guided and general for all datasets across all models.

**Table 12: Rouge L score between guided and general instructions on MWP-MISTAKE dataset**
| Datasets | Models | GPT-4o | GPT-4 | GPT-3.5Turbo | Llama-2-7b-chat | Mixtral-8x7B | Phi-3-mini |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  |  | Guided | General | Guided | General | Guided | General | Guided | General | Guided | General | Guided | General |
| GSM-8K | D | 0.57 | 0.44 | 0.67 | 0.56 | 0.53 | 0.49 | 0.26 | 0.28 | 0.46 | 0.44 | 0.32 | 0.32 |
| SM | 0.55 | 0.51 | 0.57 | 0.55 | 0.49 | 0.47 | 0.30 | 0.32 | 0.55 | 0.50 | 0.42 | 0.41 |  |
| MATH | D | 0.44 | 0.25 | 0.52 | 0.48 | 0.39 | 0.38 | 0.25 | 0.26 | 0.39 | 0.32 | 0.26 | 0.27 |
| SM | 0.51 | 0.38 | 0.54 | 0.54 | 0.45 | 0.44 | 0.30 | 0.29 | 0.48 | 0.46 | 0.38 | 0.39 |  |
| MATHBENCH | D | 0.43 | 0.41 | 0.48 | 0.46 | 0.38 | 0.36 | 0.26 | 0.28 | 0.36 | 0.36 | 0.30 | 0.30 |
|  | SM | 0.40 | 0.38 | 0.43 | 0.42 | 0.39 | 0.38 | 0.30 | 0.33 | 0.40 | 0.38 | 0.29 | 0.30 |
| JEEBENCH | D | 0.43 | 0.39 | 0.42 | 0.40 | 0.34 | 0.33 | 0.27 | 0.25 | 0.38 | 0.34 | 0.33 | 0.31 |
|  | SM | 0.32 | 0.29 | 0.34 | 0.35 | 0.31 | 0.24 | 0.22 | 0.25 | 0.26 | 0.27 | 0.20 | 0.22 |

## 16 Running Experiment Multiple Times

While running experiments on all models (LLMs and SLMs) we used the default hyperparameters to generate tokens. We ran a subset of the dataset on different prompt variations and saw comparable performance for various prompts. Due to the limitation of the API key, we were only able to run GPT-4o model on the GSM-8K dataset. On rerun we got very similar results, with an error rate of <= 0.01.

## Peer Reviews

confidence: 3
contribution: 2
initial_rating: 5
presentation: 3
questions: Questions and Suggestions:
- Figure 2 is not referenced anywhere in the paper.
- Lines 208-209: Do you mean Table 2? What is the difference between Table 2 and Table 7?
- Lines 242-243: Why is GPT-4o underperforming in both simpler and more complex datasets? A brief explanation of the possible challenges it faces with these datasets would be helpful.
- A detailed analysis of mistake types would strengthen the study. For instance, it would be valuable to know which types of mistakes models consistently struggle to detect.
soundness: 2
strengths: **Originality**:
The proposed dataset is designed to evaluate models in identifying and correcting reasoning mistakes, a critical yet underexplored area in LLM performance. Its focus on these reasoning capabilities rather than final answer accuracy bridges the gap in understanding models’ overall problem-solving capabilities.

**Quality**:
The paper evaluates a wide range of models, from large to small models, from open to proprietary models. It also provides thorough analysis from various aspects, including question variations, memorization, etc.

**Clarity**:
Dataset construction and experiment setups are generally clear. By defining different categories of mistakes (rule-based or SLM-based) and different types of reasoning failures (eg, wrong numerical values, shuffled steps), the paper provides a structured assessment that is easy to understand.

**Significance**:
Understanding reasoning errors and their correction is meanful for educational applications and the wider community. The paper also highlights potential data contamination and memorization issues, emphasizing the need for more sophisticated training and evaluation practices.
summary: This paper proposes a dataset for evaluating LLMs’ abilities in detecting and correcting mistakes. The dataset contains math word problems with both correct and wrong reasoning steps, sourced from various datasets including SVAMP, GSM-8K, MATH, MATHBENCH, and JEEBENCH. To systematically generate mistakes, the authors employ two approaches: (1) rule-based techniques, and (2) using small language models as bad reasoners. It provides a comprehensive benchmarking of various LLMs and SLMs, with insights into their capabilities and limitations.
weaknesses: - While the authors claim that rule-based mistakes mimic human reasoning errors, my concern is that some of these transformations lack natural coherence. For example, shuffling reasoning steps or numeric values often results in errors that do not reflect authentic human errors in mathematical reasoning, which can be distracting to models. Could this be the reason why models struggle more with rule-based errors compared to SLM-generated mistakes? I would suggest refining the rule-based methods to better mimic typical student errors, which could give a more valid evaluation of models’ ability to detect naturally occurring mistakes. 

- Additionally, deleting reasoning steps raises a question about whether such omissions genuinely impact the final answer’s correctness or reasoning quality. In cases where reasoning steps are deleted but the answer remains correct, it would be insightful to investigate whether models still detect these partial solutions as errors and if such ambiguities lead to lower error detection rates.

- In related work, the authors mention studies suggesting that LLMs struggle to detect their own mistakes. However, the paper focuses only on mistakes generated by smaller models, without examining those generated by models of similar or competitive capabilities. Exploring a model’s ability to detect its own mistakes in its own reasoning would add substantial value to the study, particularly given its focus on evaluating reasoning capabilities. For example, the authors could investigate whether GPT-4 or GPT-4o can detect mistakes it generates, which would offer more insights into the limitations of LLMs’ self-evaluation in more complex scenarios.

---

confidence: 4
contribution: 3
initial_rating: 5
presentation: 2
questions: Please review the weakness section
soundness: 3
strengths: This paper meticulously develops six crafted rules to assess LLMs' ability to detect reasoning errors, providing valuable insights into their mathematical reasoning capabilities in realistic settings. The authors compare current models in terms of mistake detection and correction, while also exploring potential contamination and memorization issues, thereby deepening our understanding of these models.
summary: This paper examines the capability of current LLMs to detect and correct errors in mathematical reasoning. It introduces a new benchmark, MWP-MISTAKE, featuring both correct and incorrect reasoning steps generated by rules or smaller models. Experiments reveal that LLMs still have room for improvement in consistently identifying mistakes across simple and complex datasets. The authors note that contamination issues and generalization challenges hinder reliable performance in real-world applications.
weaknesses: 1.	In the figure 1, GPT-3.5 Turbo fails to detect errors while GPT-4o succeeds. Can GPT-3.5 Turbo correctly solve this question?
2.	Regarding the metric (line 186), is the F1 score for step-level or path-level detection performance?
3.	I suggest presenting the results of model performance in solving questions alongside error detection. Are questions that models solve correctly easier for them to detect errors compared to more challenging questions?
4.	In Table 3, is the average computed based on all data samples? The data sizes from different sources are unbalanced. Additionally, compared to GSM8K, why does detection performance decrease on D but increase on SM (e.g., JEEBENCH)?
5.	I disagree with the statement in lines 253-255 that “GPT-3.5 Turbo performs similarly to GPT-4 and even surpasses it on certain datasets like GSM-8K.” The results in Table 4 do not support this as well. In lines 258-262, the authors claim that the performance of Llama-3-8b-finetuned rivals GPT-4 due to domain-specific fine-tuning. Have you tested Llama-3-8b-finetuned for memorization?
6.	In section 4.4, while GPT-4o shows high ROUGE-L, it also has the best performance in mistake detection and question answering. Could this high performance be attributed to contamination? Can you provide qualitative cases to compare these LLMs? I am also curious about the contamination scores for Llama-3-8b-finetuned and O1.
7.	Citation issue? (line 44)
8. Some relevant work in evaluation is overlooked, such as "MR-GSM8K: A Meta-Reasoning Benchmark for Large Language Model Evaluation".

---

confidence: 4
contribution: 2
initial_rating: 3
presentation: 2
questions: - The paper mentions that the correct reasoning steps are generated by GPT-4 and manually verified for correctness. I am wondering what proportion of the data has been manually checked? Among these data, what is the proportion of cases where the reasoning steps generated by GPT-4 are completely correct?
- What does “Smaller model reasoning steps” in table 3 mean? Does it mean the incorrect steps are generated by smaller language models?
- The paper evaluates the ability of LLMs to detect the correctness of reasoning steps and generate correct answers, but it does not evaluate the accuracy rate of the generated reasoning steps?
soundness: 2
strengths: - The proposed dataset is helpful to the relevant research community.
- This work proposes potential directions for enhancing LLM capabilities in reasoning.
summary: This work presents a dataset MWP-MISTAKE to evaluate LLM’s abilities to detect and correct reasoning mistakes, the experimental results show that current LLM remains weak in mistake correction.
weaknesses: - The finding that LLM remains weak in mistake detection/correction [1][2] is not novel.
- The reliability and accuracy of the MWP-MISTAKE dataset are questionable. The correct reasoning steps of MWP-MISTAKE are generated by GPT-4 and manually verified for correctness. However, there is a lack of detailed statistics on manual inspection, and it is not explained how the authors handle it to ensure data quality when the reasoning steps generated by GPT-4 are incorrect.
- The evaluation setup is hard to follow.
- There are some typos in the paper, e.g., “corect” in line 105, “their” in line 154.

[1] Large Language Models Cannot Self-Correct Reasoning Yet.

[2] SELF-[IN]CORRECT: LLMs Struggle with Discriminating Self-Generated Responses

---

confidence: 5
contribution: 2
initial_rating: 5
presentation: 3
questions: Refer to weaknesses.
soundness: 3
strengths: * The paper introduces a novel dataset (MWP-MISTAKE) that evaluates LLMs’ ability to detect and correct errors in reasoning, filling a gap in prior research that focused mainly on final answer accuracy.
* The research is comprehensive, benchmarking multiple state-of-the-art models on various tasks and datasets, revealing significant insights about their reasoning abilities.
* The experiments are well-documented, with clear descriptions of tasks and metrics. The inclusion of both rule-based and model-generated errors is a thoughtful addition.
* The findings are relevant to applications where reliable reasoning is crucial, such as educational tools, and provide a foundation for future improvements in LLMs.
summary: This paper aims to assess the reasoning capabilities of large language models (LLMs) when solving math word problems (MWPs). It emphasizes the importance of evaluating LLMs’ ability not only to solve problems but also to detect and correct errors in their reasoning processes. The authors introduce a novel dataset called MWP-MISTAKE, which contains correct and incorrect reasoning steps generated through rule-based methods and smaller language models (SLMs). The study benchmarks several LLMs, including GPT-4o, GPT-3.5Turbo, Claude-3-Opus, and others, revealing insights about their strengths and limitations in handling MWPs, especially in error detection and correction.
weaknesses: * Overall, the primary contribution lies in evaluating the reasoning abilities of state-of-the-art (SOTA) LLMs in mathematical problem-solving, with a focus on mistake detection, etc. The contribution remains somewhat limited in scope.
* The paper identifies data contamination as a limitation but could provide more detailed mitigation strategies. This weakens the reliability of some results.
* While the study covers multiple datasets, the models show reduced performance on newer, complex datasets like JEEBENCH, suggesting limited generalization.
* Although rectification abilities are discussed, the results indicate that even advanced models struggle with consistent correction, which highlights the need for stronger frameworks for error handling.
* While the paper identifies several challenges, it offers limited practical solutions or recommendations for overcoming the observed weaknesses in mistake detection.

Meta-review: The paper presents a dataset for evaluating LLMs’ abilities in detecting and correcting mathematical reasoning mistakes. The dataset contains math word problems with both correct and wrong reasoning chains. The proposed pipeline includes perturbations to the original reasoning chains based on (1) rule-based techniques, and (2) using less capable language models.

The reviewers generally found the problem studied to be important, the presentation to be clear, the dataset to be helpful for the community, and that the experiments insightful. 

However, reviewers found several issues regarding the paper, including:  
-The scope of the contributions of the work to be limited and incremental with respect to existing findings in the literature  
-The perturbation based method might not accurately reflect the mistakes made by human (partially addressed in rebuttal)  
-Some of the relevant work might be overlooked
-Evaluation setup needs better clarity 
-While the paper discusses several challenges such as contamination there are limited discussions or practical solutions discussed   

While the author response attempts to address several of these issues, key concerns—such as the overall incremental contribution—remain unresolved.

Decision: Reject