Error Reflection Prompting: Can Large Language Models Successfully Understand Errors? (2025)

Abstract: Prompting methods for language models, such as Chain-of-thought (CoT), present intuitive step-by-step processes for problem solving. These methodologies aim to equip models with a better understanding of the correct procedures for addressing a given task. Despite these advancements, CoT lacks the ability of reflection and error correction, potentially causing a model to perpetuate mistakes and errors. Therefore, inspired by the human ability for said tasks, we propose Error Reflection Prompting (ERP) to further enhance reasoning in language models. Building upon CoT, ERP is a method comprised of an incorrect answer, error recognition, and a correct answer. This process enables the model to recognize types of errors and the steps that lead to incorrect answers, allowing the model to better discern which steps to avoid and which to take. The model is able to generate the error outlines itself with automated ERP generation, allowing for error recognition and correction to be integrated into the reasoning chain and produce scalability and reliability in the process. The results demonstrate that ERP serves as a versatile supplement to conventional CoT, ultimately contributing to more robust and capable reasoning abilities along with increased interpretability in how models ultimately reach their errors.

## introduction large language models (llms) have demonstrated remarkable performance across numerous nlp tasks @xcite . yet scaling them is costly, prompting interest in more efficient methods like chain-of-thought (cot) @xcite , which guides step-by-step reasoning. a key shortcoming of cot, however, is that it lacks a mechanism to recognize and correct its own mistakes @xcite . this can lead models to persist with errors, as they lack explicit guidance to reflect on and revise flawed reasoning. drawing on the human ability to self-reflect and amend mistakes @xcite , we propose error reflection prompting (erp). erp supplies an incorrect answer alongside an explanation of the errors, then presents the correct reasoning. by exposing common pitfalls and clarifying why they are wrong, erp steers models away from these errors in future reasoning. evaluations on multiple benchmarks with gpt-3.5 and gpt-4 confirm that erp consistently outperforms standard cot, notably lowering specific error types (e.g., algebraic and calculation mistakes) and balancing the overall error distribution. thus, erp adds a complementary layer of self-correction to cot, enhancing both accuracy and transparency in model outputs. ## error analysis in this section, we explore how erp affects the distribution of errors in answering questions. although erp scored higher than cot @xcite for all the tests, the error distribution was not the same. this discrepancy is analyzed to explain erp's behavior in a llm. confidence values for annotations were calculated by taking a subset of fifty randomly sampled annotated errors from both gpt-3.5 and gpt-4 inference errors and determining their validity. these values can be found in figure 3 shows a full set of human annotated errors on gpt-4 on gsm8k @xcite . indeed, the degree of effectiveness in targeting specific errors depends on the complexity of the prompt and the model's ability to comprehend the complexity of the error and its implications. for example, misinterpretation errors were decreased in erp versus cot, although missed steps were less affected. ## conclusion by exploring how weighting errors in explanations can enhance language models' reasoning, we tackled some limitations of cot. we found that highlighting and explaining common mistakes leads to more accurate, coherent responses. though drawbacks remain-like overfitting to specific errors or creating effective rationales-our results show erp's versatility as a cot extension: it not only identifies errors but also uncovers the steps that cause them. future research into error-based prompting may further boost reasoning abilities and bring models closer to human-like thought processes.