Instruct-of-Reflection: Enhancing Large Language Models Iterative Reflection Capabilities via Dynamic-Meta Instruction (2025)

Abstract: Self-reflection for Large LanguageModels (LLMs) has gained significant attention. Existing approaches involve models iterating and improving their previous responses based on LLMs’ internal reflection ability or external feedback. However, recent research has raised doubts about whether intrinsic self-correction without external feedback may even degrade performance. Based on our empirical evidence, we find that current static reflection methods may lead to redundant, drift, and stubborn issues. To mitigate this, we introduce **I**nstruct-**o**f-**R**eflec**t**ion (**IoRT**), a novel and general reflection framework that leverages dynamic-meta instruction to enhance the iterative reflection capability of LLMs. Specifically, we propose the instructor driven by the meta-thoughts and self-consistency classifier, generates various instructions, including refresh, stop, and select, to guide the next reflection iteration. Our experiments demonstrate that IoRT achieves an average improvement of 10.1% over established baselines in mathematical and commonsense reasoning tasks, highlighting its efficacy and applicability. Our code is available at https://github.com/llp635/IoRT.

## introduction large language models (llms) @xcite @xcite have demonstrated remarkable capabilities across various natural language processing tasks @xcite @xcite , particularly when leveraging a range of prompting strategies such as chain-of-thought @xcite which improve the reasoning ability of llms without increasing model parameters @xcite @xcite . inspired by how humans iteratively refine their written text @xcite , the concept of "self-correction" @xcite has been proposed and garnered extensive attention where llms first generate an initial response, then gather feedback to refine previous responses @xcite @xcite . however, recent studies @xcite raise two doubts about the self-correction capabilities of llms: first, @xcite use oracle labels regarding the answer correctness to guide the self-correction process. however, in practice, the availability of oracle labels seems counter-intuitive because there seems to be little reason to deploy llms for problem solving if the ground truth is already known. second, without any external or human feedback, the performance after llms' intrinsic self-correction even deteriorates. our research also focuses on the reflective capabilities of llms. unlike previous studies that primarily investigate performance changes in intrinsic reflection, we explore whether the performance of llms in reflection deteriorates when external feedback is available. therefore, in our exploration experiments, we evaluate the performance of critic @xcite , which leverages the results of tool interactions as external feedback, enabling a systematic investigation of how this feedback impacts the performance of reflection. our findings ( § 3.1) indicate that neither intrinsic self-correction nor critic performs satisfactorily. to investigate the reasons behind that, we comprehensively analyze the performance across multiple iterations of reflection, which is different from prior research focusing on a single reflection. figure 1 shows that the i th reflection is derived from the (i -1) th response, this static iterative reflection leads to three issues: 1) redundant (✓⇒✓): although it does not alter accuracy in iterations, it does increase overhead. 2) drift (✓⇒✗): selfcorrection is highly random, undermining both accuracy and reliability. 3) stubborn (✗⇒✗): llms consistently persist in errors, failing to properly identify and correct potential mistakes. additionally, we found that the prevalence of these three issues varies across different llms and tasks. as a remedy to these challenges, we propose instruct of reflection (iort), an efficient and dynamic iterative reflection framework designed to continuously optimize the reflection process. the framework including three stages: (i) meta-thinker generates meta-thoughts, (ii) refreshing to generate an initial response, and reflector to self-reflect based on feedback. (iii) instruct the iteration. notably, aiming to mitigate redundant, drift, and stubborn issues, in the third stage, the instructor, augmented by meta-thought and driven by the selfconsistency classifier, generates refresh, stop, or select instructions to direct the next reflection iteration. specifically, when the instructor determines that both responses have correctly addressed the problem, it generates a stop instruction to promptly halt the reflection process, effectively reducing redundancy. if the instructor assesses that both answers are incorrect, it generates a refresh instruction, helping the llms prevent stubbornness. in cases where answers are inconsistent, the instructor selects the more optimal response as the output of the current iteration and the foundation for the next reflection, thereby reducing drift and improving performance accuracy. we evaluate the performance of iort across var-ious llms and two distinct tasks: mathematical reasoning and commonsense reasoning. the result demonstrates that iort outperforms baselines on established strong baselines, underscoring the critical role of the instructor in augmenting the iterative reflection capabilities of llms. our main contributions can be summarized as follows: ## empirical evidence for iterative reflection prior research primarily focuses on performance changes in a single reflection, which fails to fully capture the potential of reflection for long-term improvements. to gain deeper insights into how reflection can gradually refine answers, we comprehensively evaluate the performance changes of self-correct and critic @xcite across multiple reflection iterations @xcite . ## instruct-of-reflection (iort) prior sections illustrate the challenges llms encounter in static iterative reflection such as redundancy, drift and stubborn. how to precisely identify effective reflections while mitigating the impact of harmful or redundant reflections? as a remedy, we innovatively propose instruct-of-reflection (iort), a novel framework that implements dynamic-meta instruction for the iterative reflection process. we can get an overview of the iort framework in figure 4 . specifically, iort involves three main steps: generate meta-thoughts, refresh and selfreflect, and instruct the iteration. as the core module, the instructor is augmented by meta-thoughts and driven by self-consistency classifier, generating instructions including refresh, stop, and select, directing the next reflection iteration. ## conclusion we conducted a comprehensive analysis of the iterative reflection performance of llms. the empirical evidence suggests that the performances of these reflection methods are unsatisfactory, primar-ily due to the limitations of static iterative reflection, which leads to redundant, drift, and stubborn issues. to mitigate this, we propose instruct-of-reflection (iort), a dynamic iterative reflection framework that integrates abstract reasoning into the reflection, generating adaptive instruction to regulate the iterative reflection. experimental results demonstrate that iort surpasses a series of strong baselines and reduces overhead across a variety of scenarios and with different llms.