Temporal Consistency for LLM Reasoning Process Error Identification (2025)

Abstract: Verification is crucial for effective mathematical reasoning. We present a new temporal consistency method where verifiers iteratively refine their judgments based on the previous assessment. Unlike one-round verification or multi-model debate approaches, our method leverages consistency in a sequence of self-reflection actions to improve verification accuracy. Empirical evaluations across diverse mathematical process error identification benchmarks (Mathcheck, ProcessBench, and PRM800K) show consistent performance improvements over baseline methods. When applied to the recent DeepSeek R1 distilled models, our method demonstrates strong performance, enabling 7B/8B distilled models to outperform all 70B/72B models and GPT-4o on ProcessBench. Notably, the distilled 14B model with our method achieves performance comparable to Deepseek-R1.

## introduction large language models (llms) have shown impressive capabilities in reasoning tasks @xcite @xcite @xcite , but still often make mistakes when generating complex multistep solutions. to address this issue, process reward models (prms) @xcite have been introduced to guide generations. instead of providing feedback solely on the final answer, prms evaluate every intermediate step in the reasoning chain, thereby aligning the model's chain of thought with correct logical sequences. however, existing prms face several key limitations that hinder their broader applicability. first, training a prm requires large-scale, high-quality annotated datasets, making the process highly dataintensive and costly to scale @xcite . second, prms exhibit poor out-of-domain generalization; models trained on specific problem distributions often struggle to accurately evaluate reasoning steps when confronted with diverse problem types @xcite . finally, the effectiveness of prms is intrinsically limited by the capability of the base model @xcite . these challenges highlight the need for further research to develop more scalable process supervision techniques in llms. an alternative way is to adopt some trainingfree approaches like majority voting @xcite or debate-based approaches @xcite , which have shown effectiveness in aggregating opinions and resolving conflicts between multiple reasoning trajectories. nevertheless, we found that both methods show limitations when applied to mathematical process error identification tasks. majority voting often fails when errors are identified by only a minority of llms @xcite . debate-based approaches sometimes struggle due to an asymmetry in mathematical reasoning: erroneous reasoning paths tend to generate lengthy, seemingly logical justifications, while correct reasoning paths provide only simple justifications. this asymmetry can cause debate methods to favor incorrect justifications, as more elaborate (though flawed) arguments may overshadow simple (but correct) justifications. to address these limitations, we develop a simple but effective training-free approach to enhance process error identification capabilities. the intuition is to leverage the consistency between a sequence of self-reflection actions because the llms should be more likely to remain consistent and confident when asked to review correct validations. as shown in figure 2 , we propose the temporal consistency method, where each llm iteratively checks its identifications, and the final output is only produced when multiple llms demonstrate consistent self-checking over time, effectively reproblem steps initial verification phase llm self-check verifier ... iterative self-checking phase consistency with self-check? convergence check output self-check ducing unstable incorrect identifications. we further evaluate our approach across three annotated mathematical step datasets, prm800k @xcite , processbench @xcite , and mathcheck * foot_0 @xcite . our experiments demonstrated consistent performance gains across different models, benchmarks, and difficulty levels. we then conducted experiments on r1 distilled models @xcite , where our method achieved remarkable improvements: as shown in figure ## methodology in this section, we introduce our method that utilizes multiple rounds of validation to improve identification accuracy. we begin by defining the process error identification task. task definition. given a problem p and its stepby-step solution @xmath0, s 1 , . . . , s n-1 }, where each s i represents the i-th solution step, our objective is to identify the first incorrect step, if any, and output a location index @xmath1, 0, . . . , n -1}. here, @xmath2, while for @xmath3, s loc represents the first incorrect step. we now introduce the temporal consistency algorithm. this method adds a temporal dimension to the verification process by having each llm consider its own previous assessment, leveraging consistency in a sequence of self-reflection. we employ k llms as verifiers, denoted by llm 1 , . . . , llm k . the algorithm has three phases: initial verification phase. for each @xmath4, . . . , k}, given a problem p , a solution s, and a designated process error identification prompt x verify , llm i examines the solution step by step. it identifies the location of the first incorrect step loc i 1 , and provides the corresponding reasoning response res i 1 : iterative self-checking phase. for time steps @xmath5, let (loc i t-1 , res i t-1 ) represent the verification results from the previous iteration for 2 https://openrouter.ai each @xmath6, . . . , k}. with a designated selfverification prompt x self-check , llm i performs a subsequent self-assessment: the distinction between the initial verification phase and the self-checking phase is incorporating previous verification results to provide additional context. this temporal dependency enables the llms to potentially correct initial misidentifications. figure 4 illustrates the self-checking mechanism. convergence check. after each iteration t, the algorithm determines the majority identification loc t by applying a majority voting function: majorityvote(loc 1 t , . . . , loc k t ) = argmax @xmath7,••• ,n-1} {i : loc i @xmath8. (1) this function aggregates the verification outcomes from k different llms and returns the error step that is most frequently identified. specifically, {i : loc @xmath9, where she intended to serve stew. she noticed that she was out of plastic spoons, so she bought a new package of spoons. later, her husband also bought a package of 5 new spoons and gave them to julia… first, initially, julia had no spoons. she then bought a new package of spoons. her husband also bought a package of 5 new spoons and gave them to her. second, in total, she received 5 + 5= 10 new spoons from both her purchase and her husband's purchase. (s 1 is incorrect)… @xmath10. the first error occurs in paragraph (1). @xmath11... we can determine that the first error occurs in paragraph (0). @xmath12... the earliest error occurs in paragraph (0). ## conclusion we presented an temporal consistency approach for improving mathematical process error identification in language models. our method leverages temporal consistency patterns in verification behavior, allowing llms to recheck their judgments through multiple rounds. we demonstrated how this approach effectively improves verification accuracy across different models and problem types through empirical evaluation. our key insight is that the temporal stability of verifications can serve as a reliable indicator of correctness. this finding opens new directions for developing methods focusing on consistency over time rather than agreement across agents. our results suggest that incorporating temporal dynamics can enhance the reliability of mathematical reasoning methods.