Evaluating Step-by-step Reasoning Traces: A Survey (2025)

Abstract: Step-by-step reasoning is widely used to enhance the reasoning ability of large language models (LLMs) in complex problems. Evaluating the quality of reasoning traces is crucial for understanding and improving LLM reasoning. However, existing evaluation practices are highly inconsistent, resulting in fragmented progress across evaluator design and benchmark development. To address this gap, this survey provides a comprehensive overview of step-by-step reasoning evaluation, proposing a taxonomy of evaluation criteria with four top-level categories (factuality, validity, coherence, and utility). Based on the taxonomy, we review different datasets, evaluator implementations, and recent findings, leading to promising directions for future research.

## introduction large language models (llms) have demonstrated remarkable capabilities in reasoning on complex problems, such as logic, math, and science. at the core of this versatility lies step-by-step reasoning @xcite , where the llm generates an intermediate reasoning trace before presenting the final answer. the reasoning ability of llms is often measured in terms of answer accuracy, i.e., finding the correct answer for a complex reasoning problem @xcite . however, answer accuracy is generally insufficient for measuring llms' reasoning ability, as the correct answer does not imply the correctness of the preceding reasoning trace @xcite @xcite . furthermore, assessing the quality of the reasoning trace can directly lead to better reasoning ability of llms by verifierguided search @xcite @xcite and reinforcement learning @xcite . ## evaluation criteria previous studies have proposed various criteria for evaluating step-by-step reasoning @xcite @xcite , but these works failed to propose a complete taxonomy that covers diverse reasoning tasks (e.g., literature in factual reasoning and math reasoning have focused on different criteria). in this section, we propose a unified taxonomy of reasoning trace evaluation criteria that spans different reasoning tasks and evaluators. we categorize them into four key dimensions: factuality, validity, coherence, and utility (figure 3 ) 1 . ## meta-evaluation datasets datasets that annotate llm-generated reasoning traces serve as key resources for training evaluators and conducting meta-evaluations between evaluators. a summary of existing datasets is provided in among these, one of the most influential is prm800k @xcite . prm800k consists of crowdsourced tertiary validity labels (positive, negative, neutral) assigned step by step, framing reasoning trace error detection as a sequence classification problem. its design has inspired several successors @xcite , setting the paradigm for subsequent reasoning trace evaluation resources. to address different needs, several extensions have been developed. since human annotations additional details are provided in appendix b. ## evaluator types the goal of reasoning trace evaluators is to assess reasoning traces by assigning scores. however, choosing the right evaluator for the target criteria and task is non-trivial. for instance, there is no guarantee that evaluators designed for factuality and multi-hop question answering will seamlessly work on math reasoning problems. in this survey, we provide a comprehensive overview of diverse reasoning trace evaluators, @xcite . we summarize eight popular evaluator types based on the criteria they evaluate (summarized in ## further improving evaluators this section discusses some of the recent empirical findings on improving evaluators beyond choosing different types, e.g., training data, input format, and scaling compute. validity and utility are complementary validity measures if the step is logically correct, while utility measures if the step makes progress towards the correct answer. initially, utility-based process reward models were proposed as an alternative for validity, since constructing validity data often requires a costly annotation process @xcite . under the hood, there lies an implicit assumption that useful steps are mostly valid. however, recent works show that the two criteria are rather complementary, from training sequence classifiers to using critic models. @xcite trains a sequence classifier by only considering steps that are both valid (judged by critic models) and useful (by mcts-based rollouts) steps as positive, substantially improving performance over baselines that only consider validity or utility (figure 4 , sequence classifiers). @xcite ; @xcite has also shown that averaging validity and utility scores from critic models results in better performance than individual scores in best-of-n decoding. the misalignment between validity and utility is mainly caused by steps that are logically wrong but reach the correct answer @xcite @xcite . these invalid but useful steps, also known as unfaithful reasoning of llms @xcite , might lead to overestimation of reasoning ability @xcite . partial context allows efficient and accurate evaluation validity and coherence evaluate a step based on its previous steps. the most intuitive way is to use the full context (all preceding steps). however, this approach is not feasible when the trace exceeds the context length of the evaluators, e.g., large reasoning models' traces are often too long to apply critic models @xcite ). an alternative solution is to use a partial context, where only relevant parts of the query and preceding steps are selected and passed to the evaluator @xcite . these works first construct a directed entailment graph, and evaluate the step only based on the identified premises. this allows evaluators to use shorter context, which is both computationally efficient @xcite and even more accurate as distractors are removed from the context @xcite (figure 4 , partial context). moreover, the graph structure also distinguishes direct errors (premises are valid but the reasoning is invalid) and accumulated errors (premises are invalid but reasoning is valid) @xcite . test-time scaling improves evaluator performance test-time scaling is a general paradigm where investing more test-time compute leads to improved performance. test-time compute can be scaled in diverse directions, such as sampling the output multiple times @xcite or generating more tokens during a single inference @xcite @xcite . this paradigm can be extended to critic models that reason on reasoning traces, especially in meta-evaluation benchmarks. when applying majority voting of independently sampled k evaluator scores in generative models (critic model, generative verifiers), the accuracy in predicting incorrect steps increases linearly with the scale of log k @xcite @xcite (figure 4 , majority voting). furthermore, using large reasoning models (lrms) with stronger reasoning capability by generating longer traces @xcite @xcite leads to significant improvement in error detection (figure 4 , scaling (lrms)). in verifier-guided search settings, one can either scale exploration or scale evaluation. for instance, in best-of-n decoding, one can increase the number of responses or use critic models that produce longer outputs. what is the optimal strategy with a constrained computing budget? for relatively weaker evaluators, simple majority voting @xcite often outperforms verifier-guided search @xcite . however, using stronger evaluators, e.g., sequence classifiers with better training data @xcite or critic models with stronger reasoning capabilities @xcite for best-of-n can effectively outperform majority voting using the same computation budget. ## future directions evaluating real-world reasoning traces with external knowledge existing datasets for reasoning trace evaluation are mostly restricted to simple factual reasoning (e.g. factual multi-hop question answering) or self-contained problems (e.g. math problems). however, many realistic reasoning tasks such as repository-level coding @xcite , medicine @xcite , and law @xcite require external up-to-date knowledge retrieval-augmented generation @xcite . developing evaluators and meta-evaluation benchmarks for these tasks will significantly enhance the applicability of reasoning trace evaluation in more realistic scenarios. evaluating long, complex reasoning traces following openai o1 (openai, 2024b), numerous large reasoning models (lrms) that generate long, complex traces involving self-verification and backtracking were introduced (deepseek-ai, 2025; @xcite . however, existing evaluators are not suitable for these complex traces. for instance, assigning a single scalar score (e.g., sequence classifiers) will make invalid steps corrected afterwards (wait, this reasoning is not correct.) and ones not corrected indistinguishable. since lrm reasoning traces can contain critical errors @xcite , the effort to develop evaluation resources for such traces will lead to a better understanding of lrms' behaviors and further improvement in their performance and credibility. advanced methods for finding premises. nlibased validity and coherence evaluation significantly benefit from determining the previous steps that the current step uses as a premise @xcite . however, finding such steps is not a trivial task. roscoe @xcite uses the minimum nli score of all (previous step, current step) combinations, which ignores cases where a step has multiple premises. recent works @xcite @xcite ) make the reasoner llm annotate the premises of the given step. plausible but underexplored approaches include applying uncertaintybased methods @xcite or training a parser that annotates the logical dependencies between steps as graphs @xcite . ## conclusion this survey aims to organize existing criteria and methods for step-by-step reasoning evaluation, which is crucial for understanding and improving llm's reasoning capabilities. we provide a unified taxonomy for evaluation criteria, a comprehensive review of existing evaluators and their implementation, and examine recent directions on how to improve these evaluators. still, diverse challenges remain in evaluating step-by-step reasoning traces. as new reasoning tasks and methods emerge, existing evaluators often become obsolete for evaluating complex reasoning traces from new tasks and models. as llms are now involved in challenging and high-stakes reasoning tasks in the real world, understanding the nature of their errors and precisely evaluating the reasoning trace will remain important.