Assessing LLM Reasoning Steps via Principal Knowledge Grounding (2025)

Abstract: Step-by-step reasoning has become a standard approach for large language models (LLMs) to tackle complex tasks. While this paradigm has proven effective, it raises a fundamental question: How can we verify that an LLM’s reasoning is accurately grounded in knowledge? To address this question, we introduce a novel evaluation suite that systematically assesses the knowledge grounding of intermediate reasoning. Our framework comprises three key components. (1) Principal Knowledge Collection, a large-scale repository of atomic knowledge essential for reasoning. Based on the collection, we propose (2) knowledge-grounded evaluation metrics designed to measure how well models recall and apply prerequisite knowledge in reasoning. These metrics are computed by our (3) evaluator LLM, a lightweight model optimized for cost-effective and reliable metric computation. Our evaluation suite demonstrates remarkable effectiveness in identifying missing or misapplied knowledge elements, providing crucial insights for uncovering fundamental reasoning deficiencies in LLMs. Beyond evaluation, we demonstrate how these metrics can be integrated into preference optimization, showcasing further applications of knowledge-grounded evaluation. Our evaluation suite is publicly available.

## introduction recently, large language models (llms) @xcite have demonstrated remarkable reasoning capabilities. these models have achieved substantial performance gains by generating their own intermediate reasoning steps before arriving at a final answer. however, such gains are often assessed solely through endtask metrics, which evaluates only the final output. so, the answer is (d) ." we have . we can simplify this by combining the fractions with the same denominator, 14. therefore, . to address this, prior works @xcite @xcite have focused on evaluating the validity of the intermediate reasoning steps that lead to the final output. they evaluate the reasoning steps by identifying errors like hallucinations @xcite and logical flaws. yet, these approaches do not adequately measure how a model utilizes the essential knowledge required for problem-solving, making it difficult to provide interpretable feedback on model behavior. figure 1 exemplifies this limitation, showing how a model can apply partial knowledge correctly while overlooking other prerequisite steps. while it correctly applies combining fractions, it neglects necessary knowledge, such as simplifying a fraction and converting the result into a mixed number. to systematically assess how precisely a model utilizes essential knowledge, we propose an evalua-tion suite consisting of the following three components: we first construct (1) principal knowl-edge collection (pk collection), a largescale resource of atomic knowledge essential for solving the target tasks. this knowledge is collected from multiple top-performing llms, clustered into coherent chunks, and eventually refined into principal knowledge for each chunk. in particular, we built this collection based on the mmlu benchmark @xcite , resulting in 112k principal knowledge units (pk units). grounded in the pk collection, we introduce (2) novel evaluation metrics, including knowledgegrounded precision, recall, and f1 score. the llmbased evaluation process involves identifying the utilization of pk units within the predicted rationale. these metrics measure how precisely the model applies the knowledge and whether it recalls the necessary information accurately. furthermore, using proprietary llms for reasoning evaluation is costly and impractical at scale. to overcome this, we develop (3) a light-weight llm evaluator distilled from a proprietary teacher llm, achieving high agreement with reduced computational overhead. we also validate pk collec-tion for factuality and relevance, confirming that it maintains high quality in both aspects. to additionally explore the potential applications of our evaluation suite, we investigate its use in controlling reasoning conciseness with recent preference optimization techniques. following the approach of @xcite , we adopt a framework that applies direct preference optimization (dpo) @xcite . by selecting preferred samples based on our evaluation metrics, we observe that models could efficiently generate reasoning steps while maintaining end performance. this result demonstrates that aligning llm with our evaluation metrics effectively guides it to concisely reach the final result for solving the task. our experimental results and analysis demonstrate that our knowledge-grounded metrics effectively evaluate knowledge utilization and provide interpretable feedback on model behavior. moreover, reasoning preference optimization guided by our knowledge-grounded metric not only boosts performance but also enables control over the model's knowledge usage and token consumption. our contributions are summarized in threefold: ## evaluation suite for reasoning steps grounded on principal knowledge to assess the reasoning steps via knowledge grounding, we propose a novel evaluation suite: we first construct principal knowledge col-lection ( § 3.1), a large-scale resource of atomic knowledge essential for solving the target tasks. we also introduce novel evaluation metrics ( § 3.2) to measure the utilization of principal knowledge within the thinking process. to facilitate efficient evaluation, we further develop an open-weight llm evaluator ( § 3.3) by distilling a proprietary llm. figure 2 shows the overview of our evaluation suite construction and evaluation. ## aligning llm reasoning with knowledge-grounded metrics preliminary: direct preference optimization the preference optimization paradigm fine-tunes language models based on human or task-specific preferences to generate more desirable outputs. a prominent method, direct preference optimization (dpo) @xcite , trains a model π θ to prefer response y w over y l using the objective: iterative reasoning preference optimization (irpo) @xcite extends preferencebased fine-tuning to reasoning tasks by iteratively (1) sampling multiple cot and answer candidates per prompt, and (2) training the model to prefer generations with correct cot over incorrect ones. reasoning preference optimization via knowledge-grounded metrics we further extend the utilization of our evaluation suite by integrating it into irpo. as we defined, high knowledge precision (kp) indicates that a model correctly uses the required knowledge and minimizes hallucinations, while a high knowledge recall (kr) means that the model conducts an extensive exploration of the relevant knowledge. however, a correct final answer does not have to be explored through every piece of principal knowledge: sometimes more concise reasoning can suffice. building on these insights, we incorporate kr into standard preference optimization techniques, enabling us to tune the depth of a model's reasoning-from minimal knowledge usage to full coverage-based on specific goals such as efficiency or thoroughness. our approach begins by prompting llms to generate multiple cot rationales for each question. these rationales are then categorized as preferred or dispreferred based on specific selection criteria according to the baselines discussed in the following paragraphs. to improve preference selection, we explore several variants by using several different strategies to build preference pairs: answer-driven selection a simple baseline where any rationale that successfully arrives at the correct final answer is randomly designated as a preferred sample. incorrect rationales are dispreferred, following conventional irpo selection. ## experiments in this section, we present two key experiments. first, we evaluate how effectively llms apply knowledge in their reasoning using the pk col-lection and our proposed knowledge-grounded metrics. then, we examine the impact of our knowledge-grounded in reasoning alignment, demonstrating how controlling kr influences the model's reasoning process. ## example and case study this section presents illustrative example and case study to demonstrate the application of our evaluation framework. ## analysis are the principal knowledge in the pk col-lection factual and relevant? to validate the factuality of the pk collection, we employed gpt-4o with openai's websearch tool api to conduct web-based fact-checking. we performed binary classification (true/false) for each of 4k pk units across 512 mmlu questions. this process yielded an overall accuracy of 94.1%. among the 232 pk units marked as false, we manually reviewed a random sample of 100. as shown in table f , only 33% were genuinely factually incorrect, while the remaining cases were due to various factors such as ambiguity, model deficiency, or overly strict judgments. this suggests that the actual number of incorrect pk units is much smaller, and that the pk collection constitutes a highly accurate set. for relevance, we adopted a scoring rubric inspired by the prometheus approach @xcite , rating the relevance of each pk unit to its corresponding question on a 1-5 scale. using gpt-4o, we assessed the same 4k pk units, yielding an average relevance score of 4.0. together, these validations confirm that the pk collection maintains high quality in terms of both factual accuracy and relevance. 7 does providing misused or unused knowledge improve accuracy? we investigate the quality of principal knowledge in our collection by incorporating it in the prompt, adopting llama3-8b-instruct. we pick error cases from the benchmark and prompt llm to answer the questions based on one of four types of knowledge elements: (1) random-a randomly chosen knowledge element, (2) correct-knowledge that was correctly used in the original reasoning, (3) incorrect-knowledge that was previously misapplied, or (4) all-knowledge that was either misapplied or not utilized (unapplied) in the model's original reasoning. figure @xcite shows cases where all applied knowledge was correct but some required knowledge was 10% missing (@xmath0, @xmath1. as the chart shows, providing these unapplied knowledge elements significantly boosts accuracy. this suggests that while the model's reasoning was partially correct, it did not fully leverage sufficient relevant knowledge; once the missing knowledge is provided, the model can arrive at the correct answer. figure @xcite illustrates the cases where some knowledge in its original reasoning was incorrectly used (@xmath2. under these conditions, giving the misapplied knowledge can rectify the model's misconception, substantially raising the accuracy. when both unapplied and misapplied knowledge are provided, we observe accuracy surging to 66.2%, indicating that addressing sufficient and correct knowledge usage can transform previously erroneous solutions into correct ones. these results highlight how our evaluation suite effectively diagnoses model weaknesses, revealing whether failures arise from missing or misapplied knowledge. furthermore, it provides insights into targeted interventions, guiding strategies to improve the model's reasoning by supplementing critical knowledge gaps. ## conclusion we introduce a knowledge-grounded evaluation suite that assesses llms' ability to accurately recall and apply essential knowledge in reasoning. by constructing the principal knowledge collection, we establish a large-scale resource of atomic knowledge, which serves as the foundation for our knowledge-grounded metrics-precision, recall, and f1-measuring both correctness and completeness in reasoning. our lightweight evaluator llm replicates a proprietary teacher model's assessment while reducing computational costs. experimental results show that integrating these metrics into preference optimization techniques enhances model performance, enabling controlled reasoning efficiency. these findings highlight the effectiveness of our evaluation suite in improving the transparency and reliability of llm reasoning.