Stepwise Perplexity-Guided Refinement for Efficient Chain-of-Thought Reasoning in Large Language Models (2025)

Abstract: Chain-of-Thought (CoT) reasoning, which breaks down complex tasks into intermediate reasoning steps, has significantly enhanced the performance of large language models (LLMs) on challenging tasks. However, the detailed reasoning process in CoT often incurs long generation times and high computational costs, partly due to the inclusion of unnecessary steps. To address this, we propose a method to identify critical reasoning steps using perplexity as a measure of their importance: a step is deemed critical if its removal causes a significant increase in perplexity. Our method enables models to focus solely on generating these critical steps. This can be achieved through two approaches: refining demonstration examples in few-shot CoT or fine-tuning the model using selected examples that include only critical steps. Comprehensive experiments validate the effectiveness of our method, which achieves a better balance between the reasoning accuracy and efficiency of CoT.

## introduction large language models (llms) are powerful generative models capable of performing diverse tasks in different domains @xcite @xcite and demonstrating strong reasoning capabilities @xcite . recent advancements, such as few-shot/zero-shot chain-of-thought (cot) @xcite , as well as finetuning @xcite , have significantly enhanced the llms' reasoning capabilities by leveraging intermediate reasoning steps. in particular, through few-shot cot, llms can learn from the reasoning steps in the demonstration examples and apply similar reasoning patterns to target tasks. in the case of zero-shot cot, llms are prompted to"think step by step" to generate reasoning steps. ## preliminary in this section, we first present the essentials of perplexity, and then introduce our exploration on how to use perplexity to analyze the reasoning steps. ## the proposed algorithm -spirit in this section, we present the details of spirit. since few-shot cot and fine-tuning utilize data in different ways, we first provide the general idea in section 3.1 and then describe case-specific details in section 3.2 (few-shot cot, spirit-fs) and 3.3 (fine-tuning, spirit-ft), respectively. ## experiment in this section, we conduct comprehensive experiments to demonstrate the effectiveness of spirit. we present the results of spirit-fs in section 4.1 and demonstrate the performance of spirit-ft in section 4.2. both sections include the discussion and stopping, in the fine-tuning scenario, to handle the large amount of fine-tuning data and the diversity of the reasoning steps among the data, we explicitly design the merging and stopping criteria for spirit-ft. on the transferability of spirit by investigating whether the reasoning step selection process generalizes across different models. due to page limit, we postpone the ablation studies in appendix a, where we examine the impact of some key components in the design of spirit-ft. ## case study for spirit-ft in this section, we provide case studies for more details about the types of reasoning steps identified as redundant by spirit-ft foot_0 . recall from algorithm 2 that, based on the perplexity, certain reasoning steps are removed entirely, while others are merged with other steps to maintain coherence. although different models are used to compute perplexity, we observe that the patterns of step removal and merging remain consistent across models, including llama3 and qwen2.5. due to page limit, besides the following general description, we postpone the detailed quantitative analysis to appendix g. ## conclusion in this paper, we introduce spirit, a method for refining reasoning steps in few-shot cot and cot fine-tuning for improving reasoning efficiency while maintaining accuracy. based on the observation that changes in perplexity correlate with reasoning step importance, spirit works by iteratively identifying unimportant steps through evaluating the change in perplexity, then merge the unimportant steps. experiments demonstrate the effectiveness of spirit in improving the trade-off between accuracy and efficiency in both few-shot cot and cot in fine-tuning.