R e CUT : Balancing Reasoning Length and Accuracy in LLM s via Stepwise Trails and Preference Optimization (2025)

Abstract: Recent advances in Chain-of-Thought (CoT) prompting have substantially improved the reasoning capabilities of Large Language Models (LLMs). However, these methods often suffer from overthinking, leading to unnecessarily lengthy or redundant reasoning traces. Existing approaches attempt to mitigate this issue through curating multiple reasoning chains for training LLMs, but their effectiveness is often constrained by the quality of the generated data and prone to overfitting. To address the challenge, we propose Reasoning Compression Through Stepwise Trials (ReCUT), a novel method aimed at balancing the accuracy and length of reasoning trajectory. Specifically, ReCUT employs a stepwise exploration mechanism and a long-short switched sampling strategy, enabling LLMs to incrementally generate diverse reasoning paths. These paths are evaluated and used to construct preference pairs to train two specialized models (Gemini LLMs)—one optimized for reasoning accuracy, the other for shorter reasoning. A final integrated model is obtained by interpolating the parameters of these two models. Experimental results across multiple math reasoning datasets and backbone models demonstrate that ReCUT significantly reduces reasoning lengths by approximately 30-50%, while maintaining or improving reasoning accuracy compared to various baselines. All codes and data will be released via https://github.com/NEUIR/ReCUT.

## introduction recent advances in chain-of-thought (cot) @xcite have substantially improved the reasoning capabilities of large language models (llms), motivating researchers to explore a new scaling paradigm-test-time scaling @xcite . this paradigm improves the performance of llms on lots of * indicates equal contribution. to address the inefficiencies caused by overthinking, various strategies have been proposed. prompt-based methods aim to guide llms to generate more concise reasoning trajectories. sftbased methods prompt llms to sample multiple reasoning trajectories and select the concise and correct ones to synthesize sft data for fine-tuning itself @xcite . however, the former may cause the llms to omit critical intermediate steps @xcite , while the latter may lead to overfitting to the training sig-nals @xcite . reinforcement learning (rl)-based methods represent another promising research direction @xcite , where carefully designed reward functions penalize overly long reasoning trajectories and inaccurate outputs, thereby guiding llms to produce concise and accurate reasoning results. existing rl-based methods typically sample multiple complete reasoning trajectories using a single instruction to compute their rewards and construct preference data for training. however, this sampling strategy inherently limits the diversity of reasoning trajectories @xcite , resulting in convergent distributions in terms of both length and accuracy. furthermore, such approaches often overlook the fine-grained contributions of individual reasoning steps in reasoning trajectories, allowing even the correct trajectories to include redundant steps @xcite . these limitations compromise the quality of the constructed preference data, ultimately resulting in suboptimal reasoning performance, which motivates us to explore a better method for constructing high-quality preference data. recent methods that search for optimal reasoning paths by stepwise decoding have achieved advantages in reducing reasoning errors and redundancies generated by llms @xcite . this success inspires us to construct highquality and diverse preference data by prompting the model to stepwise explore reasoning paths. in this paper, we propose recut (reasoning compression through stepwise trials), a method that guides llms to progressively explore diverse reasoning trajectories and optimizes them via preference-based learning to balance reasoning accuracy and length. as shown in figure 1 , recut introduces a stepwise reasoning trajectory exploration mechanism: at each step, the llm is conditioned on a partially optimal trajectory and generates all subsequent reasoning steps using a long-short sampling strategy to encourage diversity. all trajectories produced via stepwise sampling are incorporated with the given optimal trajectory and then collected into a candidate pool. then the first step of the generated reasoning outcome is further evaluated to incrementally construct the optimal reasoning path. after trajectory collection, recut leverages this candidate pool to train two specialized llms (gemini llms)-one favoring accuracy and the other favoring shorter reasoning results. finally, recut performs parameter interpolation between these two optimized llms to achieve a trade-off between the reasoning accuracy and the length of reasoning results. our experiments demonstrate the effectiveness of recut, which significantly reduces the reasoning length, typically by 30-50%, while maintaining or even surpassing the accuracy of baseline methods. further analysis reveals that our stepwise sampling strategy enables the synthesis of diverse reasoning trajectories, helping llms regulate the number of reasoning steps through preference optimization. by leveraging parameter interpolation, recut not only achieves higher accuracy but also further shortens the reasoning process by combining the strengths of both gemini llms. moreover, recut effectively mitigates overthinking in llms by reducing redundant reasoning steps and identifying unproductive reasoning paths that may lead to incorrect answers. ## related work the test-time scaling law suggests that llms, such as deepseek-r1 @xcite and qwq @xcite , can enhance their performance on challenging reasoning and mathematical tasks @xcite by engaging in deeper reasoning @xcite and producing longer chains-of-thought (cot) @xcite . while effective, these approaches often cause llms to overthink @xcite , resulting in redundant or irrelevant content @xcite , which not only increases inference cost but may also lead to incorrect answers @xcite . to better balance reasoning efficiency and accuracy, recent work has explored efficient reasoning with llms, with a particular focus on reducing and controlling the length of reasoning chains @xcite @xcite . some approaches design specialized prompts to directly shorten the generated responses @xcite . however, such strategies may truncate essential intermediate reasoning steps, ultimately compromising accuracy @xcite . to address this problem, @xcite ; @xcite leverage advanced llms (e.g., @xcite ) to generate concise yet accurate cots as training data for supervised fine-tuning (sft). nevertheless, this method is inherently limited by the capabilities of the teacher models and may lead to overfitting to the provided training signals. benefiting from advances in reinforcement learning @xcite , recent studies have leveraged it to enable llms to adaptively control the length of their reasoning processes. @xcite introduce a length bias penalty and an accuracy reward during training to encourage llms to generate accurate reasoning using fewer tokens. @xcite propose a cosine reward mechanism that promotes reasoning trajectories of moderate length by penalizing both overly short and overly long generations. however, these approaches compute rewards based on entire reasoning trajectories, without estimating the contribution of individual steps in the whole reasoning chain. in contrast, recut leverages a step-wise sampling strategy to elicit more finegrained and diverse preference data from llms, which enhances the training process for reasoning compression. ## methodology this section presents reasoning compression through stepwise trials (recut), a method designed to reduce the reasoning length of large language models (llms) while maintaining comparable performance. as shown in figure 2 , we first introduce the long-short switch sampling strategy, which adaptively constructs a reasoning trajectory pool c q containing trajectories of varying lengths (sec. 3.1). then, we leverage these trajectories to train multiple gemini-llms with complementary strengths and interpolate their parameters to balance reasoning accuracy and length (sec. 3.2). ## experimental methodology this section first describes the datasets, evaluation metrics, and baselines, followed by the implementation details of our experiments. ## evaluation results in this section, we first present the overall performance of recut, followed by ablation studies to examine the contributions of its components. we then analyze the effectiveness of recut in reasoning compression. finally, we provide case studies for further illustration. ## conclusion in this paper, we introduce recut (reasoning compression through stepwise trials), an effective method designed to optimize llms to mitigate the overthinking issue. recut introduces a stepwise reasoning trajectory exploration mechanism to construct a more diverse reinforcement learning training dataset and train gemini llms to balance accuracy and length via parameter interpolation. our experimental analysis reveals that recut achieves considerable improvements in token efficiency and maintains or enhances accuracy across different difficulty levels of math tasks and various backbone models.