Verbosity-Aware Rationale Reduction: Sentence-Level Rationale Reduction for Efficient and Effective Reasoning (2025)

Abstract: Large Language Models (LLMs) rely on generating extensive intermediate reasoning units (e.g., tokens, sentences) to enhance final answer quality across a wide range of complex tasks. While this approach has proven effective, it inevitably increases substantial inference costs. Previous methods adopting token-level reduction without clear criteria result in poor performance compared to models trained with complete rationale. To address this challenge, we propose a novel sentence-level rationale reduction framework leveraging likelihood-based criteria, *verbosity*, to identify and remove redundant reasoning sentences. Unlike previous approaches, our method leverages *verbosity* to selectively remove redundant reasoning sentences while preserving reasoning capabilities. Our experimental results across various reasoning tasks demonstrate that our method improves performance by an average of 7.71% while reducing token generation by 19.87% compared to model trained with complete reasoning paths.

## introduction recent advances in large language models (llms) have demonstrated remarkable reasoning capabilities comparable to human cognitive abilities @xcite @xcite . these works demonstrate the capability to solve complex reasoning tasks through explicitly generating extended reasoning paths. the generation of such paths involves producing explicit reasoning units (e.g., tokens, steps) @xcite , which further enhances model performance through iterative prompting @xcite . through this iterative generation of explicit reasoning paths, the model refines and expands its thought processes while incorporating strategic planning and continuous cognitive generation @xcite . while the extensive generation of explicit reasoning units leads to improved performance, it inevitably results in higher inference costs and increased latency @xcite . furthermore, fine-tuning llms with complete reasoning paths does not necessarily guarantee enhanced performance @xcite @xcite , indicating the necessity for methods that maintain reasoning capabilities while reducing the generation of reasoning units. despite this apparent requirement, it remains underexplored how to maintain llm reasoning capabilities while reducing intermediate reasoning paths across diverse tasks. previous methods primarily focused on reducing reasoning paths from two distinct perspectives. some studies have proposed training pipelines that leverage augmented datasets, iteratively generated by foundation llms, to fine-tune subsequent llms @xcite . however, these approaches remain inherently vulnerable due to their significant dependence on the generative capabilities of llms. in response, other works have focused on directly training llms without dataset augmentation to reduce explicit reasoning paths. @xcite introduced a knowledge distillation method to distill explicit reasoning into implicit reasoning through token-level hidden states. @xcite adopted tokens as the reasoning unit for reduction and proposed a heuristic method to internalize explicit intermediate rationale tokens while @xcite compressed complete rationales into the predefined number of hidden states of tokens. however, their methods present a fundamental limitation as they lack sufficient justification for selecting tokens over more linguistically natural units (e.g., sentences) for reduction ( to address these limitations, we propose a novel training method that maintains llm reasoning performance while systematically reducing redundant reasoning units within the reasoning process. our method adopts sentences as fundamental reduction units, establishing more linguistically meaningful boundaries compared to token-level approaches. through empirical analysis, we demonstrate that sentences in early rationale steps can introduce redundancy in the llm's answer generation process. inspired by @xcite , we introduce the concept of 'verbosity', a likelihood-based criteria, to identify redundant reasoning sentences. by incorporating verbosity identification into the training process, the model excludes redundant reasoning sentences, thereby reducing intermediate token generation. finally, we demonstrate our method's effectiveness and generalizability across various real-world reasoning datasets, showing our method improves performance by an average of 7.71% while reducing token generation by 19.87% across various llms, and through systematic ablation studies, we analyze the contribution of each proposed component. ## verbosity-aware rationale reduction based on these observations, we propose the verbosity-aware rationale reduction (varr) framework. in section 4.1, we introduce the concept of 'verbosity' as a principled criterion for identifying redundant reasoning sentences. section 4.2 elaborates on how we integrate verbosity into the reduction process during cot training. in section 4.3, we extend the verbosity term by incorporating incorrect answers to enhance robustness. finally, section 4.4 presents the comprehensive varr framework. ## conclusion in this work, we propose the sentence-level rationale reduction framework varr and empirically demonstrate that models trained with nonredundant rationales achieve enhanced efficiency. we address the lack of principled criteria for identifying redundant sentences during training by developing a reduction framework that not only preserves the model's reasoning capabilities but also reduces the likelihood of generating incorrect answers. our experiments show that varr can efficiently handle a diverse range of tasks with fewer generated tokens, without sacrificing its accuracy. this work contributes novel insights to rationale reduction research, contributing to the efficient reasoning elicitation in language models.