Verbosity-Aware Rationale Reduction: Sentence-Level Rationale Reduction for Efficient and Effective Reasoning
Authors: Joonwon Jang, Jaehee Kim, WONBIN KWEON, Seonghyeon Lee, Hwanjo Yu
Accepted at: acl25

Verbosity-Aware Rationale Reduction:
Sentence-Level Rationale Reduction for Efficient and Effective Reasoning

Joonwon Jang1,5  Jaehee Kim2  Wonbin Kweon3  Seonghyeon Lee4  Hwanjo Yu1,*
POSTECH1, Seoul National University2, University of Illinois Urbana-Champaign3
Kyungpook National University4, LG AI Research5
{kaoara, hwanjoyu}@postech.ac.kr jaehhee_kim@snu.ac.kr
wonbin@illinois.edu sh0416@knu.ac.kr

Abstract
Large Language Models (LLMs) rely on generating extensive intermediate reasoning units (e.g., tokens, sentences) to enhance final answer quality across a wide range of complex tasks. While this approach has proven effective, it inevitably increases substantial inference costs. Previous methods adopting token-level reduction without clear criteria result in poor performance compared to models trained with complete rationale. To address this challenge, we propose a novel sentence-level rationale reduction framework leveraging likelihood-based criteria. verbosity, to identify and remove redundant reasoning sentences. Unlike previous approaches, our method leverages verbosity to selectively remove redundant reasoning sentences while preserving reasoning capabilities. Our experimental results across various reasoning tasks demonstrate that our method improves performance by an average of 7.71% while reducing token generation by 19.87% compared to model trained with complete reasoning paths.

1 Introduction
Recent advances in Large Language Models (LLMs) have demonstrated remarkable reasoning capabilities comparable to human cognitive abilities (Madaan et al., 2024; Shinn et al., 2024; Kumar et al., 2024). These works demonstrate the capability to solve complex reasoning tasks through explicitly generating extended reasoning paths. The generation of such paths involves producing explicit reasoning units (e.g., tokens, steps) (Yu et al., 2024b), which further enhances model performance through iterative prompting (Wang et al., 2023; Yao et al., 2023). Through this iterative generation of explicit reasoning paths, the model refines and expands its thought processes while incorporating strategic planning and continuous cognitive generation (Xi et al., 2023; Yang et al., 2024).

While the extensive generation of explicit reasoning units leads to improved performance, it inevitably results in higher inference costs and increased latency (Yu et al., 2024b; Wang et al., 2024). Furthermore, fine-tuning LLMs with complete reasoning paths does not necessarily guarantee enhanced performance (Yu et al., 2024b; Deng et al., 2024b; Liu et al., 2024), indicating the necessity for methods that maintain reasoning capabilities while reducing the generation of reasoning units. Despite this apparent requirement, it remains underexplored how to maintain LLM reasoning capabilities while reducing intermediate reasoning paths across diverse tasks.

Previous methods primarily focused on reducing reasoning paths from two distinct perspectives. Some studies have proposed training pipelines that leverage augmented datasets, iteratively generated by foundation LLMs, to fine-tune subsequent LLMs (Yu et al., 2024b; Liu et al., 2024). However, these approaches remain inherently vulnerable due to their significant dependence on the generative capabilities of LLMs.

In response, other works have focused on directly training LLMs without dataset augmentation to reduce explicit reasoning paths. Deng et al. (2023) introduced a knowledge distillation method to distill explicit reasoning into implicit reasoning through token-level hidden states. Deng et al. (2024b) adopted tokens as the reasoning unit for reduction and proposed a heuristic method to internalize explicit intermediate rationale tokens while Hao et al. (2024) compressed complete rationales into the predefined number of hidden states of tokens. However, their methods present a fundamental limitation as they lack sufficient justification for selecting tokens over more linguistically natural units (e.g., sentences) for reduction (Table 1), and they fail to provide principled criteria for the removal process. Moreover, their evaluation has primarily focused on synthetic arithmetic reason-
ing tasks, limiting their applicability to real-world scenarios.

To address these limitations, we propose a novel training method that maintains LLM reasoning performance while systematically reducing redundant reasoning units within the reasoning process. Our method adopts sentences as fundamental reduction units, establishing more linguistically meaningful boundaries compared to token-level approaches. Through empirical analysis, we demonstrate that sentences in early rationale steps can introduce redundancy in the LLM’s answer generation process. Inspired by Dong et al. (2023), we introduce the concept of ‘verbosity’, a likelihood-based criteria, to identify redundant reasoning sentences. By incorporating verbosity identification into the training process, the model excludes redundant reasoning sentences, thereby reducing intermediate token generation. Finally, we demonstrate our method’s effectiveness and generalizability across various real-world reasoning datasets, showing our method improves performance by an average of 7.71% while reducing token generation by 19.87% across various LLMs, and through systematic ablation studies, we analyze the contribution of each proposed component.

2 Related Works

2.1 Performance-Cost Tradeoffs in Reasoning Path Generation

Recent research has demonstrated the critical role of generating iterative and refined reasoning paths in enhancing model reasoning capabilities, albeit at increased computational costs (Wang et al., 2023; Yao et al., 2023; Radha et al., 2024; Wang et al., 2024; Madaan et al., 2024; Shinn et al., 2024; Kumar et al., 2024). Self-Consistency (Wang et al., 2023), Tree of Thoughts (ToT) (Yao et al., 2023), and Strategic Chain of Thought (SCoT) (Wang et al., 2024) improve reasoning accuracy through ensemble-based path selection, tree-structured exploration, and adaptive reasoning with an Inner Dialogue Agent, where each approach requires iterative reasoning path generation, resulting in substantial computational overhead.

Concurrently Self-Refine (Madaan et al., 2024) and Reflexion-based framework (Shinn et al., 2024; Kumar et al., 2024) enhance reasoning abilities through iterative feedback-based refinement and reflective path generation, respectively, though both require multiple forward passes through the model.

While the iterative generation and refinement of reasoning paths are essential for achieving optimal performance, they inherently increase inference costs and latency. Therefore, it is crucial to investigate methods for efficiently generating these paths.

2.2 Reasoning Path Reduction

To address the computational costs associated with extensive reasoning paths generation, some lines of work (Yu et al., 2024b; Liu et al., 2024) have focused on generating augmented datasets with varying rationale lengths to reduce the generation of reasoning paths. Yu et al. (2024b) employs Self-Consistency to generate multiple reasoning paths for dataset augmentation, then fine-tunes the model to produce direct answers. Liu et al. (2024) developed a heuristic approach to merge reasoning steps and iteratively trained the model to produce shorter reasoning paths, which are then integrated into the progressive training phase. While these approaches demonstrate empirical effectiveness, they exhibit two fundamental limitations: (1) their substantial dependence on LLM generation capabilities introduces inherent instability, and (2) their objective of reducing reasoning paths necessitates the paradoxical creation of datasets requiring extensive reasoning path generation.

To address these weaknesses, another line of work (Deng et al., 2023, 2024b; Hao et al., 2024) has focused on directly training LLMs without augmented datasets. Implicit-CoT (Deng et al., 2023) implements a multi-model framework where an emulator model is trained to predict the teacher’s token-level hidden states, and a student model leverages these predicted states to generate answers. ICoT-SI (Deng et al., 2024b) identifies tokens as reduction units, proposing a method to internalize explicit intermediate rationale tokens by progressively eliminating them from the beginning of the reasoning path within the CoT fine-tuning process. However, these methods demonstrate limited generalization across diverse datasets as they have been validated exclusively on simple arithmetic reasoning tasks, such as multiplication problems. This limitation raises concerns about their applicability to real-world scenarios where rationales are expressed in natural language. Furthermore, they do not explore the adoption of principled criteria and linguistically natural units (e.g., sentences). Specifically, the token-level reduction approach may eliminate critical information necessary for answer gen-
eration or distort the semantic information of the sentence. Motivated by these limitations in existing reduction approaches, we examine the redundancy of various sentence positions for potential elimination and propose a novel method with principled criteria that can effectively reduce reasoning paths while maintaining their efficacy.

3 Early Step Rationales are Redundant

![Line plot showing NLL differences across varying sizes of {r_i}. The 'original' represents the NLL with the complete rationale, while 'front', 'random', and 'back' indicate that {r_i} is sampled from the front, random, and back indices of the full index set, respectively.](page_184_563_579_377.png)

Figure 1: NLL differences across varying sizes of {r_i}. The 'original' represents the NLL with the complete rationale, while 'front', 'random', and 'back' indicate that {r_i} is sampled from the front, random, and back indices of the full index set, respectively.

3.1 Quantifying the Redundancy

Before delving into the method, we first investigate which positions within the rationale sentences should be selected for reduction. When the likelihood of the answer remains unchanged after removing a sentence from the full rationales, this indicates that the sentence may be redundant in the reasoning process. To quantify the redundancy of a sentence, we compute the negative log-likelihood (NLL) for answer \( y \) after sentence reduction as follows:

\[
\text{NLL} = -\log p_\theta(y|R', x),
\]
where \( R' = R \setminus \{r_i\}_{i \in S},\ S \subseteq I. \tag{1} \]

Let \( R \) denote the complete set of rationale sentences, and \( I \) represent the full index set of these sentences. The subset of indices corresponding to sentences selected for reduction is denoted by \( S \), and \( R' \) represents the remaining rationale sentences after their removal. \( \{r_i\}_{i \in S} \) denotes the sentences selected for reduction. For simplicity, we use \( \{r_i\} \) without the subset index notation throughout the rest of the paper.

3.2 Redundancy of Early Reasoning Sentences

We performed a pilot study to empirically demonstrate the redundancy of leading sentences within the rationales by analyzing the NLL of Mistral 7B (Jiang et al., 2023) across diverse reasoning datasets. Specifically, we varied the size of \( \{r_i\} \) from 1 to 4 and investigated general patterns of sentence removal using a stochastic approach. To compare different sentence selection configurations, we considered three sampling methods for \( \{r_i\} \): front, where initial sentences were prioritized; random, with uniform probabilities; and back, where probabilities progressively increased for later sentences\footnote{For generating \( R' \), we assign probabilities \( p_k = \frac{N-k+1}{\sum_{i=1}^N i} \) (front), \( \frac{1}{N} \) (random), and \( \frac{k}{\sum_{i=1}^N i} \) (back) where k=1,...,N denotes sentence position.}. Additionally, we computed the NLL for complete rationale sentences (i.e., \( -\log p_\theta(y|R, x) \)) as a baseline to evaluate the impact of reduction. As illustrated in Figure 1, the front (<) configuration shows only marginal NLL differences relative to complete rationale sentences. In contrast, removing sentences randomly (>) or from the back (>) results in higher NLL as the removed sentences increase, highlighting the importance of the selection of a candidate rationale position strategy for removal in the reasoning and answer prediction process (for additional analysis, see Appendix A).

4 Verbosity-Aware Rationale Reduction

Based on these observations, we propose the Verbosity-Aware Rationale Reduction (VARR) framework. In Section 4.1, we introduce the concept of ‘verbosity’ as a principled criterion for identifying redundant reasoning sentences. Section 4.2 elaborates on how we integrate verbosity into the reduction process during CoT training. In Section 4.3, we extend the verbosity term by incorporating incorrect answers to enhance robustness. Finally, Section 4.4 presents the comprehensive VARR framework.

4.1 Verbosity as Principled Criterion

To quantify the redundancy of sentences for potential removal, we introduce the fundamental concept ‘verbosity’. Given an input \( x \), full rationale \( R \), and
Figure 2: Overview of our VARR/VARR+ framework. Initially, we select a candidate sentence from the beginning of the rationale (Section 3). After selecting the candidate sentence, we evaluate Equations (5) and (9) by calculating \( verbosity(y_g) \) and \( verbosity(y_w) \). If the candidate sentence meets the verbosity evaluation criteria, it is excluded in subsequent training steps. The model then proceeds with training, where the redundant sentence is excluded from the rationale.

a reduced rationale \( R' = \{ r_j \}_{j \in I \setminus \{i\}} \), we quantify the *verbosity* of a sentence \( r_i \) on \( y \) by computing the difference in Kullback–Leibler divergence (KL-divergence) as follows:

\[
verbosity(y) = D_{KL}(q(y|x) \parallel p_\theta(y|R, x)) \\
- D_{KL}(q(y|x) \parallel p_\theta(y|R', x)),
\]

where \( q(y|x) \) is the ground truth distribution. The \( verbosity(y) \) measures the informational contribution or redundancy of a rationale sentence \( r_i \) with respect to answer \( y \). Since \( q(y|x) \) is the form of the one-hot vector (i.e., Dirac delta function), we can express the \( verbosity(y) \) as the log-likelihood ratio between \( R \) and \( R' \) as follows:

\[
verbosity(y_g) = [H_q(p_\theta(y|R, x)) - H(q(y|x))] \\
- [H_q(p_\theta(y|R', x)) - H(q(y|x))] \\
= E_q[-\log p_\theta(y|R, x)] \\
+ E_q[\log p_\theta(y|R', x)] \\
= \log \left( \frac{p_\theta(y_g|R', x)}{p_\theta(y_g|R, x)} \right),
\]

where \( y_g \) denotes the ground truth answer (i.e., \( q(y_g|x) = 1 \)). \( H_q(\cdot) \) and \( H(\cdot) \) denote the cross-entropy and the entropy, respectively\footnote{For the sake of explainability, we assume each expression’s \( y \) is represented by a single token.}. Intuitively, a higher value of \( verbosity(y_g) \) implies that the likelihood of the model generating the ground truth answer increases after removing \( r_i \), indicating that its removal is beneficial.

4.2 Verbosity Identification in CoT Training

Given an input sequence, CoT training (Nye et al., 2021) aims to train LLMs to generate complete rationale, followed by the ground truth answer:

\[
-\log p_\theta(y_g, R|x).
\]

During each training step \( t \), we evaluate each sentence \( r_i \) within \( R \) using the following criterion:

\[
verbosity(y_g) \geq 0.
\]

Here, we sequentially select \( r_i \) starting from the first sentence and construct \( R' = \{ r_j \}_{j \in I \setminus \{i\}} \), based on our analysis in Section 3, which indicates that early-stage rationales are more likely to be redundant. When the verbosity score is non-positive,
it indicates that removing \( r_i \) from \( R \) would impair the model’s performance, thus identifying the sentence as essential and preserving it in the rationale. Detailed training procedures will be described in Section 4.4.

4.3 Contrasting with Wrong Answer

Inspired by the miscalibrated log-likelihoods between accepted and rejected responses in standalone Supervised Fine-Tuning (SFT) for alignment learning (Rafailov et al., 2024; Azar et al., 2024; Hong et al., 2024), we examine whether reduced rationales \( R' \) lead to inaccurate answer generation by incorporating a wrong answer \( y_w \).

Instead of employing the correct answer distribution \( q(y|x) \) in Equation (2), we initiate our formula with the wrong answer distribution \( q' \) (i.e., \( 1 - q(y|x) \), normalized sum to 1). Through algebraic manipulation, we derive:

\[
D_{KL}(q'(y|x) \parallel p_\theta(y|R, x)) \\
\quad - D_{KL}(q'(y|x) \parallel p_\theta(y|R', x)) \\
= \left[ H_w(p_\theta(y|R, x)) - H(q'(y|x)) \right] \\
\quad - \left[ H_w(p_\theta(y|R', x)) - H(q'(y|x)) \right],
\]

where \( H_w(\cdot) \) denotes the cross-entropy calculated with the wrong answer distribution \( q' \). Due to the impracticality of computing the expectation over the entire space of \( V - 1 \) wrong answers (where \( V \) is the vocabulary size), we sample \( K \) incorrect answers for the following estimations:

\[
E_w[-\log p_\theta(y|R, x)] + E_w[\log p_\theta(y|R', x)] \\
= E_w \left[ \log \frac{p_\theta(y|R', x)}{p_\theta(y|R, x)} \right] \\
\approx \frac{1}{K} \sum_{k=1}^K \log \frac{p_\theta(y_w^{(k)}|R', x)}{p_\theta(y_w^{(k)}|R, x)},
\]

where \( \{y_w^{(k)}\}_{k \in [K]} \) is sampled from the in-batch negatives depending on the dataset. Consequently, \( verbosity(y_w) \) is computed as:

\[
verbosity(y_w) = \frac{1}{K} \sum_{k=1}^K \log \frac{p_\theta(y_w^{(k)}|R', x)}{p_\theta(y_w^{(k)}|R, x)}.
\]

Since computational constraints necessitate sampling incorrect answers to calculate \( verbosity(y_w) \), we evaluate the effectiveness of removal by comparing \( verbosity(y_w) \) against \( verbosity(y_g) \) rather than solely using \( verbosity(y_w) \) as follows:

\[
verbosity(y_w) - verbosity(y_g) \leq 0.
\]

When both conditions \( verbosity(y_g) \geq 0 \) and \( verbosity(y_w) - verbosity(y_g) \leq 0 \) are satisfied, it indicates that removing \( r_i \) from \( R \) not only improves the model’s performance but also increases its preference for the ground truth answer over incorrect answers, supporting the removal of \( r_i \).

4.4 CoT Training with Rationale Reduction

In our framework, the model is trained using Equation (4) for predefined warm-up stage to inject its reasoning capabilities. Subsequently, at each training step \( t \), we evaluate each sentence \( r_i \) sequentially from the first sentence in \( R \), using either Equation (5) alone (denoted as VARR) or the combination of Equations (5) and (9) (denoted as VARR+). Sentences that satisfy these respective criteria are identified for removal and excluded from subsequent training steps. The maximum removable number of sentences at each training step \( t \) is determined based on a linear schedule, adopting ICoT-SI (Deng et al., 2024b)’s setting as follows:

\[
r(t) = \lfloor N_t \cdot (t/T) \rfloor,
\]

where \( T \) represents the total number of training steps, \( N_t \) is the total number of rationale sentences at step \( t \), and \( r(t) \) indicates the maximum number of sentences that can be removed at that step. Note that unlike ICoT-SI, which enforce removes a predefined number of tokens during training, our method preserves the essential reasoning steps by employing principled removal criteria.

5 Experiments

5.1 Training Configuration

Datasets We conducted experiments across two categories to provide a comprehensive evaluation of VARR’s versatility and effectiveness, in contrast to prior research that predominantly focuses on simple arithmetic tasks like multi-digit multiplication (Deng et al., 2023, 2024b). Initially, we evaluate with arithmetic reasoning tasks, including datasets like MathQA (MQA; Amini et al. 2019a) and GSM8K (G8K; Cobbe et al. 2021a). We also examine the performance of our method on commonsense reasoning tasks, employing datasets including CommonsenseQA (CQA; Talmor et al. 2019), TriviaQA (TQA; Joshi et al. 2017), and StrategyQA (SQA; Geva et al. 2021). Note that
Figure 3: Pareto plot of accuracy versus the number of generated tokens. The gray dotted lines connect the Pareto frontiers of the baselines, and our VARR (or VARR+) consistently outperforms the pareto frontiers across all subplots. While ICoT-SI and Coconut substantially trade-off accuracy for efficiency, VARR/VARR+ maintains high accuracy while reducing generated tokens, demonstrating its superior efficiency-performance balance.

unlike previous works (Deng et al., 2023, 2024b), we do not synthesize training data (especially the intermediate steps) to validate the generalizability and applicability of our method.

Models We trained Mistral 7B (Jiang et al., 2023) as our base model for comparisons. We also trained a series of Llama3.2 models (AI@Meta, 2024) scaling from 1B to 3B to demonstrate our method’s generalization capabilities.

Implementation Details The warm-up stage is set to 0.1 of the total training steps, and its impact is analyzed in Section 5.5.3. In addition, an optimizer is reinitialized at the beginning of each epoch to stabilize the model training inspired by Deng et al. (2024b)’s setting, with its effects described in Appendix E. All methods are trained for 5 epochs for fair comparison, with detailed implementation of VARR/VARR+ provided in Appendix B.

5.2 Baselines

We compared our method against the following baselines: Explicit-CoT (Nye et al., 2021), where the model is finetuned with explicit chain-of-thought reasoning; ICoT-SI (Deng et al., 2024b), where the model is fine-tuned using a linear token elimination schedule; and Coconut (Hao et al., 2024), where the model is fined-tuned to compress rationales into a predefined number of token hidden states. We excluded Implicit-CoT (Deng et al., 2023) from our evaluation due to its substantial computational demands, specifically requiring three models to be trained simultaneously on a single GPU. Moreover, Deng et al. (2024b) demonstrated that this method achieves a lower performance compared to ICoT-SI. Given that both our baselines and VARR aim to maintain Explicit-CoT’s performance while reducing the number of generated tokens, we establish Explicit-CoT’s performance as our primary baseline for comparison. All baselines were trained on a single A100-80GB GPU, and detailed training configurations for each method are provided in Appendix C.

5.3 Evaluation

We employ two evaluation metrics: First, we evaluate the accuracy of each method in generating the final answer for the respective tasks. Second, we count the generated tokens to evaluate reasoning efficiency while maintaining performance.
5.4 Main results

In Figure 3, we present the results for each reasoning task across different models. VARR/VARR+ achieves comparable or superior performance compared to Explicit-Cot across most datasets while reducing the average token generation. Specifically, VARR+ significantly increases performance by an average of 7.71% across all datasets and models, while improving efficiency by reducing token generation by 19.37% on average.

It is noteworthy that these findings contrast with ICoT-SI, demonstrating that performance can be improved while reducing the number of generated tokens. This suggests that existing reasoning data contains unnecessary reasoning sentences that may harm performance. Furthermore, effective reasoning can be achieved by selectively removing sentences based on appropriate criteria.

However, the baselines exhibit performance degradation compared to Explicit-Cot. For ICoT-SI, we observe an average performance decline of 21.98%, while Coconut shows a degradation of 25.20%, demonstrating their imbalanced trade-off in efficiency. These results suggest that heuristic reasoning reduction approaches do not effectively induce implicit reasoning within the model as discussed by ICoT-SI and Coconut. This indicates that identifying and retaining appropriate reasoning units through principled criteria is crucial for maintaining performance in practical applications. A more detailed discussion of the impact of the choice of reduction unit and criterion is provided in Section 5.5.1.

Furthermore, qualitative analysis (refer to Appendix J) confirms that ICoT-SI and Coconut fail to generate valid reasoning paths for answer generation, while VARR+ produces concise, yet effective reasoning paths that lead to correct answers. Additionally, the incorporation of incorrect answers in VARR+ resulted in performance improvements across most datasets. While VARR alone enhanced generation efficiency, VARR+ effectively preserves rationales that help calibrate the probability distribution between correct and incorrect answers, contributing to improved training robustness and stability.

5.5 Ablation Studies

In this section, we conduct ablation studies to empirically validate our method. All experiments are implemented using Mistral 7B due to its higher base capacity compared to other models.

5.5.1 Identifying Appropriate Units for Removal

<table>
  <tr>
    <th></th>
    <th>MQA</th>
    <th>G8K</th>
    <th>CQA</th>
    <th>TQA</th>
    <th>SQA</th>
  </tr>
  <tr>
    <th>Exp-CoT</th>
    <td>55.84<br>(105.02)</td>
    <td><b>55.26</b><br>(122.54)</td>
    <td>84.33<br>(42.84)</td>
    <td>82.94<br>(63.46)</td>
    <td>74.70<br>(46.47)</td>
  </tr>
  <tr>
    <th>ICoT-SI</th>
    <td>35.84<br>(113.55)</td>
    <td>28.27<br>(104.41)</td>
    <td>67.82<br>(6.0)</td>
    <td>77.09<br>(22.03)</td>
    <td>61.33<br>(22.86)</td>
  </tr>
  <tr>
    <th>VARR-Tok</th>
    <td>46.79<br>(90.39)</td>
    <td>47.53<br>(94.16)</td>
    <td>82.60<br>(25.07)</td>
    <td>67.14<br>(38.10)</td>
    <td>71.22<br>(31.92)</td>
  </tr>
  <tr>
    <th>VARR-Sent</th>
    <td><b>56.95</b><br>(91.04)</td>
    <td>54.98<br>(100.38)</td>
    <td><b>89.56</b><br>(33.55)</td>
    <td><b>83.45</b><br>(39.17)</td>
    <td><b>78.19</b><br>(35.12)</td>
  </tr>
</table>

Table 1: Analysis across various reasoning reduction units and the application of principled criteria. Each row presents accuracy in the first line, with average generated tokens shown in parentheses in the second line.

In this section, we empirically investigate the necessity of our criteria and demonstrate why sentences are more effective than tokens as reasoning reduction units. While ICoT-SI removes tokens without specific criteria, we first apply VARR+ at the token level (denoted as VARR-Tok) to assess whether tokens can serve as effective reduction units when combined with our criteria. As shown in Table 1, VARR+ applied at the token level achieves an average performance gain of 24.74% compared to ICoT-SI, demonstrating that our principled criteria contribute to robust performance. Furthermore, expanding the reduction units from tokens to sentences (denoted as VARR-Sent) yields an additional performance gain of 15.98% over VARR-Tok. These findings highlight that sentences provide natural and effective boundaries for the reduction process.

5.5.2 Analyzing the Impact of Sentence Position on Removal Efficacy

In Section 3, we demonstrated that gradually removing sentences from random and back positions can degrade model performance. To further explore this finding and assess the robustness of removing sentences from the front position, we conducted experiments with unguided random sentence removal (denoted as No Rule) and applied VARR+ with random position and reverse sentence order (denoted as Random and Back, respectively). As shown in Table 2, unguided random sentence removal resulted in a 25.30% decrease in performance relative to our method, highlighting the critical role of the verbosity evaluation even after selecting sen-
<table>
  <tr>
    <th></th>
    <th>MQA</th>
    <th>G8K</th>
    <th>CQA</th>
    <th>TQA</th>
    <th>SQA</th>
  </tr>
  <tr>
    <td>Exp-CoT</td>
    <td>55.84 (105.02)</td>
    <td><b>55.26</b> (122.54)</td>
    <td>84.33 (42.84)</td>
    <td>82.94 (63.46)</td>
    <td>74.70 (46.47)</td>
  </tr>
  <tr>
    <td>No Rule</td>
    <td>35.06 (59.92)</td>
    <td>25.85 (63.56)</td>
    <td>74.78 (23.56)</td>
    <td>74.14 (6.84)</td>
    <td>72.38 (23.38)</td>
  </tr>
  <tr>
    <td>Random</td>
    <td>55.34 (99.13)</td>
    <td>52.31 (115.59)</td>
    <td>83.47 (49.19)</td>
    <td>79.62 (44.98)</td>
    <td>73.83 (50.51)</td>
  </tr>
  <tr>
    <td>Back</td>
    <td>49.71 (92.64)</td>
    <td>48.52 (103.25)</td>
    <td>85.21 (19.86)</td>
    <td>70.0 (36.22)</td>
    <td>74.41 (18.17)</td>
  </tr>
  <tr>
    <td>Front</td>
    <td><b>56.95</b> (91.04)</td>
    <td>54.98 (100.38)</td>
    <td><b>89.56</b> (33.55)</td>
    <td><b>83.45</b> (39.17)</td>
    <td><b>78.19</b> (35.12)</td>
  </tr>
</table>

Table 2: Performance across 5 different reasoning tasks, evaluated with different sentence position removal. Each row presents accuracy in the first line, with average generated tokens shown in parentheses in the second line.

sences as units of reduction. Furthermore, Random and Back strategies exhibited an average 7.50% performance degradation relative to our method. These results further support our observation that earlier sentences in the reasoning path tend to contain more redundancy, and their prioritized removal effectively balances rationale reduction while maintaining reasoning performance.

<table>
  <tr>
    <th></th>
    <th>G8K</th>
    <th>SQA</th>
    <th>TQA</th>
  </tr>
  <tr>
    <td>Exp-CoT</td>
    <td>55.84 (122.54)</td>
    <td>74.70 (46.47)</td>
    <td>82.94 (63.46)</td>
  </tr>
  <tr>
    <td>RAND-SENT REMOVAL</td>
    <td>25.85 (63.56)</td>
    <td>72.38 (23.38)</td>
    <td>74.14 (6.84)</td>
  </tr>
  <tr>
    <td>FRONT-SENT REMOVAL (1)</td>
    <td>38.36 (87.43)</td>
    <td>72.96 (24.34)</td>
    <td>79.74 (22.73)</td>
  </tr>
  <tr>
    <td>FRONT-SENT REMOVAL (2)</td>
    <td>23.50 (47.64)</td>
    <td>73.83 (9.97)</td>
    <td>76.79 (8.32)</td>
  </tr>
  <tr>
    <td>VARR+</td>
    <td>54.98 (100.38)</td>
    <td><b>78.19</b> (35.12)</td>
    <td><b>83.45</b> (39.17)</td>
  </tr>
</table>

Table 3: Performance across different front sentence removal strategies. RAND-SENT-REMOVAL refers to the Random strategy, while (·) in the FRONT-SENT REMOVAL indicates the number of enforced removed front sentences. Each row presents accuracy in the first line, with average generated tokens shown in parentheses in the second line.

To further validate whether VARR enhances robustness compared to enforced removal of front sentences, we conducted additional experiments where the first and second sentences were enforced to be removed during the initial two training epochs. As shown in Table 3, enforced removal of front sentences leads to substantial performance degradation, while VARR demonstrates its contribution to preserving robustness.

5.5.3 Varying the Warm-up Ratio

![Accuracy barplot and marker across various warm-up stages on TriviaQA and CommonsenseQA.](page_1012_186_382_246.png)

Figure 4: Accuracy (barplot) and the average generated token (marker) across various warm-up stages on TriviaQA and CommonsenseQA.

We evaluated our method against various warm-up stages on the TriviaQA and CommonsenseQA datasets. As the duration of the warm-up stages increases, the model becomes more fitted to the non-reduction dataset, which inhibits VARR’s ability to eliminate redundant sentences from the reasoning path. Consequently, as illustrated in Figure 4, longer warm-up periods result in an increase in generated tokens and a decrease in accuracy. These results suggest that 0.1 training steps provide sufficient time to inject reasoning abilities while enabling the systematic reduction of redundant reasoning sentences during the learning process. The zero warm-up configuration demonstrates comparable performance, though 0.1 warm-up steps are slightly better (detailed in Appendix F). Therefore, we set 0.1 as the default setting for the warm-up steps.

5.5.4 Removal Ratio Analysis

![Removal ratio of redundant sentences during training.](page_1012_1402_382_246.png)

Figure 5: Removal ratio of redundant sentences during training. The y-axis shows the removal ratio, calculated as the number of removed sentences divided by the maximum potential removal sentences (\( \text{size}(\{r_i\})/r(t) \)). The x-axis represents training epochs.

In Figure 5, we analyze the actual amount of
rationale sentences removed during training. We examine it by calculating the removal ratio, the proportion of actual removed sentences to the maximum potential removal sentences \( r(t) \). Our analysis indicates that not all sentences designated for maximum removal range are always eliminated during the training process. Notably, a significant proportion of redundant sentences are removed in the early stages of training, with fewer sentences being removed as the model progresses through the middle to later stages, thereby stabilizing its training. A similar trend is observed across other datasets, as detailed in Appendix G.

6 Conclusion

In this work, we propose the sentence-level rationale reduction framework VARR and empirically demonstrate that models trained with non-redundant rationales achieve enhanced efficiency. We address the lack of principled criteria for identifying redundant sentences during training by developing a reduction framework that not only preserves the model’s reasoning capabilities but also reduces the likelihood of generating incorrect answers. Our experiments show that VARR can efficiently handle a diverse range of tasks with fewer generated tokens, without sacrificing its accuracy. This work contributes novel insights to rationale reduction research, contributing to the efficient reasoning elicitation in language models.

Limitations

While our work provides novel insights into rationale reduction research, our experiments were primarily conducted using a relatively small large language model and limited batch size, constrained by computational costs (i.e., a single A100-80GB GPU). Additionally, for the same reasons, it was not feasible to test the model with datasets featuring long sequences in both queries and rationales (Reddy et al., 2024; Yu et al., 2024a). Nevertheless, given the systematic design principles underlying the VARR/VARR+ frameworks, we believe their effectiveness would extend to larger-scale implementations. Furthermore, we reserve the application of VARR/VARR+ in iterative reasoning path generation and refinement/reflexion-based evaluation discussed in Section 2.1 for future work.

Ethical Considerations

Our work explores how LLMs can maintain their reasoning performance while improving efficiency. To this end, we conducted verbosity-aware rationale reduction (i.e., reasoning sentence pruning)-based CoT fine-tuning, requiring computational resources comparable to standard CoT fine-tuning. Additionally, we used only open-source LLMs and publicly available reasoning datasets with minimal preprocessing using gpt4o-mini’s (OpenAI, 2024) API. Therefore, we do not anticipate significant ethical issues arising from our work. On the contrary, we believe future works could leverage our analysis to reduce computational overhead in CoT inference settings.

Acknowledgements

We sincerely appreciate to Keonwoo Kim and Hyowon Cho for their sharp feedback during writing, Sangyeop Kim, Yukyung Lee, and Minjin Jeon for their continuous encouragement and inspiration during our study sessions, and Eunbi Choi for her invaluable help during the rebuttal process. This work was supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government(MSIT) (No.RS-2019-II191906, Artificial Intelligence Graduate School Program(POSTECH)), Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No.2018-0-00584, (SW starlab) Development of Decision Support System Software based on Next-Generation
Machine Learning), National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. RS-2024-00335873), and National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. RS-2023-00217286).

References

LlamaTeam AI@Meta. 2024. The llama 3 herd of models. Preprint, arXiv:2407.21783.

Aida Amini, Saadia Gabriel, Shanchuan Lin, Rik Koncel-Kedziorski, Yejin Choi, and Hannaneh Hajishirzi. 2019a. Mathqa: Towards interpretable math word problem solving with operation-based formalisms. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 2357–2367.

Aida Amini, Saadia Gabriel, Shanchuan Lin, Rik Koncel-Kedziorski, Yejin Choi, and Hannaneh Hajishirzi. 2019b. MathQA: Towards interpretable math word problem solving with operation-based formalisms. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 2357–2367. Minneapolis, Minnesota. Association for Computational Linguistics.

Mohammad Gheshlaghi Azar, Zhaohan Daniel Guo, Bilal Piot, Remi Munos, Mark Rowland, Michal Valko, and Daniele Calandriello. 2024. A general theoretical paradigm to understand learning from human preferences. In International Conference on Artificial Intelligence and Statistics, pages 4447–4455. PMLR.

Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021a. Training verifiers to solve math word problems. Preprint, arXiv:2110.14168.

Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021b. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168.

Yihe Deng, Weitong Zhang, Zixiang Chen, and Quanquan Gu. 2024a. Rephrase and respond: Let large language models ask better questions for themselves. Preprint, arXiv:2311.04205.

Yuntian Deng, Yejin Choi, and Stuart Shieber. 2024b. From explicit cot to implicit cot: Learning to internalize cot step by step. Preprint, arXiv:2405.14838.

Yuntian Deng, Kiran Prasad, Roland Fernandez, Paul Smolensky, Vishrav Chaudhary, and Stuart Shieber. 2023. Implicit chain of thought reasoning via knowledge distillation. Preprint, arXiv:2311.01460.

Qingxiu Dong, Jingjing Xu, Lingpeng Kong, Zhifang Sui, and Lei Li. 2023. Statistical knowledge assessment for large language models. In Thirty-seventh Conference on Neural Information Processing Systems.

Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant. 2021. Did aristotle use a laptop? a question answering benchmark with implicit reasoning strategies. Transactions of the Association for Computational Linguistics, 9:346–361.

Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiteng Hu, Jason Weston, and Yuandong Tian. 2024. Training large language models to reason in a continuous latent space. arXiv preprint arXiv:2412.06769.

Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the math dataset. NeurIPS.

Jiwoo Hong, Noah Lee, and James Thorne. 2024. Orpo: Monolithic preference optimization without reference model. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 11170–11189.

Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. 2023. Mistral 7b. Preprint, arXiv:2310.06825.

Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. 2017. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1601–1611.

Seungone Kim, Se June Joo, Doyoung Kim, Joel Jang, Seonghyeon Ye, Jamin Shin, and Minjoon Seo. 2023. The cot collection: Improving zero-shot and few-shot learning of language models via chain-of-thought fine-tuning. In The 2023 Conference on Empirical Methods in Natural Language Processing.

Aviral Kumar, Vincent Zhuang, Rishabh Agarwal, Yi Su, John D Co-Reyes, Avi Singh, Kate Bauml, Shariq Iqbal, Colton Bishop, Rebecca Roelofs, Lei M Zhang, Kay McKinney, Disha Shrivastava, Cosmin Paduraru, George Tucker, Doina Precup, Feryal Behbahani, and Aleksandra Faust. 2024. Training language models to self-correct via reinforcement learning. Preprint, arXiv:2409.12917.
Tengxiao Liu, Qipeng Guo, Xiangkun Hu, Cheng Jiayang, Yue Zhang, Xipeng Qiu, and Zheng Zhang. 2024. Can language models learn to skip steps? In The Thirty-eighth Annual Conference on Neural Information Processing Systems.

Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In International Conference on Learning Representations.

Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. 2024. Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems, 36.

Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, et al. 2021. Show your work: Scratchpads for intermediate computation with language models. arXiv preprint arXiv:2112.00114.

OpenAI. 2024. Gpt-4o mini: advancing cost-efficient intelligence.

Santosh Kumar Radha, Yasamin Nouri Jelyani, Ara Ghukasyan, and Oktay Goktas. 2024. Iteration of thought: Leveraging inner dialogue for autonomous large language model reasoning. Preprint, arXiv:2409.12618.

Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2024. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36.

Varshini Reddy, Rik Koncel-Kedziorski, Viet Dac Lai, Michael Krundick, Charles Lovering, and Chris Tanner. 2024. Docfinqa: A long-context financial reasoning dataset. arXiv preprint arXiv:2401.06915.

Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2024. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36.

Damien Sileo. 2024. tasksource: A large collection of NLP tasks with a structured dataset preprocessing framework. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 15655–15684, Torino, Italia. ELRA and ICCL.

Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019. Commonsenseqa: A question answering challenge targeting commonsense knowledge. In Proceedings of the 2019 Conference of the North, page 4149. Association for Computational Linguistics.

Shubham Tosnhiwal, Wei Du, Ivan Moshkov, Branislav Kisacanin, Alexan Ayrapetyan, and Igor Gitman. 2024. Openmathinstruct-2: Accelerating ai for math with massive open-source instruction data. arXiv preprint arXiv:2410.01560.

Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations.

Yu Wang, Shiwan Zhao, Zhihu Wang, Heyuan Huang, Ming Fan, Yubo Zhang, Zhixing Wang, Haijun Wang, and Ting Liu. 2024. Strategic chain-of-thought: Guiding accurate reasoning in Ilms through strategy elicitation. Preprint, arXiv:2409.03271.

Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, Rui Zheng, Xiaoran Fan, Xiao Wang, Limao Xiong, Yuqiao Zhou, Weiran Wang, Changhao Jiang, Yicheng Zou, Xiangyang Liu, Zhangyue Yin, Shihan Dou, Rongxiang Weng, Wensen Cheng, Qi Zhang, Wenjuan Qin, Yongyan Zheng, Xipeng Qiu, Xuanjing Huang, and Tao Gui. 2023. The rise and potential of large language model based agents: A survey. Preprint, arXiv:2309.07864.

Ling Yang, Zhaochen Yu, Tianjun Zhang, Shiyi Cao, Minkai Xu, Wentao Zhang, Joseph E. Gonzalez, and Bin CUI. 2024. Buffer of thoughts: Thought-augmented reasoning with large language models. In The Thirty-eighth Annual Conference on Neural Information Processing Systems.

Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik R Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models. In Thirty-seventh Conference on Neural Information Processing Systems.

Zhangyue Yin, Qiushi Sun, Qipeng Guo, Zhiyuan Zeng, Xiaonan Li, Junqi Dai, Qinyuan Cheng, Xuan-Jing Huang, and Xipeng Qiu. 2024. Reasoning in flux: Enhancing large language models reasoning through uncertainty-aware adaptive guidance. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2401–2416.

Longhui Yu, Weisen Jiang, Han Shi, Jincheng YU, Zhengying Liu, Yu Zhang, James Kwok, Zhenghou Li, Adrian Weller, and Weiyang Liu. 2024a. Metamath: Bootstrap your own mathematical questions for large language models. In The Twelfth International Conference on Learning Representations.

Ping Yu, Jing Xu, Jason Weston, and Ilia Kulikov. 2024b. Distilling system 2 into system 1. arXiv preprint arXiv:2407.06023.
A Additional Datasets Analysis

As shown in Section 3, the NLL tends to rise with increasing size of \( \{r_i\} \) in random and back configurations, compared to the front configuration, as illustrated in Figure 6, this trend is consistent across both the GSM8K and StrategyQA datasets.

![Line plot showing NLL differences across varying sizes of {r_i}. The 'original' represents the NLL for the full rationale, while 'front', 'random', and 'back' indicate that r_i are sampled from the front, random, and back indices of the full index set, respectively.](page_120_495_474_377.png)

Figure 6: NLL differences across varying sizes of \( \{r_i\} \). The ‘original’ represents the NLL for the full rationale, while ‘front’, ‘random’, and ‘back’ indicate that \( r_i \) are sampled from the front, random, and back indices of the full index set, respectively.

B VARR Implementation

The detailed implementation of VARR+ is outlined in Algorithm 1. After the warm-up stages, each datum in the current batch is evaluated using the verbosity equation—specifically, using only Equation 5 in VARR and both Equations 5 and 9 in VARR+. As mentioned in Section 3 and 4, the process assesses the redundancy of each sentence in data starting from the first index during every single epoch. Therefore, all data can potentially shorten the length of the rationale progressively.

C Additional Experimental Details

For all experiments, we employ the AdamW optimizer (Loshchilov and Hutter, 2019), configured with a weight decay of 0.005. For the Mistral 7B model, we utilize an effective batch size of 12 with gradient accumulation set to 3, while the smaller Llama3.2 models use an effective batch size of 15. For Coconut (Hao et al., 2024), we set the max\_latent\_stage to 5 while maintaining all other hyperparameters as their default repository values unless otherwise mentioned. A constant learning rate of \( 5 \times 10^{-6} \) is applied across all datasets, with bfloat16 precision. For Multiple-Choice and True/False tasks, complete sets of non-correct labels are employed to configure in-batch negatives to enhance the stability of the verbosity evaluation. To ensure a fair comparison, all baselines and methods are trained on a single A100 GPU with 80GB of memory for up to 5 epochs or 36 hours, whichever is reached first and experimented with single run evaluation (note that our setting is different from prompting/inference only setting). Regarding licensing, Mistral 7B is licensed under Apache License, Version 2.0, while Llama3.2 is governed by the Llama 3.2 Community License.

Algorithm 1 Training Procedure of VARR+

<table>
  <tr>
    <th>1:</th>
    <td>\( \mathcal{D} \): Training dataset</td>
  </tr>
  <tr>
    <th>2:</th>
    <td>\( \mathcal{B} \): Training batch</td>
  </tr>
  <tr>
    <th>3:</th>
    <td>\( E \): Total training epochs</td>
  </tr>
  <tr>
    <th>4:</th>
    <td>\( S \): Total training steps per epoch</td>
  </tr>
  <tr>
    <th>5:</th>
    <td>\( T = E \times S \): Total training steps</td>
  </tr>
  <tr>
    <th>6:</th>
    <td>\( T_{warmup} = 0.1 \times T \): Warm-up steps</td>
  </tr>
  <tr>
    <th>7:</th>
    <td>\( R_B \): A buffer to store removed sentences</td>
  </tr>
  <tr>
    <th>8:</th>
    <td>\( r(t) \): Maximum number of removable sentences at \( t \)</td>
  </tr>
  <tr>
    <th>9:</th>
    <td>\( N_i \): Number of rationale sentences at \( t \)</td>
  </tr>
  <tr>
    <th>10:</th>
    <td>\( \theta \): Trained model parameters</td>
  </tr>
  <tr>
    <th>11:</th>
    <td><b>for</b> epoch = 1 to E <b>do</b></td>
  </tr>
  <tr>
    <th>12:</th>
    <td><b>for</b> step = 1 to S <b>do</b></td>
  </tr>
  <tr>
    <th>13:</th>
    <td>Sample training batch \( \mathcal{B} \) from \( \mathcal{D} \)</td>
  </tr>
  <tr>
    <th>14:</th>
    <td>\( t = (epoch - 1) \times S + step \)</td>
  </tr>
  <tr>
    <th>15:</th>
    <td><b>if</b> \( t \leq T_{warmup} \) <b>then</b></td>
  </tr>
  <tr>
    <th>16:</th>
    <td>pass</td>
  </tr>
  <tr>
    <th>17:</th>
    <td><b>end if</b></td>
  </tr>
  <tr>
    <th>18:</th>
    <td><b>for each</b> \( d \in \mathcal{B} \) <b>do</b></td>
  </tr>
  <tr>
    <th>19:</th>
    <td>\( R_B \leftarrow \{\} \)</td>
  </tr>
  <tr>
    <th>20:</th>
    <td><b>for</b> i = 1 to \( N_i \) <b>do</b></td>
  </tr>
  <tr>
    <th>21:</th>
    <td><b>if</b> \( r_i \) satisfies Equations 5 and 9 <b>then</b></td>
  </tr>
  <tr>
    <th>22:</th>
    <td>remove \( r_i \) from \( d \)</td>
  </tr>
  <tr>
    <th>23:</th>
    <td>Add \( r_i \) to \( R_B \)</td>
  </tr>
  <tr>
    <th>24:</th>
    <td><b>if</b> \( |R_B| \geq r(t) \) <b>then</b></td>
  </tr>
  <tr>
    <th>25:</th>
    <td><b>break</b></td>
  </tr>
  <tr>
    <th>26:</th>
    <td><b>end if</b></td>
  </tr>
  <tr>
    <th>27:</th>
    <td><b>end if</b></td>
  </tr>
  <tr>
    <th>28:</th>
    <td><b>end for</b></td>
  </tr>
  <tr>
    <th>29:</th>
    <td><b>end for</b></td>
  </tr>
  <tr>
    <th>30:</th>
    <td>Forward pass</td>
  </tr>
  <tr>
    <th>31:</th>
    <td>Backward pass and update \( \theta \)</td>
  </tr>
  <tr>
    <th>32:</th>
    <td><b>end for</b></td>
  </tr>
  <tr>
    <th>33:</th>
    <td>Reinitialize optimizer</td>
  </tr>
  <tr>
    <th>34:</th>
    <td><b>end for</b></td>
  </tr>
</table>

D Details and Statistics of Datasets

For our experimental analysis, we carefully selected a diverse set of five datasets used in prior works (Deng et al., 2023, 2024a; Liu et al., 2024; Yu et al., 2024b; Yin et al., 2024). To ensure explicit sentence boundaries, all datasets were prepro-
<table>
  <tr>
    <th>Dataset</th>
    <th>Reasoning Task</th>
    <th>Source</th>
    <th>Answer Format</th>
    <th># TRAIN.</th>
    <th># VALID.</th>
    <th># TEST.</th>
  </tr>
  <tr>
    <td>GSM8K (Cobbe et al., 2021a)</td>
    <td>Arithmetic</td>
    <td>Cobbe et al. (2021b)</td>
    <td>Number</td>
    <td>7000</td>
    <td>473</td>
    <td>1319</td>
  </tr>
  <tr>
    <td>MathQA (Amini et al., 2019a)</td>
    <td>Arithmetic</td>
    <td>Amini et al. (2019b)</td>
    <td>Multi-choice</td>
    <td>29837</td>
    <td>4475</td>
    <td>2985</td>
  </tr>
  <tr>
    <td>TriviaQA (Joshi et al., 2017)</td>
    <td>Commonsense</td>
    <td>Kim et al. (2023)</td>
    <td>Natural Language</td>
    <td>8844</td>
    <td>552</td>
    <td>1659</td>
  </tr>
  <tr>
    <td>CommonsenseQA (Talmor et al., 2019)</td>
    <td>Multi-choice</td>
    <td>Kim et al. (2023)</td>
    <td>Multi-choice</td>
    <td>609</td>
    <td>38</td>
    <td>115</td>
  </tr>
  <tr>
    <td>StrategyQA (Geva et al., 2021)</td>
    <td>Commonsense</td>
    <td>Sileo (2024)</td>
    <td>T/F</td>
    <td>1832</td>
    <td>114</td>
    <td>344</td>
  </tr>
</table>

Table 4: Comprehensive statistics of the datasets used in our experiments are provided. GSM8K and MathQA are sourced directly from their original datasets, while the remaining datasets were obtained from Kim et al. (2023) and Sileo (2024) to access complete reasoning paths. GSM8K, StrategyQA, and CommonsenseQA are licensed under the MIT License, whereas MathQA and TriviaQA are distributed under the Apache License, Version 2.0.

cessed using gpt-4o-mini (OpenAI, 2024) to establish clear sentence demarcation (e.g., ‘He bikes 20*2=«20*2=40»40 miles each day for work So he bikes 40*5=«40*5=200»200 miles for work’ becomes ‘He bikes 202=«202=40»40 miles each day for work. So he bikes 405=«405=200»200 miles for work’). Table 4 comprehensively outlines each dataset, including its source and the size of the training, validation, and test samples.

<table>
  <tr>
    <th></th>
    <th>MQA</th>
    <th>G8K</th>
    <th>CQA</th>
    <th>TQA</th>
    <th>SQA</th>
  </tr>
  <tr>
    <td>w/o reinit</td>
    <td>48.70 (92.86)</td>
    <td>45.18 (115.91)</td>
    <td>87.82 (32.95)</td>
    <td>83.72 (38.27)</td>
    <td>76.74 (33.89)</td>
  </tr>
  <tr>
    <td>w/ reinit</td>
    <td>56.95 (91.04)</td>
    <td>54.98 (100.38)</td>
    <td>89.56 (33.55)</td>
    <td>83.45 (39.17)</td>
    <td>78.19 (35.12)</td>
  </tr>
</table>

Table 5: Performance comparison before and after the reinitializing optimizer in every training epoch. Each row presents accuracy in the first line, with average generated tokens shown in parentheses in the second line.

E Reinitializing the Optimizer

We reinitialized the optimizer after each training epoch to stabilize training, inspired by Deng et al. (2024b). Our implementation uses the AdamW optimizer (Loshchilov and Hutter, 2019), where the first and second moments are gradually updated based on current gradients. Consequently, when VARR reduces rationales for certain data points between epochs, the training process could become unstable. As shown in table 6, the average 19.32% performance improvement achieved through optimizer reinitialization extends the findings of Deng et al. (2024b) beyond simple tasks (e.g., multiplication; synthesized dataset) to demonstrate effectiveness across diverse datasets with complex semantic and syntactic reasoning structures.

F Zero Warm-up Stage

To validate whether the warm-up stage sensitivity could limit applicability, we conducted additional experiments extending warm-up stages to zero. As shown in Table, the zero warm-up stage shows comparable effectiveness and efficiency to the 0.1 warm-up stage, demonstrating the robustness of our method across different application scenarios. These findings suggest a zero-warm stage can be a good option for various application scenarios, but starting with a 0.1 warm-up stage can boost its stability.

<table>
  <tr>
    <th>warm-up</th>
    <th>0.0</th>
    <th>0.1</th>
    <th>0.2</th>
    <th>0.3</th>
    <th>0.4</th>
  </tr>
  <tr>
    <td>TQA</td>
    <td>82.01 (40.11)</td>
    <td>83.72 (38.27)</td>
    <td>83.45 (47.10)</td>
    <td>81.19 (46.55)</td>
    <td>81.97 (47.97)</td>
  </tr>
  <tr>
    <td>CQA</td>
    <td>90.34 (32.81)</td>
    <td>89.56 (35.07)</td>
    <td>87.82 (39.38)</td>
    <td>87.82 (40.98)</td>
    <td>86.08 (43.61)</td>
  </tr>
</table>

Table 6: Accuracy and average generated tokens across the various warm-up stages.

![Line plot showing removal ratio over epochs for GSM8K, Strategy-QA, and Math-QA datasets](page_1012_1342_410_246.png)

Figure 7: Additional analysis of the Removal ratio on other datasets. The Removal ratio, indicated on the y-axis, is calculated by dividing the number of sentences actually removed by the maximum potential removal ratio (i.e., \( \text{size}(\{r_i\}) / r(t) \)). The x-axis represents the index of the epochs.

In the GSM8K, StrategyQA, and MathQA datasets, similar trends are observed as in Section 5.5.4. Additionally, we observe that our method reduces fewer sentences in later train-
ing stages, demonstrating that verbosity preserves essential reasoning sentences even when larger \( r(t) \) encourages substantial reduction. Notably, in StrategyQA, the Removal ratio converges more rapidly than in other datasets. This is attributed to the dataset’s characteristics, where the sentences within the rationale predominantly list simple information rather than forming complex, interrelated structures. Consequently, redundant sentences are rapidly removed, leading to a rapid decrease in further removal activities.

H Exploration on Complex Task

<table>
  <tr>
    <th></th>
    <th>Exp-CoT</th>
    <th>VARR+</th>
  </tr>
  <tr>
    <th>Acc</th>
    <td>9.01</td>
    <td>10.27</td>
  </tr>
  <tr>
    <th>Gen Tok</th>
    <td>300.17</td>
    <td>252.00</td>
  </tr>
</table>

Table 7: Performance comparison between Exp-CoT and VARR+ on MATH datasets.

To further validate that our method remains effective on complex tasks, we conducted additional experiments with the MATH (Hendrycks et al., 2021) dataset. Considering computational constraints, we selected 500 samples from the math subset of OpenMathInstruct-2 (Toshniwal et al., 2024) for the training set and evaluated on the official test set. As described in Table 8, our results on MATH further demonstrate the versatility and future applicability of VARR+ on more complex tasks.

I Training Time Analysis

<table>
  <tr>
    <th>Dataset</th>
    <th>GSM8K (7K)</th>
    <th>TQA (2K)</th>
  </tr>
  <tr>
    <th>Training Time (\( \uparrow \))</th>
    <td>0.18</td>
    <td>0.30</td>
  </tr>
  <tr>
    <th>Token Efficiency (\( \downarrow \))</th>
    <td>0.81</td>
    <td>0.61</td>
  </tr>
  <tr>
    <th>Accuracy (\( \uparrow \))</th>
    <td>0.98</td>
    <td>1.01</td>
  </tr>
</table>

Table 8: Training time analysis on GSM8K and TQA. All scores are calculated as the value for Explicit-CoT divided by the value for VARR+ (i.e., Explicit-CoT/VARR+).

As VARR computes the likelihood of each sentence to ensure robust performance, training time inevitably increases as dataset size grows. We therefore quantitatively measure its efficiency on GSM8K and TQA relative to Explicit-CoT. Although there is a trade-off in training time, VARR+ improves token efficiency while preserving accuracy. Since training occurs only once while the improved efficiency benefits persist continuously afterward in test-time, we believe the additional training cost is justified.

J Qualitative Analysis

Table 9 illustrates the generation outputs for each method on the GSM8K test set. While Explicit-CoT generates a complete reasoning path, its redundant generation of ‘needs a 3 piece place setting’ introduces hallucinations into the reasoning process, ultimately leading to an incorrect answer. ICoT-SI’s token-level elimination of reasoning paths, on the other hand, results in generations that lack sufficient reasoning capacity. Additionally, Coconut’s predefined hidden-states fail to compress the complete reasoning path in its token-level compression approach. In contrast, VARR+ demonstrates efficient generation by concisely producing ‘$15.00’ in a single sentence, achieving both effectiveness and efficiency in its output. Additional qualitative examples on the MathQA dataset are outlined in Table 10.
<table>
  <tr>
    <th>Question</th>
    <td>Avery needs to buy a 3 piece place setting (dinner & salad plate and a bowl) for her holiday dinner. She’s having 12 people over for dinner. If the dinner plates cost $6.00 each and bowls each cost $5.00 and the salad plates cost $4.00, how much will she spend on place settings?</td>
  </tr>
  <tr>
    <th>Explicit-CoT</th>
    <td>She’s having 12 people over and needs a 3 piece place setting for each so she needs 12*3 = 36 place settings.<br>
The dinner plates cost $6.00 each and she needs 36 of them so that comes to 6*36 = $216.00.<br>
The bowl costs $5.00 each and she needs 36 of them so that comes to 5*36 = $180.00.<br>
The salad plates cost $4.00 each and she needs 36 of them so that comes to 4*36 = $144.00.<br>
All total, the place settings will cost $216 for dinner plates, $180 for bowls and $144 for salad plates for a total of 216+180+144 = $540.00<br>
#### 540</td>
  </tr>
  <tr>
    <th>ICoT-SI</th>
    <td>each so 12*6 = $72.00 She’s buying 3 pieces of place settings (dinner plates, salad plates and bowls) for $72.00 each so 3*72 = $216.00<br>
#### 216</td>
  </tr>
  <tr>
    <th>Coconut</th>
    <td>&lt;|start-latent|&gt;&lt;|latent|&gt;&lt;|latent|&gt;&lt;|latent|&gt;&lt;|end-latent|&gt;<br>
#### 4.00</td>
  </tr>
  <tr>
    <th>VARR+</th>
    <td>She needs 12 place settings and each setting costs $15.00 so she will spend 12*15 = $180.00<br>
#### 180</td>
  </tr>
  <tr>
    <th>Ground Truth Answer</th>
    <td>#### 180</td>
  </tr>
</table>

Table 9: Qualitative Example: Generation outputs on GSM8K test set.
<table>
  <tr>
    <th>Question</th>
    <td>The cost price of 20 articles is the same as the selling price of \( x \) articles. if the profit is 25%, find out the value of x.<br>a) 13, b) 14, c) 15, d) 16, e) 17</td>
  </tr>
  <tr>
    <th>Explicit-CoT</th>
    <td>explanation : let the cost price 1 article = rs . 1<br>cost price of \( x \) articles = rs . \( x \)<br>selling price of x articles = 125% of \( x \) = rs . ( \( 125 / 100 \times x \) ) = rs . ( \( 25 / 20 \times x \) ) = rs . 5 / 4 \( x \)<br>\( => x = 20 \)<br><span style="color: #ff6b6b;">cost price of 20 articles = rs . 20 selling price of 20 articles = rs . ( \( 25 / 4 \times 20 \) ) = rs . ( \( 25 / 4 \times 20 \) ) / 20 = rs . 1</span><br>\( => x = 20 \) answer : option a<br>#### a</td>
  </tr>
  <tr>
    <th>ICoT-SI</th>
    <td>of \( x \) articles = \( x \)<br>s.p of \( x \) articles = rs . 20<br>gain = rs . ( \( 20 - x \) )<br>gain % = 25 % of \( x \) = rs . ( \( 20 - x \) ) / \( x \) \( \times \) 100 = 25<br>\( => 20 - x = 5 \) \( x \) \( => 4 \) \( x = 20 - x \) \( => 5 \) \( x = 20 - x \) \( => 20 = 5 \) \( x \) \( => 200 = 5 \) \( x^2 \) \( => 400 = 5 \) \( x^2 \) \( => 400 / 5 = x^2 \) \( => x^2 = 80 \) \( => x = 10 \) or 10 . c<br>#### c</td>
  </tr>
  <tr>
    <th>Coconut</th>
    <td>&lt;start-latent!&gt;&lt;llatent!&gt;&lt;llatent!&gt;&lt;llatent!&gt;&lt;end-latent!&gt;<br>#### c</td>
  </tr>
  <tr>
    <th>VARR+</th>
    <td>explanation : let the cost price of \( x \) articles = \( x \)<br>s.p of \( x \) articles = 20<br>20 = 1.25 \( x \) \( => x = 16 \) option d.<br>#### d</td>
  </tr>
  <tr>
    <th>Ground Truth Answer</th>
    <td>#### d</td>
  </tr>
</table>

Table 10: Qualitative Example: Generation outputs on MathQA test set.

Reviewer paper summaries: The paper proposes VARR/VARR+ to evaluate the validity of the rationale. 
By cutting the rationale and judging whether each sentence can increase the likelihood of the correct answer, it can judge the validity of each sentence and remove invalid sentences.
Experiments prove that the method can effectively enhance the performance of the model while reducing the number of generated tokens, which proves the effectiveness of the method.

This paper introduces a novel sentence-level rationale reduction framework called Verbosity-Aware Rationale Reduction (VARR) for Large Language Models (LLMs). The authors address the trade-off between the extensive generation of reasoning steps that improves performance and the increased inference costs it causes. Unlike previous approaches that use token-level reduction without clear criteria, VARR leverages a likelihood-based metric called "verbosity" to identify and remove redundant reasoning sentences while preserving essential ones. Experimental results across various reasoning tasks demonstrate that their method improves performance by an average of 7.71% while reducing token generation by 19.87% compared to models trained with complete reasoning paths.

This paper proposes a sentence-level rationale reduction framework that dynamically removes redundant reasoning sentences based on a likelihood-based verbosity metric to improve the efficiency of the reasoning process.
Summary of strengths: 1. The method is simple yet effective and can be easily adapted to different tasks.
2. Experiments show the effectiveness of the method on various models and tasks.

1. The paper introduces a principled criterion "verbosity" for identifying redundant sentences in reasoning paths, which is conceptually sound and addresses limitations in previous work.
2. The authors evaluate their approach across diverse reasoning datasets (arithmetic and commonsense) and multiple model sizes (from 1B to 7B parameters), demonstrating the generalizability of their method.
3. Unlike previous methods that sacrifice accuracy for efficiency, VARR/VARR+ shows both improved performance and reduced token generation, advancing the Pareto frontier.

1.Compressing reasoning chains in LLMs to enhance efficiency is an emerging and valuable direction. The proposed sentence-level redundancy reduction (VARR) shortens the reasoning paths while maintaining performance comparable to explicit Chain-of-Thought (CoT).
2.The authors tested the method on various reasoning tasks, including mathematical and commonsense reasoning, demonstrating its generalizability across different task types.
Summary of weaknesses: 1. The experimental results in Figure 3 are confusing. Why does the performance of Llama3.2-1B on GSM8K significantly underperform the official results (44.4, see https://ai.meta.com/blog/llama-3-2-connect-2024-vision-edge-mobile-devices/)?
2. The effectiveness of the method on complex tasks remains to be validated. Since the problems in datasets like MathQA and GSM8K are relatively simple, resulting in shorter rationales, it is still uncertain whether the brief rationales of VARR will lead to a performance decline in more complex tasks (such as MATH and GPQA). These tasks often require longer rationales to achieve correct results.
3. The efficiency of the method is concerning. Although the authors claim that VARR improves efficiency compared to previous methods, the cost of calculating VARR for each sentence in the rationale could be substantial, especially when the training dataset is very large.

If the authors could address the above questions, I would be happy to increase the score.

1. The authors mention but don't explore the application of VARR/VARR+ to iterative reasoning path generation methods like Tree of Thoughts or Strategic Chain of Thought, which could be an important direction.
2. The experiments are limited to relatively small LLMs (up to 7B parameters) due to computational constraints. It remains unclear whether the findings would generalize to larger state-of-the-art models.
3. The method requires a warm-up stage (10% of total training steps) to inject reasoning capabilities before reduction can begin. The sensitivity to this hyperparameter could limit applicability in some scenarios.

1.In Equation 2, the assumption that the true distribution is a Dirac function (i.e., one-hot) may be limiting. In practical tasks, especially multiple-choice or open-ended generation tasks, this assumption could affect the method's applicability across different task formats.
2.In Figure 3, the performance of the two baseline methods, ICoT-SI and Coconut, appears to be significantly lower than the results reported in their original papers. For example, on the G8K dataset, ICoT-SI reported an accuracy of 51 for Mistral7B, and Coconut reported an accuracy of 34.1 for GPT2. However, in Figure 3, the accuracies for these baselines on Mistral7B are around 30 and 10, respectively. The authors should verify whether the experimental settings are consistent with those in the original papers of the baseline methods and provide explanations for the performance discrepancies.
3.Early sentences may contain logical premises that are crucial for subsequent reasoning. The presence of redundant sentences and their positions may vary across different tasks (e.g., mathematical reasoning, symbolic reasoning). Focusing solely on removing early sentences might lead to errors in downstream reasoning for certain tasks. The sensitivity of the verbosity metric in complex reasoning chains needs further discussion.
4.According to Table 1, the reduction in token generation by VARR is only about 15%. The authors should provide additional quantitative comparisons with other methods in terms of reasoning efficiency to more comprehensively demonstrate the advantages of their approach.
Comments, suggestions and typos: 1. Figure 1 should be set at the top of the page.
2. Experiments on larger-scale models would better demonstrate the effectiveness of this work. Why were experiments conducted on mistral-7b instead of Llama3.1-8B, considering that both models are of similar scale, but Llama3.1-8B is more widely used?

The authors' method aims to reduce the number of output tokens, which seemingly contrasts with the test-time scaling paradigm where adding more computation (and often more tokens) at inference time typically leads to better performance. This presents an interesting tension that warrants further exploration. A valuable extension of this work would be investigating how VARR/VARR+ could be effectively combined with test-time scaling techniques to maximize the efficacy of each generated token.

1.Include the latest published or publicly available baseline methods as reliable comparison objects.
2.It is recommended to add human evaluation experiments (e.g., human annotation of redundant sentences) to verify whether the removal of redundant sentences aligns with human judgment.
3.In Figure 1, the NLL curves for different configurations are not sufficiently distinguishable. The authors should consider adding legends or using clearer markers to improve the clarity of the figure.
Overall assessment scores: 3.5

4

2