R-Capsule: Compressing High-Level Plans for Efficient Large Language Model Reasoning

## Abstract

Abstract Chain-of-Thought (CoT) prompting helps Large Language Models (LLMs) tackle complex reasoning by eliciting explicit step-by-step rationales. However, CoT’s verbosity increases latency and memory usage and may propagate early errors across long chains. We propose the Reasoning Capsule (R-Capsule) , a framework that aims to combine the efficiency of latent reasoning with the transparency of explicit CoT. The core idea is to compress the high-level plan into a small set of learned latent tokens (a Reasoning Capsule) while keeping execution steps lightweight or explicit. This hybrid approach is inspired by the Information Bottleneck (IB) principle, where we encourage the capsule to be approximately minimal yet sufficient for the task. Minimality is encouraged via a low-capacity bottleneck, which helps improve efficiency. Sufficiency is encouraged via a dual objective: a primary task loss for answer accuracy and an auxiliary plan-reconstruction loss that encourages the capsule to faithfully represent the original textual plan. The reconstruction objective helps ground the latent space, thereby improving interpretability and reducing the use of uninformative shortcuts. Our framework strikes a balance between efficiency, accuracy, and interpretability, thereby reducing the visible token footprint of reasoning while maintaining or improving accuracy on complex benchmarks. Our codes are available at: https://anonymous.4open.science/r/Reasoning-Capsule-7BE0

## 1 Introduction

Large Language Models (LLMs) exhibit strong multi-step reasoning when prompted with Chain-of-Thought (CoT) Wei et al. (2022); Lightman et al. (2023). By instructing models to generate an explicit sequence of intermediate steps, CoT significantly improves performance on tasks ranging from arithmetic and commonsense reasoning to symbolic manipulation. However, explicit chains are costly: generating long sequences increases inference latency and memory usage. Furthermore, these long-form generations are susceptible to cascading errors, where a mistake in an early step compromises the entire reasoning process. As LLMs are increasingly deployed in latency- and cost-sensitive applications, the community has sought alternatives that preserve CoT’s accuracy benefits while reducing its overhead.

Existing approaches to mitigate these issues can be grouped into three broad families, each with distinct trade-offs.
Ensemble and sampling-based methods (e.g., self-consistency Wang et al. (2022); Yao et al. (2023a); Besta et al. (2024); Chen et al. (2025)) improve accuracy by aggregating multiple reasoning chains. While effective, they multiply inference-time cost and do not address verbosity at its root.
Implicit or latent reasoning methods Deng et al. (2024); Hao et al. (2024) compress intermediate computation into dense vectors and decode short outputs, saving tokens. However, they are often opaque: compressing both planning and execution can entangle them, hindering verifiability and inviting shortcuts when the latent channel is unconstrained.

Hierarchical and modular reasoning approaches (e.g., plan-then-solve Huang et al. (2022); Wang et al. (2023)) separate high-level planning from low-level execution. This structure enhances faithfulness and controllability; however, plans are typically generated explicitly in natural language or via tool calls, reintroducing token overhead and sensitivity to exposure bias and plan-execution mismatch.
These limitations motivate a more targeted question: can we obtain the efficiency of latent reasoning without sacrificing the structure and transparency of explicit plans?
We answer this in the affirmative by introducing the Reasoning Capsule. This framework compresses only the high-level plan into a compact, continuous latent while keeping execution lightweight and optionally explicit. Our key observation, supported by systematic experiments on arithmetic and commonsense reasoning benchmarks, is twofold: (1) explicitly generating textual plans before steps often degrades accuracy due to longer sequences and increased opportunities for error; yet (2) compressing the plan into a small number of contiguous latent tokens, while leaving execution uncompressed or lightly decoded, yields consistent gains over generating steps directly, and substantially outperforms compressing the execution itself. In other words, the plan is the right target for compression, whereas compressing the full CoT tends to discard useful inductive biases and supervision signals.

In the latent-planning stage, a decoder-only LLM projects its internal state through a low-capacity bottleneck to produce K capsule tokens. The model then conditions on these tokens (e.g., as soft prompts/prefix) for subsequent generation, replacing explicit plan text. In the execution stage, an auxiliary one-layer transformer (plan decoder) is trained to reconstruct the high-level plan and supervise CoT and answer generation from the capsule. At inference, we skip explicit CoT and directly generate the answer conditioned on the capsule; the auxiliary decoder is used only during training. This design respects the hierarchical nature of reasoning—planning versus execution—while minimizing the visibility of tokens.
To make capsules compact and semantically meaningful, we adopt an IB-inspired design. Tishby et al. (2000). The bottleneck projection enforces minimality by constraining capacity. In contrast, a dual objective enforces sufficiency: a standard next-token loss for answer generation and an auxiliary reconstruction loss that trains the model to recover the high-level textual plan from the capsule. This reconstruction grounds the capsule in an interpretable strategy and counteracts latent collapse, avoiding uninformative shortcuts. Empirically, we find that (i) learning to first generate an explicit plan and then steps often harms performance; (ii) compressing the plan into latent tokens improves over step-first baselines; and (iii) further compressing the steps substantially degrades results—highlighting the asymmetry between plan and execution in what should be compressed.
We validate our approach on arithmetic benchmarks and commonsense reasoning benchmarks. Across datasets, Reasoning Capsules deliver competitive or improved accuracy with fewer visible tokens and reduced latency compared to explicit CoT. They outperform entirely latent CoT schemes that compress both plan and steps. Ablations varying capsule length and bottleneck dimension illustrate a robust accuracy–efficiency trade-off. Qualitative analyses indicate that decoded plans remain faithful, and the model’s attention is concentrated on capsule tokens during execution. Our contributions are threefold:

- •
We introduce Reasoning Capsules, a framework that compresses high-level plans into compact latent tokens to drive downstream execution, reconciling the efficiency of latent reasoning with the structure and interpretability of explicit plans.
- •
We provide a principled grounding via the Information Bottleneck, and instantiate it with an architectural bottleneck plus a plan-reconstruction objective that yields minimal yet sufficient, semantically grounded latents.
- •
We present a practical training recipe that integrates with GPT-based decoders and a lightweight one-layer decoder for supervision, along with comprehensive experiments on arithmetic and commonsense reasoning showing consistent token savings, latency reductions, and accuracy gains over explicit-plan and entirely latent CoT baselines.

## 2 Methodology

In this paper, we introduce Reasoning Capsule, a framework designed to enhance the efficiency and accuracy of multi-step reasoning in Large Language Models (LLMs). The core idea is to compress high-level strategic plans into compact, continuous latent representations. This approach mitigates the computational and statistical inefficiencies of generating verbose textual reasoning chains. We first present the overall framework, then provide a theoretical justification from the perspective of the Information Bottleneck principle, and finally detail the training objective.

### 2.1 From Chain-of-Thought to Latent Planning

Standard Chain-of-Thought (CoT) tackles a problem $Q$ by generating an explicit sequence of reasoning steps $S=(s_{1},s_{2},\dots,s_{N})$ before producing a final answer $A$. The whole generation process is modeled as $p(S,A|Q)$. While effective, generating the explicit sequence $S$ token by token is computationally expensive and can introduce cascading errors.

Our key insight is that reasoning chains often exhibit a hierarchical structure: a high-level strategic plan (e.g., “first, calculate the discount; then, compute the final price”) followed by low-level, step-by-step execution (e.g., “$\langle\langle 100\times 0.2=20\rangle\rangle$”, “$\langle\langle 100-20=80\rangle\rangle$”). We hypothesize that the high-level plan is a primary candidate for compression, as its semantic essence is more critical than its specific wording for guiding the final solution.

We therefore propose Latent Planning, a paradigm where the LLM first generates a compact latent representation of the high-level plan—the Reasoning Capsule—and then conditions on this capsule to execute the low-level reasoning steps. Let the explicit high-level plan be $P$ and the subsequent execution steps be $S_{\text{exec}}$. Our approach bifurcates the reasoning process:

Latent Planning Stage: Given $Q$, the model generates a set of capsules $C=\{c_{1},\dots,c_{K}\}$ that encode the high-level strategy originally articulated in $P$. This stage models $p(C|Q)$.

Conditioned Execution Stage: The model generates the execution steps $S_{\text{exec}}$ and the final answer $A$ conditioned on both the question $Q$ and the latent plan $C$. This stage models $p(S_{\text{exec}},A|Q,C)$.

The overall generative process is thus factorized as $p(S_{\text{exec}},A|Q,C)p(C|Q)$.

### 2.2 Architecture: Generating and Utilizing Reasoning Capsules

Our architecture is built upon a standard decoder-only transformer. We introduce a mechanism to generate and consume Reasoning Capsules within the forward pass (see Figure [1](https://arxiv.org/html/2509.22131v2#S2.F1)).

##### Capsule Generation.

To generate a capsule, we prompt the model to emit a special ‘[CAPSULE]‘ token at the point where a textual plan would typically be articulated. The hidden state $h_{t}\in\mathbb{R}^{D}$ from the final transformer layer corresponding to this token is used as input to a bottleneck network. This network projects the high-dimensional hidden state into a low-dimensional capsule $c\in\mathbb{R}^{d}$, where $d\ll D$:

$$ $c=\text{Proj}(h_{t})=W_{p}h_{t}+b_{p},$ (1) $$

where $W_{p}\in\mathbb{R}^{d\times D}$ and $b_{p}\in\mathbb{R}^{d}$ are learnable parameters. This projection acts as a structural implementation of the compression objective in the Information Bottleneck principle, forcing the model to distill the most salient strategic information from the context-rich hidden state $h_{t}$.

##### Conditioning on Capsules.

Once generated, the capsule $c$ must guide subsequent reasoning. We project the capsule back into the model’s input embedding space using a separate linear transformation. This projected embedding is then fed as input to the transformer at the beginning of the execution stage. This allows the capsule to condition the generation of all subsequent tokens ($S_{\text{exec}}$ and $A$), effectively acting as a compact, latent instruction that steers the model’s computations.

Figure: Figure 1: A conceptual diagram of our Reasoning Capsule framework. The LLM generates a compact latent capsule representing the high-level plan, which is passed through a bottleneck. This capsule conditions the subsequent generation of the execution steps and the final answer. An auxiliary reconstruction decoder ensures the capsule is semantically grounded by forcing it to reconstruct the original textual plan, guided by the Information Bottleneck principle.
Refer to caption: x1.png

### 2.3 Theoretical Grounding: The Information Bottleneck Perspective

A key challenge in latent variable models is ensuring the representations are both compressed and meaningful. Our design is formally motivated by the Information Bottleneck (IB) principle Tishby et al. (2000). The IB principle provides a framework for learning a compressed representation $Z$ of a source variable $X$ that is maximally informative about a target variable $Y$. The objective is to learn a mapping $p(Z|X)$ that maximizes the Lagrangian $\mathcal{L}_{\text{IB}}=I(Z;Y)-\beta I(X;Z)$, where $I(\cdot;\cdot)$ denotes mutual information and $\beta$ is a Lagrange multiplier.
The source variable $X$ is the hidden state $h_{t}$, which contains rich, high-bandwidth information about the question $Q$ and reasoning context.
The compressed representation $Z$ is the Reasoning Capsule $c$.
The target variable $Y$ is the information required to solve the task, i.e., the execution steps and final answer $(S_{\text{exec}},A)$.

The goal is to learn a capsule $c$ that is a minimal sufficient statistic for the reasoning task.

Minimality (Compression): The capsule $c$ must be a compressed version of $h_{t}$. This corresponds to minimizing the mutual information $I(h_{t};c)$, which forces the model to discard irrelevant information like specific phrasing or syntactic variations. Our bottleneck architecture (Eq. [1](https://arxiv.org/html/2509.22131v2#S2.E1)) directly serves this goal. By projecting $h_{t}\in\mathbb{R}^{D}$ into a low-dimensional space $c\in\mathbb{R}^{d}$ where $d\ll D$, we constrain the information capacity of the capsule, providing a strong inductive bias for compression.

Sufficiency (Informativeness): The capsule $c$ must retain all information from $h_{t}$ that is relevant for producing the correct solution by maximizing the mutual information $I(c;S_{\text{exec}},A)$.

Directly optimizing $I(c;S_{\text{exec}},A)$ is intractable. We instead use the original high-level textual plan, $P$, as an effective proxy. We hypothesize that $P$ encapsulates the core strategic information needed for the task. Therefore, we aim to maximize $I(c;P)$ as a surrogate objective for sufficiency. This ensures that the latent capsule is semantically grounded in the human-interpretable reasoning plan. Our training objective, detailed next, is a practical realization of this IB-based formulation.

### 2.4 Grounded Training Objective

To operationalize the IB principle, we train the model with a multi-task objective that balances task performance (sufficiency for the answer) and representational fidelity (sufficiency for the plan). The total loss $\mathcal{L}$ is a weighted sum of an execution loss and a plan reconstruction loss,

$$ $\mathcal{L}=\mathcal{L}_{\text{exec}}+\lambda\mathcal{L}_{\text{recon}},$ (2) $$

where $\lambda$ is a hyperparameter balancing the two objectives.

Execution Loss ($\mathcal{L}_{\text{exec}}$). This is the primary task loss, ensuring the capsule is sufficient for solving the problem. It is a standard auto-regressive cross-entropy loss for generating the target sequence $T=(S_{\text{exec}},A)$, which includes both the intermediate execution steps and the final answer. The generation is conditioned on the question $Q$ and the generated set of capsules $C$:

$$ $\mathcal{L}_{\text{exec}}=-\log p(T|Q,C).$ (3) $$

Minimizing this loss implicitly maximizes the mutual information $I(C;T)$, encouraging the capsules to be directly helpful for the downstream task.

Reconstruction Loss ($\mathcal{L}_{\text{recon}}$). This auxiliary loss serves as our practical method for maximizing $I(C;P)$, grounding the latent space and ensuring interpretability. We employ a separate, shallow transformer decoder that takes the sequence of capsules $C$ as input and is trained to reconstruct the original high-level textual plan $P$:

$$ $\mathcal{L}_{\text{recon}}=-\log p(P|C).$ (4) $$

This loss forces each capsule to encode sufficient information to recover its corresponding textual plan component, ensuring the latent plan is a faithful and high-fidelity representation of the explicit one. This prevents the model from learning uninterpretable latent shortcuts, making the reasoning process more robust.

By combining these components, our framework learns to form compact, efficient, and semantically meaningful plans, thereby practically and effectively realizing the goals of the Information Bottleneck principle.

## 3 Experiments

To validate the effectiveness and efficiency of our Reasoning Capsule framework, we conduct a comprehensive set of experiments on various reasoning benchmarks. Our evaluation is designed to answer several key research questions that stem from the claims made in our methodology.

- •
RQ1 (Effectiveness): Does our Reasoning Capsule framework outperform strong baselines, exceptionally standard Chain-of-Thought fine-tuning (CoT-SFT), in terms of reasoning accuracy?
- •
RQ2 (Generalizability): Is the performance improvement of Reasoning Capsules consistent across mathematical and commonsense reasoning domains?
- •
RQ3 (Scalability): How does the benefit of our latent planning approach scale with the size of the base language model?
- •
RQ4 (Efficiency): Does the latent planning paradigm lead to a more compact and efficient reasoning process, measured by generation length and latency?
- •
RQ5 (Interpretability): Do the latent capsules encode genuine, verifiable planning information?

### 3.1 Experimental Setup

#### 3.1.1 Datasets

We evaluate our method on standard benchmarks for mathematical and commonsense reasoning.

- •
Mathematical Reasoning: GSM8K Cobbe et al. (2021) (grade-school math word problems), MultiArith Roy & Roth (2016) (math problems requiring multiple reasoning steps), and AQuA Ling et al. (2017) (multiple-choice algebraic word problems).
- •
Commonsense Reasoning: StrategyQA Geva et al. (2021) (yes/no questions requiring a multi-step reasoning strategy) and CommonsenseQA 2.0 (CSQA2) Talmor et al. (2018; 2022) (a challenging multiple-choice QA dataset requiring prior knowledge).

#### 3.1.2 CoT Data Generation

We generate the CoT data using the gpt-o3 model via few-shot prompting. For each problem, we prompt the model to produce a solution plan and step-by-step reasoning. During this process, the final answer is withheld from the model. We employ rollout sampling with a temperature of 1.0 and repeat the generation up to 10 times per problem, stopping once a process yielding the correct answer is found. If all 10 attempts fail to produce a proper solution, we then provide the model with the correct answer and prompt it to generate a valid reasoning path, including the plan and steps. The generated prompts and cases are provided in Appendix [B](https://arxiv.org/html/2509.22131v2#A2).

#### 3.1.3 Base Models

To test the scalability (RQ3), we conduct experiments on three decoder-only transformer models of varying sizes: GPT-2 (0.2B) Radford et al. (2019), LLaMA-3 (1B) Dubey et al. (2024), LLaMA-3.1 (7B) Dubey et al. (2024), Qwen-3 (8B) Yang et al. (2025). All methods are fine-tuned on the same pre-trained checkpoints for a fair comparison.

#### 3.1.4 Baselines

We compare our Reasoning Capsule framework against a series of strong baselines:

Standard SFT (w/o CoT): A standard supervised fine-tuning baseline where the model is trained to directly predict the final answer A from the question Q, that is, modeling p (AQ). This establishes the performance without any explicit reasoning steps.

CoT-SFT: The standard Chain-of-Thought fine-tuning approach Wei et al. (2022). The model is trained to generate the complete textual reasoning chain S followed by the final answer A, modeling p (S, AQ). This is our main and strongest baseline.

Coconut: A method that improves reasoning by generating multiple reasoning paths and using a verifier to select the most consistent one, thereby enhancing robustness Hao et al. (2024).

iCoT:
a method that allows language models to gradually internalize chain-of-thought (CoT) reasoning steps by incrementally removing intermediate CoT tokens and fine-tuning, thereby achieving implicit CoT reasoning with high accuracy and fast inferenceDeng et al. (2024).

Plan-SFT:
a unified post-training framework that distills synthetic ”planning trajectories” (task decompositions) from large-scale LLMs and fine-tunes smaller open-source LLMs via supervised fine-tuningParmar et al. (2025).

#### 3.1.5 Implementation Details

We employ the AdamW optimizer with a learning rate of $5\times 10^{-6}$, where $\beta_{1}$ and $\beta_{2}$ are set to 0.9 and 0.999, respectively, and the weight decay is 0.01. The learning rate follows a cosine schedule with a linear warmup over the first 10% of total training steps. We use a total batch size of 32 (4 per GPU across 8 NVIDIA A800 GPUs) without gradient accumulation.
The training epochs are set differently for various models: 5 epochs for GPT2, and 3 epochs for LLaMA3-1B, LLaMA3-7B, and Qwen3-8B. The length of the Reasoning Capsule is fixed at 2. For the loss function, the reconstruction loss (adopting MSE loss) is weighted by $\lambda=0.5$ against the main task loss.

### 3.2 Main Results (RQ1 & RQ2)

**Table 1: Main results on mathematical and commonsense reasoning benchmarks. We report accuracy (%) on five datasets for methods applied to GPT-2 (115M) and LLaMA-3 (1B) models. Our R-Capsule consistently outperforms the strong CoT-SFT baseline.**
|  | Mathematical Reasoning | Commonsense Reasoning |  |  |  |
| --- | --- | --- | --- | --- | --- |
| Method | GSM8K | MultiArith | AQuA | StrategyQA | CSQA2 |
| Model: GPT-2 (150M) |  |  |  |  |  |
| Standard SFT | 19.1 | 78.5 | 28.1 | – | – |
| CoT-SFT | 42.9 | 86.9 | 33.2 | – | – |
| Plan-SFT | 37.5 | 82.6 | 30.9 | – | – |
| Coconut | 34.1 | 84.8 | 32.8 | – | – |
| iCoT | 41.5 | 85.2 | 33.0 | – | – |
| R-Capsule (Ours) | 46.2 | 92.4 | 37.9 | – | – |
| Model: LLaMA-3 (1B) |  |  |  |  |  |
| Standard SFT | 44.1 | 89.0 | 35.5 | 60.5 | 55.1 |
| CoT-SFT | 59.7 | 94.1 | 48.4 | 62.9 | 57.2 |
| Plan-SFT | 59.7 | 94.1 | 44.8 | 63.4 | 56.5 |
| R-Capsule (Ours) | 63.8 | 96.5 | 52.1 | 66.8 | 59.8 |

The results in Table [1](https://arxiv.org/html/2509.22131v2#S3.T1) demonstrate the effectiveness and generalizability of our Reasoning Capsule framework (RQ1 and RQ2). Across both GPT-2 and LLaMA-3 (1B) backbones, our method consistently surpasses the strong CoT-SFT baseline on all five mathematical and commonsense reasoning benchmarks. For instance, on GSM8K, R-Capsule provides a +3.3% and +4.1% absolute improvement for GPT-2 and LLaMA-3, respectively. This confirms the core benefits of our latent planning approach on established model sizes.

### 3.3 Scalability Analysis (RQ3)

To address our third research question (RQ3) concerning the scalability of our approach, we conducted a focused evaluation on two contemporary 7-billion-parameter models: LLaMA-3-7B and Qwen3-8B. In this analysis, we compare our R-Capsule against the CoT-SFT baseline on the GSM8K (mathematical reasoning) and StrategyQA (commonsense reasoning) benchmarks. This enables us to evaluate whether the performance gains of our method scale effectively with model size across various reasoning domains.

Figure: (a) Performance of models with increasing parameter counts. R-Capsule maintains and, in some cases, enhances its accuracy gains at larger scales.
Refer to caption: figure/scalability_analysis_square_layout.png

The results, presented in Figure [2(a)](https://arxiv.org/html/2509.22131v2#S3.F2.sf1), demonstrate that the advantages of the R-Capsule framework persist and are even amplified on these larger models.
For instance, on LLaMA-3-7B, R-Capsule achieves a significant absolute improvement of 2.7% on GSM8K and 2.3% on StrategyQA over the CoT-SFT baseline.
A similar trend is observed on Qwen3-8B, where our method yields a notable improvement of 3.27% on GSM8K and 3.3% on StrategyQA.
These consistent gains across two distinct and powerful foundation models strongly suggest that the structural benefits of latent planning, as embodied by our R-Capsule, represent a general principle that scales effectively with model capability. This finding provides a robust affirmative answer to RQ3.

### 3.4 Ablation Study: Where to Compress

We ablate which part of the reasoning process to compress. We augment the reasoning chain with an explicit textual plan, creating two components: the plan and the steps. We test four variants:

- •
Plan-SFT: Explicit plan $\rightarrow$ explicit steps (no compression).
- •
R-Capsule (Plan-only): Latent plan $\rightarrow$ explicit steps. This is our main proposal.
- •
R-Capsule (Steps-only): Latent steps, analogous to implicit CoT.
- •
R-Capsule (Plan+Steps): Latent plan $\rightarrow$ latent steps (maximal compression).

Figure [2(b)](https://arxiv.org/html/2509.22131v2#S3.F2.sf2) shows that compressing only the plan (R-Capsule (Plan-only)) achieves the best accuracy-efficiency trade-off. It improves accuracy over the explicit Plan-SFT baseline while reducing generated tokens by over 60% (e.g., from 244 to 86 on Qwen3 8B). This suggests that encoding the high-level plan latently provides a robust guide for generating explicit, low-level steps. In contrast, compressing the detailed steps (Steps-only or Plan+Steps) is less effective, with the latter showing a drop in accuracy despite achieving maximum compression. Finally, ablating the projection layer or the weak decoder leads to significant performance degradation (e.g., -7.4% acc. on Qwen3 without the weak decoder), confirming their architectural importance.

### 3.5 Efficiency and Length Analysis (RQ4)

**Table 2: Token budget and latency comparison on GSM8K (Qwen3). Latency is measured with a batch size of 1 on the A100.**
| Method | Tokens to Answer | Compression Ratio | Latency (s) |
| --- | --- | --- | --- |
| Explicit Plan-CoT | 447 | 1.0 | 3.12 |
| R-Capsule (Plan-Latent) | 232 | 0.52 | 1.47 |

To address our fourth research question (RQ4) concerning efficiency, we analyze the generation length and inference latency of our proposed method. As detailed in Table [2](https://arxiv.org/html/2509.22131v2#S3.T2), we evaluate our R-Caps (Plan-Latent) approach against the Explicit Plan-CoT baseline on the GSM8K benchmark. We measure two key metrics: the total number of tokens generated to reach the final answer and the end-to-end inference latency on a single A100 GPU.
The results demonstrate a substantial improvement in efficiency. Our R-Capsule (Plan-Latent) method requires only 232 tokens to derive an answer, marking a 48% reduction compared to the 447 tokens used by the baseline. This corresponds to a compression ratio of 0.52, indicating that our method can produce solutions that are nearly half the length of standard explicit reasoning chains.
This significant reduction in token generation directly translates to a notable decrease in latency. The inference time drops from 3.12 seconds for Explicit Plan-CoT to just 1.47 seconds for our method, achieving a 2.12x speedup. This efficiency gain stems from our model’s ability to operate on compact latent representations of the plan, bypassing the need to generate verbose, token-intensive intermediate steps.

### 3.6 Interpretability of Latent Capsules (RQ5)

To validate our core hypothesis, that the bottleneck architecture distills a plan into a compact, abstract latent representation, we investigate the information encoded within these latent capsules. By analyzing the output of a weak decoder fed with a corresponding capsule, we aim to provide qualitative evidence that these capsules preserve the plan’s essential logical structure, effectively functioning as abstract ’latent thoughts’ rather than mere textual compressions.
As the case demonstrates, the output from the weak decoder (weak decoder plan) is significantly more concise than the original plan. It strips away redundant descriptive language, distilling the core steps into direct. This provides strong evidence for our central hypothesis: the bottleneck architecture incentivizes the model not merely to compress the plan, but to compile it into an abstract computational graph.

Further analysis (Appendix [C](https://arxiv.org/html/2509.22131v2#A3)) supports these findings:

- •
Vocabulary Distribution: Projecting latent tokens into the vocabulary space reveals a focus on abstract verbs (e.g., ‘calculate‘, ‘total‘) rather than specific numbers, indicating they capture high-level intent.
- •
Attention Analysis: The decoder attends heavily to the latent plan tokens when generating subsequent calculation steps, confirming they serve as a guide.

This evidence confirms that latent capsules are most effective for representing high-level strategic plans, while explicit generation remains crucial for detailed, procedural steps.

## 4 Related Work

Chain-of-Thought Prompting
Chain-of-Thought (CoT) promptingWei et al. (2022) has been shown to significantly enhance performance on complex tasks by generating explicit, step-by-step reasoning tracesSun et al. (2024). Variants like self-consistency aggregationWang et al. (2022) further improve reliability by ensembling multiple reasoning chains. Representative examples of this paradigm include o1El-Kishky (2024) and DeepSeek R1DeepSeek-AI et al. (2025)—both reasoning models that have achieved strong performance. Collectively, these findings confirm that CoT prompting effectively addresses the challenges of complex task reasoning: its explicit step-by-step trace design breaks down intricate problems into manageable logical segments, while variant optimizations (e.g., tree of thoughtsYao et al. (2023b),self-refineYang et al. (2024)) mitigate uncertainty in reasoning processes. The strong performance of models such as o1 and DeepSeek R1 further validates that the CoT paradigm is not only theoretically sound but also practically impactful, becoming a foundational approach for enhancing reasoning capabilities in advanced models.

Latent Planning and the Information Bottleneck
However, the verbosity of CoT not only increases inference latency and memory overheadHong et al. (2025) but also risks propagating errors across long reasoning sequences. Implicit or latent reasoning schemesDeng et al. (2023); Ye et al. (2025) compress intermediate reasoning steps into dense vectorsHao et al. (2024), enabling faster inferenceCheng & Durme (2024) but at the cost of reduced interpretability. Without explicit grounding, such representations may encode spurious shortcuts. By contrast, we leverage the Information Bottleneck principleTishby et al. (2000) to achieve two key goals: (i) enforcing minimality via a low-dimensional bottleneck, and (ii) ensuring sufficiency through an auxiliary reconstruction loss that recovers the original high-level plan. This dual objective guarantees that each *Reasoning Capsule* is both compact and semantically faithful. For additional details or supplementary materials about the content discussed in this section/chapter, readers are kindly referred to Appendix [D](https://arxiv.org/html/2509.22131v2#A4).

Hierarchical and Modular Reasoning
Hierarchical reasoning frameworks decouple planning from execution, e.g., in plan-and-solve prompting or modular CoT Parmar et al. (2025). Tool-oriented methods (e.g., ToolformerSchick et al. (2023), ReActYao et al. (2023c)) similarly structure the reasoning process into tool selection and execution. These approaches, however, rely on generating and parsing explicit plans or tool invocation commandsHao et al. (2023), which introduces extra computational overheadWang et al. (2024). Our method internalizes high-level planning within a latent space, eliminating the need for explicit plan text or tool-specific syntax. A subsequent reconstruction module then verifies that this latent plan accurately encapsulates the intended reasoning strategy, thereby unifying structural rigor and inference efficiency within a single model.

## 5 Conclusion

In this paper, we introduce Reasoning Capsules (R-Capsule), a hybrid framework that reconciles the efficiency of latent reasoning with the transparency of explicit CoT. Our key insight is to decouple high-level planning from low-level execution, compressing only the strategic plan into a compact set of latent tokens—the capsule.
Grounded in the Information Bottleneck principle, our method enforces the capsule to be both minimal, by discarding redundant information via a low-capacity bottleneck, and sufficient, by optimizing a dual objective for task accuracy and plan reconstruction. This design preserves the high-level reasoning structure while drastically reducing token overhead.
Extensive experiments on mathematical (e.g., GSM8K) and commonsense (e.g., StrategyQA) reasoning benchmarks demonstrate that R-Capsule significantly outperforms strong baselines across various model sizes. Ablation and interpretability studies confirm that our approach of compressing only the plan yields an optimal trade-off and that the capsules encode meaningful strategic intent. R-Capsule establishes that targeted compression of high-level plans is a principled and effective path toward efficient, accurate, and interpretable LLM reasoning.

## References

- Besta et al. (2024)
Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michal
Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Hubert
Niewiadomski, Piotr Nyczyk, et al.
Graph of thoughts: Solving elaborate problems with large language
models.
In *Proceedings of the AAAI conference on artificial
intelligence*, volume 38, pp. 17682–17690, 2024.
- Chen et al. (2025)
Qiguang Chen, Libo Qin, Jinhao Liu, Dengyun Peng, Jiannan Guan, Peng Wang,
Mengkang Hu, Yuhang Zhou, Te Gao, and Wanxiang Che.
Towards reasoning era: A survey of long chain-of-thought for
reasoning large language models.
*arXiv preprint arXiv:2503.09567*, 2025.
- Cheng & Durme (2024)
Jeffrey Cheng and Benjamin Van Durme.
Compressed chain of thought: Efficient reasoning through dense
representations, 2024.
URL [https://arxiv.org/abs/2412.13171](https://arxiv.org/abs/2412.13171).
- Cobbe et al. (2021)
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz
Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano,
et al.
Training verifiers to solve math word problems.
*arXiv preprint arXiv:2110.14168*, 2021.
- DeepSeek-AI et al. (2025)
DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang,
Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang,
Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao,
Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu,
Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Deli Chen,
Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao,
Guanting Chen, Guowei Li, H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang,
Honghui Ding, Huajian Xin, Huazuo Gao, Hui Qu, Hui Li, Jianzhong Guo, Jiashi
Li, Jiawei Wang, Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, J. L.
Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaige Gao, Kang Guan,
Kexin Huang, Kuai Yu, Lean Wang, Lecong Zhang, Liang Zhao, Litong Wang, Liyue
Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Meng
Li, Miaojun Wang, Mingming Li, Ning Tian, Panpan Huang, Peng Zhang, Qiancheng
Wang, Qinyu Chen, Qiushi Du, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang,
R. J. Chen, R. L. Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen,
Shengfeng Ye, Shiyu Wang, Shuiping Yu, Shunfeng Zhou, Shuting Pan, S. S. Li,
Shuang Zhou, Shaoqing Wu, Shengfeng Ye, Tao Yun, Tian Pei, Tianyu Sun,
T. Wang, Wangding Zeng, Wanjia Zhao, Wen Liu, Wenfeng Liang, Wenjun Gao,
Wenqin Yu, Wentao Zhang, W. L. Xiao, Wei An, Xiaodong Liu, Xiaohan Wang,
Xiaokang Chen, Xiaotao Nie, Xin Cheng, Xin Liu, Xin Xie, Xingchao Liu, Xinyu
Yang, Xinyuan Li, Xuecheng Su, Xuheng Lin, X. Q. Li, Xiangyue Jin, Xiaojin
Shen, Xiaosha Chen, Xiaowen Sun, Xiaoxiang Wang, Xinnan Song, Xinyi Zhou,
Xianzu Wang, Xinxia Shan, Y. K. Li, Y. Q. Wang, Y. X. Wei, Yang Zhang,
Yanhong Xu, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Wang, Yi Yu, Yichao Zhang,
Yifan Shi, Yiliang Xiong, Ying He, Yishi Piao, Yisong Wang, Yixuan Tan,
Yiyang Ma, Yiyuan Liu, Yongqiang Guo, Yuan Ou, Yuduan Wang, Yue Gong, Yuheng
Zou, Yujia He, Yunfan Xiong, Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang
Zhou, Y. X. Zhu, Yanhong Xu, Yanping Huang, Yaohui Li, Yi Zheng, Yuchen Zhu,
Yunxian Ma, Ying Tang, Yukun Zha, Yuting Yan, Z. Z. Ren, Zehui Ren, Zhangli
Sha, Zhe Fu, Zhean Xu, Zhenda Xie, Zhengyan Zhang, Zhewen Hao, Zhicheng Ma,
Zhigang Yan, Zhiyu Wu, Zihui Gu, Zijia Zhu, Zijun Liu, Zilin Li, Ziwei Xie,
Ziyang Song, Zizheng Pan, Zhen Huang, Zhipeng Xu, Zhongyu Zhang, and Zhen
Zhang.
Deepseek-r1: Incentivizing reasoning capability in llms via
reinforcement learning, 2025.
URL [https://arxiv.org/abs/2501.12948](https://arxiv.org/abs/2501.12948).
- Deng et al. (2023)
Yuntian Deng, Kiran Prasad, Roland Fernandez, Paul Smolensky, Vishrav
Chaudhary, and Stuart Shieber.
Implicit chain of thought reasoning via knowledge distillation, 2023.
URL [https://arxiv.org/abs/2311.01460](https://arxiv.org/abs/2311.01460).
- Deng et al. (2024)
Yuntian Deng, Yejin Choi, and Stuart Shieber.
From explicit cot to implicit cot: Learning to internalize cot step
by step.
*arXiv preprint arXiv:2405.14838*, 2024.
- Dubey et al. (2024)
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad
Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan,
et al.
The llama 3 herd of models.
*arXiv e-prints*, pp. arXiv–2407, 2024.
- El-Kishky (2024)
Ahmed El-Kishky.
Openai o1 system card.
*ArXiv*, abs/2412.16720, 2024.
URL [https://api.semanticscholar.org/CorpusID:272648256](https://api.semanticscholar.org/CorpusID:272648256).
- Geva et al. (2021)
Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan
Berant.
Did aristotle use a laptop? a question answering benchmark with
implicit reasoning strategies.
*Transactions of the Association for Computational Linguistics*,
9:346–361, 2021.
- Hao et al. (2023)
Shibo Hao, Yi Gu, Haodi Ma, Joshua Jiahua Hong, Zhen Wang, Daisy Zhe Wang, and
Zhiting Hu.
Reasoning with language model is planning with world model, 2023.
URL [https://arxiv.org/abs/2305.14992](https://arxiv.org/abs/2305.14992).
- Hao et al. (2024)
Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston,
and Yuandong Tian.
Training large language models to reason in a continuous latent
space.
*arXiv preprint arXiv:2412.06769*, 2024.
- Hong et al. (2025)
Jialiang Hong, Taihang Zhen, Kai Chen, Jiaheng Liu, Wenpeng Zhu, Jing Huo, Yang
Gao, Depeng Wang, Haitao Wan, Xi Yang, Boyan Wang, and Fanyu Meng.
Reconsidering overthinking: Penalizing internal and external
redundancy in cot reasoning, 2025.
URL [https://arxiv.org/abs/2508.02178](https://arxiv.org/abs/2508.02178).
- Huang et al. (2022)
Wenlong Huang, Pieter Abbeel, Deepak Pathak, and Igor Mordatch.
Language models as zero-shot planners: Extracting actionable
knowledge for embodied agents.
In *International conference on machine learning*, pp. 9118–9147. PMLR, 2022.
- Lightman et al. (2023)
Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker,
Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe.
Let’s verify step by step.
In *The Twelfth International Conference on Learning
Representations*, 2023.
- Ling et al. (2017)
Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom.
Program induction by rationale: Learning to solve algebraic word
problems.
*arXiv preprint arXiv:1705.04146*, 2017.
- Parmar et al. (2025)
Mihir Parmar, Palash Goyal, Xin Liu, Yiwen Song, Mingyang Ling, Chitta Baral,
Hamid Palangi, and Tomas Pfister.
Plan-tuning: Post-training language models to learn step-by-step
planning for complex problem solving.
2025.
- Radford et al. (2019)
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya
Sutskever, et al.
Language models are unsupervised multitask learners.
*OpenAI blog*, 1(8):9, 2019.
- Roy & Roth (2016)
Subhro Roy and Dan Roth.
Solving general arithmetic word problems.
*arXiv preprint arXiv:1608.01413*, 2016.
- Schick et al. (2023)
Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli,
Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom.
Toolformer: Language models can teach themselves to use tools, 2023.
URL [https://arxiv.org/abs/2302.04761](https://arxiv.org/abs/2302.04761).
- Sun et al. (2024)
Jiankai Sun, Chuanyang Zheng, Enze Xie, Zhengying Liu, Ruihang Chu, Jianing
Qiu, Jiaqi Xu, Mingyu Ding, Hongyang Li, Mengzhe Geng, Yue Wu, Wenhai Wang,
Junsong Chen, Zhangyue Yin, Xiaozhe Ren, Jie Fu, Junxian He, Wu Yuan, Qi Liu,
Xihui Liu, Yu Li, Hao Dong, Yu Cheng, Ming Zhang, Pheng Ann Heng, Jifeng Dai,
Ping Luo, Jingdong Wang, Ji-Rong Wen, Xipeng Qiu, Yike Guo, Hui Xiong, Qun
Liu, and Zhenguo Li.
A survey of reasoning with foundation models, 2024.
URL [https://arxiv.org/abs/2312.11562](https://arxiv.org/abs/2312.11562).
- Talmor et al. (2018)
Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant.
Commonsenseqa: A question answering challenge targeting commonsense
knowledge.
*arXiv preprint arXiv:1811.00937*, 2018.
- Talmor et al. (2022)
Alon Talmor, Ori Yoran, Ronan Le Bras, Chandra Bhagavatula, Yoav Goldberg,
Yejin Choi, and Jonathan Berant.
Commonsenseqa 2.0: Exposing the limits of ai through gamification.
*arXiv preprint arXiv:2201.05320*, 2022.
- Tishby et al. (2000)
Naftali Tishby, Fernando C Pereira, and William Bialek.
The information bottleneck method.
*arXiv preprint physics/0004057*, 2000.
- Wang et al. (2024)
Evan Z Wang, Federico Cassano, Catherine Wu, Yunfeng Bai, William Song, Vaskar
Nath, Ziwen Han, Sean M. Hendryx, Summer Yue, and Hugh Zhang.
Planning in natural language improves LLM search for code
generation.
In *The First Workshop on System-2 Reasoning at Scale,
NeurIPS’24*, 2024.
URL [https://openreview.net/forum?id=B2iSfPNj49](https://openreview.net/forum?id=B2iSfPNj49).
- Wang et al. (2023)
Lei Wang, Wanyu Xu, Yihuai Lan, Zhiqiang Hu, Yunshi Lan, Roy Ka-Wei Lee, and
Ee-Peng Lim.
Plan-and-solve prompting: Improving zero-shot chain-of-thought
reasoning by large language models.
*arXiv preprint arXiv:2305.04091*, 2023.
- Wang et al. (2022)
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang,
Aakanksha Chowdhery, and Denny Zhou.
Self-consistency improves chain of thought reasoning in language
models.
*arXiv preprint arXiv:2203.11171*, 2022.
- Wei et al. (2022)
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V
Le, Denny Zhou, et al.
Chain-of-thought prompting elicits reasoning in large language
models.
*Advances in neural information processing systems*,
35:24824–24837, 2022.
- Yang et al. (2025)
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen
Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al.
Qwen3 technical report.
*arXiv preprint arXiv:2505.09388*, 2025.
- Yang et al. (2024)
Yuqing Yang, Yan Ma, and Pengfei Liu.
Weak-to-strong reasoning.
In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.),
*Findings of the Association for Computational Linguistics: EMNLP 2024*,
pp. 8350–8367, Miami, Florida, USA, November 2024. Association for
Computational Linguistics.
doi: 10.18653/v1/2024.findings-emnlp.490.
URL [https://aclanthology.org/2024.findings-emnlp.490/](https://aclanthology.org/2024.findings-emnlp.490/).
- Yao et al. (2023a)
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L Griffiths, Yuan Cao,
and Karthik Narasimhan.
Tree of thoughts: Deliberate problem solving with large language
models, 2023.
*URL https://arxiv. org/abs/2305.10601*, 3:1,
2023a.
- Yao et al. (2023b)
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan
Cao, and Karthik Narasimhan.
Tree of thoughts: Deliberate problem solving with large language
models, 2023b.
URL [https://arxiv.org/abs/2305.10601](https://arxiv.org/abs/2305.10601).
- Yao et al. (2023c)
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan,
and Yuan Cao.
React: Synergizing reasoning and acting in language models,
2023c.
URL [https://arxiv.org/abs/2210.03629](https://arxiv.org/abs/2210.03629).
- Ye et al. (2025)
Jiaran Ye, Zijun Yao, Zhidian Huang, Liangming Pan, Jinxin Liu, Yushi Bai, Amy
Xin, Liu Weichuan, Xiaoyin Che, Lei Hou, and Juanzi Li.
How does transformer learn implicit reasoning?, 2025.
URL [https://arxiv.org/abs/2505.23653](https://arxiv.org/abs/2505.23653).

## Appendix A Use of Large Language Models

We utilized Large Language Models (LLMs), such as Gemini-2.5-Pro, during the preparation of this manuscript. The usage was twofold: 1) for polishing the language, which included correcting grammatical errors and improving sentence clarity; and 2) as a brainstorming partner to discuss and refine technical details. The authors retained complete control over the content, and all final ideas, claims, and text are our own. We take full responsibility for the entire paper.

## Appendix B Chain-of-Thought Data Generation Details

To ensure the reproducibility of our Chain-of-Thought (CoT) training data, this section provides the complete few-shot prompt template used for generating high-level plans and step-by-step execution steps via GPT-o3, along with representative examples of the generated outputs.

### B.1 Few-Shot Prompt Template for CoT Generation

The prompt template follows a structured format to guide GPT-o3 in distinguishing between high-level plans (task decomposition logic without specific calculations) and low-level execution steps (concrete mathematical operations or reasoning details). The template includes explicit instructions and two reference examples, as shown below:

### B.2 Representative Generated CoT Examples

We provide two complete examples of CoT outputs generated using the above template, corresponding to a mathematical reasoning problem (from GSM8K) and a commonsense reasoning problem (from StrategyQA), respectively. These examples illustrate the consistent distinction between ”plan” (task decomposition) and ”steps” (concrete execution) in our data.

#### B.2.1 Example from GSM8K (Mathematical Reasoning)

#### B.2.2 Example from StrategyQA (Commonsense Reasoning)

## Appendix C Interpretability

Figure: (a) First latent plan token.
Refer to caption: figure/latent0_top50_freq_top50.png

To understand why this implicit plan representation is so effective, we conducted an in-depth analysis of the information encoded within these latent tokens. We set the number of latent plan tokens to two. We analyzed the vocabulary distribution of the language model head’s logits corresponding to each token across a sample of 200 different problems. The results, visualized in Figure [3](https://arxiv.org/html/2509.22131v2#A3.F3), reveal a fascinating specialization of roles between the latent tokens.

The First Latent Plan Token: Encoding Intent and Initiation.
As shown in Figure [3](https://arxiv.org/html/2509.22131v2#A3.F3)(a), the vocabulary distribution for the first latent token is dominated by high-frequency ”discourse markers” and ”initiator” words. Tokens such as Here, Plan, Step, Let’s, and First appear with high probability. This suggests that the first token has learned to function as a structural signal, activating a ”planning” or ”reasoning-initiation” mode within the model. Furthermore, the presence of instructional verbs like Calculate, Think, Find, and Determine in the mid-frequency range indicates that this token also captures the high-level intent or the primary cognitive action required for the initial part of the plan. It essentially tells the model, ”Begin reasoning, and the first major goal is to calculate/find something.”

The Second Latent Plan Token: Encoding Structure and Execution.
The distribution for the second latent token, shown in Figure [3](https://arxiv.org/html/2509.22131v2#A3.F3)(b), paints a different but complementary picture. This token’s top predictions are heavily skewed towards structural and mathematical symbols, including newline characters (\n), comparison operators (¡, ¿), parentheses ((, )), and arithmetic operators (-, *, =). This strongly indicates that the second token has specialized in encoding the executional and structural format of the subsequent calculation step. It prepares the model for the precise symbolic manipulation required, acting as a bridge between the high-level intent (from the first token) and the low-level, formatted output of the CoT step. The co-occurrence of tokens like Sum, Total, number, divide, and multiply further solidifies its role in priming the model for specific mathematical operations.

### C.1 Attention Analysis

To validate the Reasoning Capsule’s role in guiding generation, we quantitatively analyzed the decoder’s attention mechanism. Using 143 samples from the GSM8K dataset, we tracked the attention from the generated Steps & Answer sequence to the Latent Plan (Reasoning Capsule).

For the analysis, we segmented the input into three regions: Question, Latent Plan, and Steps & Answer. We then calculated the average attention from the Steps & Answer region to the Latent Plan region, aggregated across all samples. The results are visualized as:

- •
Attention Curve (Figure [4(a)](https://arxiv.org/html/2509.22131v2#A3.F4.sf1)): Plots the average attention weight against the normalized position in the Steps & Answer sequence, showing the attention trend during generation.
- •
Inter-Region Attention Heatmap (Figure [4(b)](https://arxiv.org/html/2509.22131v2#A3.F4.sf2)): An aggregated heatmap showing the attention flow between all defined regions (Query and Key).

Figure: (a) Aggregated causal attention weights across 143 test samples
Refer to caption: x2.png

### C.2 Attention Curve (Figure 4(a) ): Sustained Guidance

The attention to the Latent Plan remains high and stable throughout the generations, confirming its continuous guiding role.

- •
Step Generation (Normalized Position 0–0.8): Attention is stable, indicating that the model continuously references the high-level plan while generating low-level calculation steps.
- •
Answer Generation (Normalized Position 0.8–1.0): Attention slightly increases as the model cross-references the plan to ensure the final answer aligns with the initial strategy.

### C.3 Inter-Region Attention Heatmap (Figure 4(b) ): Latent Plan Dominance

The heatmap quantifies the Latent Plan’s dominance in attention allocation.

- •
Strong Guidance for Generation: The attention from Steps to the Latent Plan (0.745) and from Answer to the Latent Plan (0.310) is significantly higher than to the original Question (0.121 and 0.162, respectively). This confirms the capsule acts as the primary strategic guide.
- •
Information Compression: Low attention to the Question suggests the Latent Plan effectively extracts and condenses all necessary information, making repeated access to the original problem unnecessary.
- •
Reduced Error Propagation: Near-zero self-attention within the generated steps (Steps $\rightarrow$ Steps) indicates the model relies on the global Latent Plan as a unified reference rather than on preceding steps, which helps mitigate cascading errors.

## Appendix D Information Bottleneck Formulation for Reasoning Capsules

We formalize the training objective of Reasoning Capsules under the Information Bottleneck (IB) principle (Tishby et al., 2000).
Our goal is to learn a compressed latent representation $C\in\mathbb{R}^{d}$ of the high-level plan that is *minimal* yet *sufficient* for both reconstructing the original plan $P$ and predicting the final answer $A$.

### D.1 IB Objective

Given the hidden state $\mathbf{h}_{t}\in\mathbb{R}^{D}$ at the capsule-token position, we seek a stochastic encoding $p(C\mid\mathbf{h}_{t})$ that solves

$$ $\boxed{\min_{p(C\mid\mathbf{h}_{t})}\ \underbrace{I(\mathbf{h}_{t};C)}_{\text{compression}}-\beta\underbrace{I(C;P)}_{\text{plan reconstruction}}-\gamma\underbrace{I(C;A)}_{\text{answer prediction}}}$ (5) $$

where

- •
$I(\mathbf{h}_{t};C)$ enforces minimality by limiting the information contained in the capsule;
- •
$I(C;P)$ ensures sufficiency for reconstructing the textual plan $P$;
- •
$I(C;A)$ guarantees that the capsule is predictive of the final answer $A$;
- •
$\beta,\gamma>0$ are Lagrange multipliers controlling the trade-off between compression and sufficiency.

### D.2 Parametric Approximation

In practice, we employ a deterministic encoder with a linear bottleneck

$$ $C=\mathrm{Proj}(\mathbf{h}_{t})=\mathbf{W}_{p}\mathbf{h}_{t}+\mathbf{b}_{p},\qquad\mathbf{W}_{p}\in\mathbb{R}^{d\times D},\;d\ll D.$ (6) $$

This structural bottleneck enforces hard minimality: under a Gaussian assumption, the mutual information is upper-bounded by $I(\mathbf{h}_{t};C)\leq\frac{d}{2}\log(2\pi e\sigma^{2})$.

### D.3 Training Objective as IB Surrogate

We optimize a variational upper bound on Eq. equation [5](https://arxiv.org/html/2509.22131v2#A4.E5):

$$ $\boxed{\mathcal{L}_{\mathrm{IB}}=\underbrace{-\log p_{\theta}(P\mid C)}_{\text{plan reconstruction}}+\lambda\underbrace{-\log p_{\phi}(A\mid C,Q)}_{\text{answer prediction}}}$ (7) $$

where

- •
$p_{\theta}(P\mid C)$ is a *shallow* transformer decoder that regenerates the plan;
- •
$p_{\phi}(A\mid C,Q)$ is the primary model that produces the final answer;
- •
$\lambda$ balances the two losses, and is numerically equivalent to the ratio $\beta/\gamma$ in Eq. equation [5](https://arxiv.org/html/2509.22131v2#A4.E5).

## Appendix E Latent Token Number Ablation

To validate the choice of latent token number $K$ (fixed as 2 in the main text), we conduct ablation experiments on $K\in\{1,2,3,4\}$ using GSM8K (mathematical reasoning) and StrategyQA (commonsense reasoning) datasets. We evaluate accuracy, generated tokens (before answer), and inference latency (batch size=1 on A100), with results shown in Table [3](https://arxiv.org/html/2509.22131v2#A5.T3).

**Table 3: Latent Token Number $K$ Ablation on Qwen3-8B**
| Model | $K$ | GSM8K Acc. (%) | GSM8K Tokens | StrategyQA Acc. (%) | Latency (s) |
| --- | --- | --- | --- | --- | --- |
| Qwen3-8B | 1 | 87.9 | 82 | 78.1 | 1.53 |
| 2 | 89.7 | 86 | 79.8 | 1.65 |  |
| 3 | 86.2 | 103 | 79.5 | 1.89 |  |
| 4 | 85.5 | 118 | 78.9 | 2.11 |  |

Key observations:
1. $K=2$ achieves the highest accuracy across models/datasets. $K=1$ under-represents the plan (lower accuracy), while $K\geq 3$ increases token count/latency without accuracy gains (redundant information).
2. Latency grows linearly with $K$, as more latent tokens require additional projection/computation. Thus, $K=2$ balances accuracy and efficiency.

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 4
presentation: 2
questions: 1.	Definition & validation of “high-level plan.”

- How exactly do you define a high-level plan independently of the training signal?
- Can you provide automatic plan-quality metrics (e.g., plan edit distance vs. reference; causal interventions showing that altering the capsule alters the decoded plan and the solution in predictable ways)? 

2.	Re-planning and flexibility.

- Can the model revise its capsule mid-generation when encountering contradictions or errors?
- Have you tried a two-stage or iterative capsule (plan → steps → re-capsule → revised steps) to emulate human re-planning? Any empirical evidence that a fixed K=2 capsule is sufficient? 

3.	Stronger baselines and larger models.

- Please compare against recent long-CoT / reflective baselines and strong distillations (e.g., DeepSeek-distill-Qwen, R1-style training).
- Provide results on ≥30B models to substantiate scalability claims. What happens at 32B/70B? Do gains widen or vanish? 

4.	Data generation ablations (GPT-o3).

- If you only fine-tune the base model on o3-generated reasoning (no capsule machinery), how much of the gain remains?
- Conversely, if you control for the total o3 tokens seen, does the latent-plan path still outperform explicit short-plan SFT? 

5.	Ablations on harder benchmarks and token budgets.

- Report accuracy vs. token-budget curves on AIME’24/’25, MATH500, HMMT’25. Does plan-only compression still dominate?
- Provide per-problem error analysis where capsules help vs. hurt (e.g., multi-turn algebra vs. combinatorics). 

6.	Capsule interpretability and diagnostics.

- Beyond the weak-decoder visualization, can you probe capsules with linear classifiers to predict plan attributes (number of subgoals, operation types), or perform targeted perturbations that yield controlled changes in decoded plans?
soundness: 2
strengths: - Clarity: The paper is generally well written and easy to follow, with a clean separation of planning vs. execution and an intuitive training diagram. 
- Core idea: Compressing plans rather than steps is an interesting and plausible hypothesis about where “information” resides in CoT (supported by ablations indicating plan-only compression is best among tested options). 
- Methodological framing: The Information Bottleneck view (minimality via a bottleneck; sufficiency via plan reconstruction) is a neat, principled story that helps explain design choices. 
- Efficiency gains: The approach reduces visible tokens and latency compared to explicit Plan-CoT on GSM8K (Qwen3), which is practically valuable.
summary: This paper proposes R-Capsule, a framework that compresses the high-level plan of a reasoning chain into a small number of learned latent tokens, while leaving execution lightweight or explicit. The design is motivated by an Information Bottleneck objective: a low-capacity projection enforces minimality, and a plan-reconstruction loss encourages sufficiency and a semantically grounded latent (via a shallow decoder). Experiments on GSM8K, MultiArith, AQuA, StrategyQA, and CSQA2 with small/medium base models (e.g., GPT-2 ~150M, Llama-3-1B, and Qwen3-8B; limited results for 7B/8B) show modest accuracy gains over CoT-SFT and reduced token counts/latency.
weaknesses: 1.	Questionable premise / under-specified concept of “high-level plans”.

The paper repeatedly leans on “high-level plan” without a precise operational definition or an objective test of plan quality/faithfulness. Reasoning is often non-linear and flexible (re-planning, reflection, backtracking), not strictly “plan-then-execute.” The work assumes a fixed latent plan is the right anchor, yet provides limited analysis of cases where plans need to be revised mid-trajectory. 

2.	Experimental scope is limited and misses key settings.

- Model regime: Main results focus on base or very small models (150M–1B) with a small excursion to Qwen3-8B/Llama-3-7B. Today’s “long-CoT” capable models (e.g., o1/R1-style or strong distillations) are absent, making it hard to judge competitiveness in the regime where planning helps most. 
- Scaling: The largest model is 8B; there is no 30B+ evidence. The scalability claim therefore remains weakly supported. Figure 2a suggests only small gains at 7B/8B and does not explore larger scales. 
- Datasets: Benchmarks are comparatively easy/standard (GSM8K, MultiArith, AQuA, StrategyQA/CSQA2). Stronger contemporary math sets (AIME’24/’25, MATH500, HMMT’25, etc.) are not included, which limits the external validity of the efficiency–accuracy trade-off. 
- Baselines: Missing comparisons to models specifically trained for long CoT or modern open distillations (e.g., DeepSeek-distill-Qwen variants), leaving open whether R-Capsule improves over competitive long-reasoning approaches. 
- Data generation confound: Training CoT/plan data are synthesized with GPT-o3. The paper does not disentangle how much of the observed improvement comes simply from exposing the base model to high-quality o3 trajectories versus the proposed latent-plan mechanism. 

3.	Effect size concerns.

On Qwen3-8B, the reported improvements over Plan-SFT/CoT-SFT appear small (Figure 2a), raising questions about whether the extra complexity (capsule projection + weak decoder + auxiliary loss) is justified at scale. 

4.	Technical under-specification.

The paper does not give a rigorous, task-agnostic definition of “plan,” nor a measurable criterion for when a latent token “faithfully” encodes a plan. The link between the abstract plan concept and a fixed number of latent tokens (e.g., K=2) seems heuristic, and the benefits vs. simply using short explicit plans are not thoroughly isolated.

---

confidence: 4
contribution: 2
initial_rating: 4
presentation: 3
questions: Please refer to Weaknesses.
soundness: 2
strengths: The central contribution—selectively compressing the high-level plan while leaving the execution steps explicit—is both novel and significant. While prior work has explored latent reasoning or hierarchical planning, this paper's core hypothesis that the plan is the right component to compress is a fresh and compelling perspective. It offers a practical solution to the well-known trade-off between the performance of explicit CoT and the efficiency of implicit reasoning. The findings could influence how future models are designed for efficient and structured reasoning.
summary: This paper introduces the "Reasoning Capsule" (R-Capsule), a novel framework designed to improve the efficiency and accuracy of large language models (LLMs) in complex reasoning tasks. The core idea is to address the high latency and verbosity of standard Chain-of-Thought (CoT) prompting by decoupling the reasoning process into a high-level plan and low-level execution steps. Instead of generating an explicit textual plan, the model learns to compress it into a small set of latent tokens—the Reasoning Capsule.

The method is theoretically grounded in the Information Bottleneck (IB) principle. The capsule is encouraged to be minimal through a low-capacity architectural bottleneck and sufficient through a dual training objective. This objective combines a primary task loss (for answer accuracy) with an auxiliary plan-reconstruction loss, where a separate, shallow decoder is trained to reconstruct the original textual plan from the capsule. This reconstruction loss grounds the latent representation, making it more interpretable and preventing the model from learning uninformative shortcuts.

Experiments on mathematical and commonsense reasoning benchmarks (GSM8K, StrategyQA, etc.) with various model sizes (from GPT-2 to 8B models) show that R-Capsule consistently outperforms standard CoT fine-tuning and other baselines in accuracy, while significantly reducing the number of generated tokens and inference latency.
weaknesses: 1. The experimental section lacks performance evaluation for each model without any training. Including the performance of different prompt strategies (Standard QA, CoT).
2. It is necessary to clarify which version of qwen-3 is being used, whether it is the "instruct" or "thinking" version. The experimental part needs to supplement the performance comparison of different versions of qwen-3.
3. The experimental section requires a more complete comparison, such as the training results of qwen-3-8b on Standard SFT.
4. The method proposed by the author is very similar to the paper "Guiding Language Model Reasoning with Planning Tokens", and the author needs to make a comparison with this method in the experimental part. And analyze what the differences are in the methods.
5. The dataset used by the author is of relatively low difficulty. It is recommended to use some datasets of higher difficulty for evaluation. For example, MATH, HotpotQA, etc.

---

confidence: 4
contribution: 2
initial_rating: 2
presentation: 2
questions: 1. Could you please provide the precise architectural details and hyperparameters for the "bottleneck network" and the "shallow decoder" (e.g., hidden dimension, layers, attention heads)?
2. Can you comment on the robustness and cost of the data generation pipeline? What was the approximate failure rate of gpt-o3 in generating correct and logically sound (plan, steps) pairs, and how much manual filtering was required?
3. Have you conducted any experiments on out-of-distribution generalization? 
4. How do you see the R-Capsule framework being applied to reasoning tasks that do not have a clear, pre-defined plan -> step structure?
soundness: 2
strengths: 1. The paper is well motivated. It studies the practical problem of balancing the efficiency and performance of chain-of-thought reasoning.
2. The proposed method is theoretically grounded.
3. Strong empirical results across datasets and model sizes.
4. Extensive ablation studies on the main design choices.
summary: This paper introduces the "Reasoning Capsule" (R-Capsule), a framework to improve the efficiency of CoT reasoning. The core idea is to compress the high-level plan into a small set of latent tokens (the "capsule") which then conditions the generation of explicit execution steps. This method is grounded in the Information Bottleneck principle, using a structural bottleneck to enforce minimality (compression) and a dual-loss (task accuracy + plan reconstruction) to ensure sufficiency. Experiments on math and commonsense benchmarks show R-Capsule improves both accuracy and efficiency (fewer tokens, lower latency) over strong CoT baselines.
weaknesses: Despite the strengths, the paper has several significant weaknesses, primarily centered on reproducibility, generalizability, and the rigidity of the proposed framework.

1.The paper lacks crucial details about its novel architectural components, making the work very difficult to reproduce.
2. All experiments appear to be conducted on in-distribution test sets. It is unclear how the learned latent "plan" representations would generalize to out-of-distribution data, such as problems from a different domain, with different phrasing, or of greater complexity than those seen during training.
3. The method assumes a rigid, hierarchical plan -> step structure for reasoning. This limits the method's applicability, as this structure may not be suitable for all types of reasoning problems. This rigid structure also imposes a significant and non-trivial data annotation burden. The framework requires a large dataset of (plan, steps, answer) tuples. The paper uses a large proprietary model (gpt-o3) to generate this data, but the practical challenges, costs, and failure rates of this data generation pipeline are not discussed. This is a major practical barrier to scaling the method to new tasks or domains.
4. The paper includes a model scaling analysis (Fig 2a), which is commendable. However, it is missing a data scaling analysis. Given the high cost of the specialized training data, a crucial analysis would be to show how performance scales with the amount of training data. Does R-Capsule still outperform baselines in a low-data regime (e.g., with 10% or 25% of the data)? This analysis is essential for understanding the method's data efficiency and practical utility.

---

confidence: 4
contribution: 2
initial_rating: 0
presentation: 2
questions: 1. What’s the architecture of the weak decoder in Figure 1?
2. Why does table 1 only contain the results of two small models? Also, why it does not have the result of GPT-2 for commonsense reasoning? It also does not contain the result of coconut and iCoT for 1B model.  Since for the two small models, the most standard CoT+SFT achieves better performance than other methods (such as iCoT, coconut, etc.), does that mean the models chosen in the table are too small such that other methods do not show advantage? Or in other words, does that mean R-capsule can only outperform other methods in the regime of <= 150M models?
3. I’m not sure whether the claim in Section 3.4 that R-capsule (Plan-only) achieves the best accuracy-efficiency is very convincing. According to Figure 2(b), the accuracy difference between R-capsule (plan-only) and R-capsule (Plan+steps) is quite small and even negligible, while their efficiency differs a lot. Also, what’s the task tested in Figure 2(b)?
4. Does the weak decoder aim to reproduce the plan word by word or only keep its semantics? It seems you’re calculating a reconstruction MSE loss so you need to reproduce it exactly, but in section 3.6 you also mentioned that the weak decoder will output a more concise plan. Am I missing anything?
5. Does the weak decoder only accept latent capsules as the input? Since in your experiments, the number of latent tokens are just 2, I’m curious how two tokens can compress all information about the plan?  For example, the output of the weak decoder in page 9 at least contains information about “Peter”, “Christina”, “number of word”, “misspelling”; are they are stored in two tokens? Or does the weak decoder also accept the problem description as the input?
soundness: 2
strengths: 1. The questions studied is important, i.e., improving LLM reasoning capability while reducing inference cost and increasing the interpretability.
2. The idea of using a high-level plan in latent space before the detailed execution steps makes sense.
summary: The paper proposes a new method for LLM reasoning, R-Capsule, where LLMs first output high-level plans which are in a latent space and then textual detailed steps and finally the answer. The authors choose several benchmarks on math reasoning (such as GSM-8k) and commensense reasoning (such as strategyQA). They tested four models: GPT-2 (0.2B), LLaMA-3 (1B), LLaMA3.1 (7B)  and Qwen-3 (8B) and compared with baselines such as standard SFT, SFT+CoT, iCoT, coconut, etc.
weaknesses: 1. I think the experiments are not comprehensive to make solid conclusions. The author chooses several benchmarks, four models and several methods. However, for each research question (RQ), they only choose a subset of benchmarks, models or methods to claim their method is better, while for difference RQs they use different subsets. This is not convincing to prove the effectiveness of the proposed method due to a “selective bias”.
2. The chosen benchmark are kind of “obsolete” and not challenging enough. For example, for math reasoning, the authors might consider test the proposed method in MATH or AIME, which are widely used for many current research.
3. I believe the authors miss an important previous work [1]. The previous work [1] proposed a method that is very similar to this paper (the major difference might be in their paper, the high-level tokens are in a discrete latent space while in this paper the high-level planning capsules are in a continuous latent space). However, the whole ideas are quite similar. I think it is important to discuss the differences between the two methods and include [1] as in important baseline to show the proposed method is better.
4. For RQ4, I don’t think it is fair to compare R-capsule with Plan-SFT. In the main table, Plan-SFT performs worse than CoT-SFT and the authors also claimed that CoT-SFT is the strongest baseline. In that case, the authors should compare generation length and latency with CoT-SFT to claim the proposed method is efficient.
5. The format looks a bit messy. For example, for citation, many \citet should be \citep and they are not separated from the main text, which makes reading difficult. In the related work section, the title of each paragraph should be bolded or should have a period.

**Reference:**

[1] Su, DiJia, Hanlin Zhu, Yingchen Xu, Jiantao Jiao, Yuandong Tian, and Qinqing Zheng. "Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoning." In Forty-second International Conference on Machine Learning.