Title: System-1.5 Reasoning: Traversal in Language and Latent Spaces with Dynamic Shortcuts
Venue: NeurIPS 2025
Accepted: yes
Keywords: Efficient Reasoning Model

Latent-space Reasoning

Implicit Chain-of-Thought

Dynamic Early Exit

Conditional Computation
Abstract: Chain-of-thought (CoT) reasoning enables large language models (LLMs) to move beyond fast System-1 responses and engage in deliberative System-2 reasoning. However, this comes at the cost of significant inefficiency due to verbose intermediate output. Recent latent-space reasoning methods improve efficiency by operating on hidden states without decoding into language, yet they treat all steps uniformly, failing to distinguish critical deductions from auxiliary steps and resulting in suboptimal use of computational resources. In this paper, we propose System-1.5 Reasoning, an adaptive reasoning framework that dynamically allocates computation across reasoning steps through shortcut paths in latent space.Specifically, System-1.5 Reasoning introduces two types of dynamic shortcuts. The model depth shortcut (DS) adaptively reasons along the vertical depth by early exiting non-critical tokens through lightweight adapter branches, while allowing critical tokens to continue through deeper Transformer layers. The step shortcut (SS) reuses hidden states across the decoding steps to skip trivial steps and reason horizontally in latent space. Training System-1.5 Reasoning involves a two-stage self-distillation process: first distilling natural language CoT into latent-space continuous thought, and then distilling full-path System-2 latent reasoning into adaptive shortcut paths (System-1.5 Reasoning).Experiments on reasoning tasks demonstrate the superior performance of our method.
For example, on GSM8K, System-1.5 Reasoning achieves reasoning performance comparable to traditional CoT fine-tuning methods while accelerating inference by over 20× and reducing token generation by 91.0\% on average.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces System-1.5 Reasoning, a novel adaptive reasoning framework designed to optimize computational efficiency by combining latent-space reasoning and early-exit mechanisms. The framework utilizes two shortcuts, depth shortcut (DS) and step shortcut (SS), and trained through a two-stage distillation process to selectively allocate computational resources based on step criticality. Empirical evaluations on GSM8K, GSM-HARD, and StrategyQA show that System-1.5 significantly reduces computational overhead and inference time while achieving performance comparable to traditional CoT fine-tuning.\n\nSTRENGTHS AND WEAKNESSES:\n**Strength:**\n- The paper effectively combines latent-space reasoning with early-exit mechanisms, proposing a well-motivated approach to reduce computational cost in reasoning.\n- Empirical results on benchmarks such as GSM8K, GSM-HARD, and StrategyQA consistently demonstrate that System-1.5 Reasoning maintains high performance while substantially accelerating inference.\n- Ablation study validates the effectiveness of the proposed architecture and highlights its flexibility in controlling computational budget at inference time.\n\n**Weakness:**\n- Several technical details regarding the computation process of each layer and the exact implementation of shortcuts are unclear (see Questions).\n- Motivation behind certain algorithmic choices and module designs remains insufficiently justified (see Questions).\n- The author does not clearly illustrate or examine the impact of the early-exit loss in Equations 10 and 11 on the labeled intermediate reasoning steps. \n- The evaluation methodology has limitations; specifically, using GSM-HARD as an OOD benchmark is questionable given its close relationship with the GSM8K training dataset. Additional evaluation on other datasets, such as MATH, would enhance the reliability of OOD generalization claims.\n\nQUESTIONS:\n- The reasoning pattern in Figure 1 is unclear, particularly regarding the process of transformer block computation. Given that computation at a transformer block requires hidden states from all previous steps from the same layer, the figure suggests certain layers are entirely skipped. Is the lightweight LoRA branch on those also skipped? How exactly is the transformer block, especially attention, computed in cases where layers or branches appear entirely skipped?\n- Following from the first question, if transformer block computation necessitates some computation for each layer, how does the proposed layer skipping in Equation 1 work during inference? Specifically, is skipping (using a lightweight branch) at an earlier layer independent, or does it influence the decision-making for skipping subsequent layers?\n- What is the motivation behind combining depth and step shortcuts in Equation 5 during training, given that they operate as independent branches during inference?\n-  Why does Equation 6 focus specifically on optimizing hidden space consistency only for the answer part? Would it not be more logical to optimize either exclusively the reasoning trace or a combined approach?\n- Why is joint updating of reasoning steps and answers necessary for the teacher model in Equation 7? How does this joint updating specifically contribute to the model's reasoning capabilities?\n- The rationale behind minimizing MSE for critical steps in early layers and minimizing non-critical steps with later layers in Equation 10 is unclear. \n- While the paper states that models are built on LLaMA 3.2 1B and GPT-2 124M backbones, it is not clearly specified which backbone is employed in the experimental results reported. Additionally, it would be helpful to know precisely how the baselines are configured with respect to these backbones to ensure fair comparison.\n\nLIMITATIONS:\nPlease see Questions and Weaknesses above.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 1, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-20"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper mitigates the inefficiency of Chain-of-Thought reasoning in Large Language Models. While standard CoT improves reasoning accuracy (System-2), it is slow. Existing latent-space reasoning methods improve speed by avoiding text generation but are still suboptimal as they apply uniform computation to all reasoning steps, regardless of their complexity.  To overcome this, the authors propose System-1.5 Reasoning, an adaptive framework that dynamically allocates computation via Depth Shortcut and Step Shortcut. Experiments on mathematical (GSM8K, GSM-HARD) and commonsense (StrategyQA) reasoning tasks show that System-1.5 Reasoning achieves accuracy comparable to standard CoT fine-tuning while providing massive efficiency gains, including an over 20x inference speedup on GSM8K.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n1. The performance gains reported are impressive. Achieving a >20x speedup and >90% token reduction on benchmarks like GSM8K, while maintaining the accuracy of the much slower CoT baseline.\n2. The idea of combining both vertical (depth-wise) and horizontal (step-wise) shortcuts is novel.\n\n**Weaknesses**\n1. The shortcut learning stage relies on criticality labels generated by an external \"atomic thought decomposition\" process. This introduces a dependency and a potential point of failure. How sensitive is the performance of System-1.5 Reasoning to the quality of these annotations? Have the authors considered scenarios with noisy or imperfect criticality labels?\n2. The proposed framework is fundamentally supervised, relying entirely on distillation from a teacher model that generates explicit, pre-defined CoT paths. The critical supervisor signal—the reasoning criticality labels—is also derived from these static teacher traces. This makes it unclear how the method could be extended to a Reinforcement Learning (RL) setting like DeepSeek R1.\n3. The full framework is quite involved, requiring a pre-trained base model, a CoT-finetuned teacher, a System-2 student, a criticality annotation pipeline, and finally the System-1.5 model.\n\nQUESTIONS:\nThe shortcut learning stage is guided by 'reasoning criticality' labels derived from atomic thought decomposition. How sensitive is the final performance of System-1.5 Reasoning to the quality and granularity of these annotations?\n\nLIMITATIONS:\nAs mentioned above.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-23"}

comments: SUMMARY:
This paper carries out an important and innovative study to decrease the inference cost of reasoning LLMs by training LLM with the mechanisms of depth and step shortcuts. By varying the inference cost on each token by weighing their significance, the model focuses on more important reasoning token and step, and skip the trivial tokens early.

Overall, the proposed method is novel and the writing is clear. However, I have questions on the construction of the dataset for shortcut learning. The authors do not specifically illustrate the rationality of this process and the trustworthiness of the method. Meanwhile, I am curious why the sub-step criticality can be used to determine the criticality of the token in this step?

STRENGTHS AND WEAKNESSES:
I think this paper has many strengths. Hence, I will not list the strengths specifically since I think the paper can be accepted if the questions on dataset construction can be answered,

The main weakness is: how the dataset for shortcut learning is constructed, Why atom-of-thought to construct directed acyclic graph is one decent way? Is there any analysis or methods to verify the correctness? Meanwhile, why the importance assessment on the sub-step can be used to assess the token importance?

QUESTIONS:
As listed above,

LIMITATIONS:
In appendix. Need to be moved to main content.
TITLE: Official Review
CLARITY: 3
IMPACT: 4
ORIGINALITY: 4
DATE: 2025-06-24

comments: SUMMARY:
This paper proposes an approach to achieve adaptive reasoning to improve the efficiency of reasoning via language tokens. The approach is evaluated on two categories of tasks (GSM8K and StrategyQA) to demonstrate the it achieves the same accuracy as language CoT but significantly improves inference efficiency. The main approach is distillation from language CoT with a curriculum.

STRENGTHS AND WEAKNESSES:
Strength:
The paper addresses an important problem of improve reasoning efficiency and provides new empirical results on reasoning in latent space. The design of the method to be adaptive and yet controllable is an appealing property for test-time compute scaling.

Weaknesses:
The training is very complex and lacks in-depth ablations to understand the importance of different components, e.g. the importance of curriculum, what are important factors for optimization, e.g. should the router be jointly learnt, how does freezing parameters affect training, etc.

QUESTIONS:
1. What's the comparison with different baselines in terms of training throughput (speed)?
2. The approach is evaluated in both math reasoning and commonsense reasoning. But how does the performance differ with problems of different levels of difficulty? 
3. Regarding evaluating controllable test-time scaling, can you also report pass@k with increasing k and compare it to the trend of baselines?

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 2
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces System-1.5 Reasoning, an adaptive reasoning framework for large language models that uses dynamic shortcuts in latent space to balance efficiency and performance in chain-of-thought reasoning. The approach leverages two shortcut types—model depth shortcuts and decoding step shortcuts—allowing the system to allocate more computation to important reasoning steps and less to easier ones. Experiments on standard reasoning benchmarks show that System-1.5 Reasoning matches or surpasses existing baselines, while significantly reducing inference cost.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n- The proposed framework is well-motivated, introducing flexible, dynamic computational shortcuts at both the model depth and decoding step levels.\n- The method delivers substantial speed-ups—up to 20× faster inference on GSM8K—while maintaining or even improving accuracy over prior work.\n\n**Weaknesses**\n- The experimental section feels a bit thin. Table 1 only reports final results and lacks detailed ablation studies. For example, how do depth and step shortcuts individually contribute to speed and accuracy? Figure 3 examines student selection and joint/full-parameter learning, but does not directly analyze the role or interaction between depth and step shortcuts. What happens if only one type of shortcut is used?\n- There are no concrete case studies showing how depth and step shortcuts are routed in practice. How does the system decide which shortcut to take for different types of questions? Are there potential issues with load balancing?\n- All experiments are conducted on relatively small models (GPT-2 124M, LLaMA 3.2B). While understandable for research, this limits the relevance for real-world applications, where much larger models (>7B) are common.\n\nQUESTIONS:\n- How is the routing for depth and step shortcuts initialized or warmed up? Does the router start with random predictions, and are there training curves available showing its learning progress?\n- Is this method compatible with mainstream inference engines (such as vllm or sglang)? These engines are already highly optimized for fast inference. If System-1.5 is not compatible, then its practical advantage may be limited despite the speed improvements shown in the paper.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}