Title: SCOUT: Teaching Pre-trained Language Models to Enhance Reasoning via Flow Chain-of-Thought
Venue: NeurIPS 2025
Accepted: yes
Keywords: Recursive reasoning

Flow Chain-of-Thought
Abstract: Chain-of-Thought (CoT) prompting improves the reasoning performance of large language models (LLMs) by encouraging step-by-step thinking. However, CoT-based methods depend on  intermediate reasoning steps, which limits scalability and generalization. Recent work explores recursive reasoning, where LLMs reuse internal layers across iterations to refine latent representations without explicit CoT supervision. While promising, these approaches often require costly pretraining and lack a principled framework for how reasoning should evolve across iterations.
We address this gap by introducing **Flow Chain-of-Thought (Flow CoT)**, a reasoning paradigm that models recursive inference as a progressive trajectory of latent cognitive states. Flow CoT frames each iteration as a distinct cognitive stage—deepening reasoning across iterations without relying on manual supervision. To realize  this, we propose **SCOUT** (*Stepwise Cognitive Optimization Using Teachers*), a lightweight fine-tuning framework that enables Flow CoT-style reasoning without the need for pretraining. SCOUT uses progressive distillation to align each iteration with a teacher of appropriate capacity, and a cross-attention-based retrospective module that integrates outputs from previous iterations while preserving the model’s original computation flow.
Experiments across eight reasoning benchmarks show that SCOUT consistently improves both accuracy and explanation quality, achieving up to 1.8\% gains under fine-tuning. Qualitative analyses further reveal that SCOUT enables progressively deeper reasoning across iterations—refining both belief formation and explanation granularity. These results not only validate the effectiveness of SCOUT, but also demonstrate the practical viability of Flow CoT as a scalable framework for enhancing reasoning in LLMs.


Peer reviews:
comments: SUMMARY:
The paper proposes a flow CoT for recursive reasoning paradigm with depth-aware trajectory of latent cognitive states.  The authors develop a light-weight fine-tuning framework with a step wise optimization technique using teachers.  The introduction of a progressive distillation strategy enables the reasoning to be supervised by a teacher. The paper also presents a non-intrusive retrospective module using cross-attention that ensures the original reasoning flow. The authors empirically show the improvements in the performance on multiple benchmarks.

STRENGTHS AND WEAKNESSES:
The major strength of the paper lies in its capabilities of multi-step reasoning without requiring explicit annotations at each step. The paper attempts to train the model through progressive distillation using teachers with matching capacity unlike distillation requiring strong uniform supervision. The proposed retrospective module enhances depth-aware reasoning capabilities in pre-trained LLMs. The authors present a comparative study of the performance of the proposed model with different baselines. The token level probabilities heatmap shows a clear progressive trajectory towards the correct answer using the approach.

It is not clear why Qwen 2.5 model series is chosen for the task. Similarly if Qwen is used, the reason behind not using an update Qwen model is also missing.

QUESTIONS:
1. The authors should clearly mention the reason for the model preference.

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper points out a problem in existing recursive reasoning models: recursive reasoning should not be viewed as black-box repetition, but as a structured cognitive evolution. It introduces Flow CoT, a reasoning paradigm that envisions recursive inference in LLMs as a progressive trajectory of cognitive states, moving beyond conventional chain-of-thought, which depends on explicit supervision for intermediate steps. To operationalize this paradigm, the authors propose SCOUT, a fine-tuning framework that combines progressive distillation with a cross-attention-based retrospective reasoning module.  The approach is extensively evaluated on 8 benchmarks (covering commonsense QA, multi-step math, code, etc.), demonstrating improved accuracy and explanation quality versus several recent baselines.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n(1) Novel Concept: The paper introduces the concept of Progressive Distillation and leverages latent variables to enhance performance on reasoning tasks. This is a valuable contribution to the reasoning research community.\n\n(2) Comprehensive Evaluation: The proposed SCOUT method is evaluated on eight benchmarks covering various types of tasks. The paper also explores different training settings and provides insights into the key aspects of the method.\n\n\nWeaknesses:\n(1) Scalability: The method depends on multiple teacher models with different abilities, raising concerns about scalability—especially when applying it to larger models. For very strong models, it may be difficult to find appropriately leveled teacher models for supervision. Additionally, experiments only go up to a 7B teacher model, which limits confidence in the method’s effectiveness at larger scales (e.g., 72B). The approach should be validated on a wider range of model sizes.\n\n(2)Baseline Comparison: An additional baseline should be included where a single model is distilled progressively using teachers of sizes 1.5B to 7B in sequence, with inference performed only once (to distinguish from R-Distill-EQ). Also, the fair comparison for SCOUT should be with CoT-style SFT, not just pure SFT as currently reported. Comparisons with latent CoT-based methods would also strengthen the evaluation. At least the differnet should be discussed thoroughly.\n\n(3) Lack of Efficiency Analysis: Although the method could offer efficiency advantages over traditional explicit reasoning approaches like R1, there is insufficient analysis of wall-clock time and inference speed. In particular, Flow CoT should theoretically be faster than decoding a model three times in a standard CoT setup, but this is not discussed.\n\nQUESTIONS:\nThe concept of Flow CoT appears to share similarities with diffusion models. Is there a deeper connection between language diffusion models and your work? Could you elaborate on this relationship?\n\nIn Line 131, you state, \"This misalignment can distort latent states and harm downstream refinement.\" This is a strong claim. Do you have any citations or concrete analyses to support this strong claim about existing recursive methods?\n\nSome aspects of the method are still unclear. For instance, how are the head block and tail block explicitly defined when working with LLMs? Could you provide more details about the training and inference processes?\n\nCould you explain why CoQA performance decreases as the number of reasoning steps increases?\n\nBenchmarks such as GLUE, CoQA, and MBPP are generally not considered to require extensive reasoning. As a result, they may not be suitable for evaluating a model’s multi-step reasoning abilities. Can you explain more about the model behavior of these datasets and why they can be benefited from SCOUT?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

comments: SUMMARY:
The paper proposes Flow CoT and SCOUT, a novel reasoning and lightweight training method.  They propose having recursive reasoning stages within an LLM, where each stage learns from the last stage and improves upon it. Experiments show gains over the prior methods. Moreover, they show qualitatively that each reasoning stage learns how to correctly answer progressively difficult queries.

STRENGTHS AND WEAKNESSES:
Strenthgs:
- Experiments are performed across different reasoning types like commonsense, mathematical, reading comprenhsion, and code generation. 
- Results demonstrate improved performancce compared to standard  SFT/distillation and recursive hard label supervision.

Weaknesses:
- Writing could be improved, it is not clear when the input is in latent form and when it is in discrete embedding form
- it is not clear how are teachers trained or matched for each recursive block.
- The paper does not talk about the training or inference resources used
- The paper would benefit from an analysis of accuracy on the entire datasets using each recursive block.

QUESTIONS:
Q1: Can you please provide more information on how exactly are the teachers trained, how are they matched for each recursive block?
Q2: How much resources are required

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 2
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-03