SCOUT: Teaching Pre-trained Language Models to Enhance Reasoning via Flow Chain-of-Thought
Authors: Guanghao Li
Wenhao Jiang
Mingfeng Chen
Yan Li
Hao Yu
Shuting Dong
Tao Ren
Ming Tang
Chun Yuan
Venue: NeurIPS 2025 poster ()
Keywords: Recursive reasoning
Flow Chain-of-Thought
Primary area: applications

Abstract: Chain-of-Thought (CoT) prompting improves the reasoning performance of large language models (LLMs) by encouraging step-by-step thinking. However, CoT-based methods depend on  intermediate reasoning steps, which limits scalability and generalization. Recent work explores recursive reasoning, where LLMs reuse internal layers across iterations to refine latent representations without explicit CoT supervision. While promising, these approaches often require costly pretraining and lack a principled framework for how reasoning should evolve across iterations.
We address this gap by introducing **Flow Chain-of-Thought (Flow CoT)**, a reasoning paradigm that models recursive inference as a progressive trajectory of latent cognitive states. Flow CoT frames each iteration as a distinct cognitive stage—deepening reasoning across iterations without relying on manual supervision. To realize  this, we propose **SCOUT** (*Stepwise Cognitive Optimization Using Teachers*), a lightweight fine-tuning framework that enables Flow CoT-style reasoning without the need for pretraining. SCOUT uses progressive distillation to align each iteration with a teacher of appropriate capacity, and a cross-attention-based retrospective module that integrates outputs from previous iterations while preserving the model’s original computation flow.
Experiments across eight reasoning benchmarks show that SCOUT consistently improves both accuracy and explanation quality, achieving up to 1.8\% gains under fine-tuning. Qualitative analyses further reveal that SCOUT enables progressively deeper reasoning across iterations—refining both belief formation and explanation granularity. These results not only validate the effectiveness of SCOUT, but also demonstrate the practical viability of Flow CoT as a scalable framework for enhancing reasoning in LLMs.

SCOUT: Teaching Pre-trained Language Models to Enhance Reasoning via Flow Chain-of-Thought
Guanghao Li
Wenhao Jiang
Mingfeng Chen' Yan Li*
Hao Yu!
Shuting Dong'
Tao Ren
Ming Tang* Chun Yuan
'Tsinghua Shenzhen International Graduate School, Tsinghua University
2Southern University of Science and Technology
3 Guangdong Laboratory of AT and Digital Economy (SZ)
"The Hong Kong University of Science and Technology
GGuanghua School of Management, Peking University
Ligh24@mails tsinghua, edu. cawhjiang4gmal com
yuanc0sz. tsinghua. edu. cn tangm3@sustecc.. edu. cn
Abstract
Chain-of-Though (CoT) prompting improves the reasoning performance of large language models (LLMs) by encouraging step-by-step thinking. However, CoT- based methods depend on intermediate reasoning steps, which limits scalability and generalization. Recent work explores recursive reasoning, where LLMs reuse internal layers across iterations to refine latent representations without explicit CoT supervision. While promising, these approaches often require costly pretraining and lack principled framework for how reasoning should evolve across itera- tions. We address this gap by introducing Flow ohain-of-TThouhh (Flow CoT), reasoning paradigm that models recursive inference as a progressive trajectory latent cognitive states. Flow CoT frames each iteration as a distinct cognitive stage deee-nnepn reasoning across iterations without relying manual supervi- sion. To realize this, we propose SCOUT (Stepwise Cognitive Optimization Using Teachers), lightweight fine-tuning framework that enables Flow CoT-style reason- ing without the need for pretraining. SCOUT uses progressive distillation to align each iteration with teacher of appropriate capacity, and cooss-attention-aeee based retrospective module that integrates outputs from previous iterations while preserv- ing the model's i original computation flow. Experiments across eight reasoning benchmarks show that SCOUT consistently improves both accuracy and explana- tion quality, achieving up 1.8% gains under fine-tuning. Qualitative analyses further reveal that SCOUT enables progressively deeper reasoning across itera- tion--refining both belief formation and explanation granularity. These results not only validate the effectiveness of SCOUT, but also demonstrate the practical viability of Flow CoT as scalable framework for enhancing reasoning in LLMs.
Introduction
Chain-of-Thougee (CoT) reasoning improves large language models' (LLMss') performance by prompting them to generate intermediate steps that mimic human thought processes [ 2 Fun- damentally, CoT trades additional computation for improved accuracy. However, most CoT-based methods rely on supervised learning over curated step-by-step data, which limits their generalizability
Equual contribution Cooresponding author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

(3) Iteration 1 Complex Torget
(b) neration 2 Complex Target
[c] heeaaion Complex Target
Complex Target Distributton
Complex Detr
Complex Distribution
Itenitoo Model
0.4
0.2
0.1
0.0
10
12
12
10
(a) Uniform supervision: all iterations fit the same complex larget.
lteration Simple Target
(b) Ranation 2 Medium Target
Beration Complex Target
0.8
Simple Target Distribution
Mediuum Torg Distribution
Complex Target Distribution
0.7
Iteration 1 Model Fit
Iteration 2 Model Rit
merationn 3 Model Fit
0.6-
0.5
0.4
0.3
10
12
0
6 30 12
8
10
(b) Progressive supervision: each itcration fits an appropriate target.
Figure/ 1: Motivation comparison between progressive and uniform supervision training. Pro- gressive supervision aligns the supervision signal with the model's evolving capacity, avoiding over-regularization in early stages and undertraining in later ones.
and scalability [ To address these limitations, recent studies have explored recursive reasoning, in which LLMs are iteratively applied refine internal representations in latent space [5). While these approaches increase computational depth without requiring explicit CoT supervision, they often rely on costly pretraining and still lack principled framework for how reasoning should evolve across iterations. In this work, we extend this line of research and introduce Flow CoT), a reasoning paradigm that conceptualizes recursive inference as a progressive trajectory of cognitive states. Each recursive iteration in Flow CoT represents a refined reasoning stage, mirroring how human cognition gradually unfolds. Crucially, Flow CoT enables reasoning to emerge from recursive computation- without reliance on labeled CoT traces.
(Flow
Although Flow CoT provides compelling reasoning paradigm, existing recursive methods face two main obstacles in applying it effectively. First, they supervise all iterations using identical hard labels, ignoring that each iteration differs in capacity and function. Each iteration serves dual role: generating a plausible output and constructing an internal representation that supports future reasoning [6]. Applying strong supervision to early iterations can over-regularize them and distort intermediate states, thereby impairing downstream iterations. As illustrated in Figure ITa enforcing single target distribution across all iterations suboptimal for capturing the inherently progressive nature of reasoning. Second, most recursive methods implement simple strategies like feeding previous outputs back as inputs 7 or concatenating them with the original prompt [4] While these mechanisms are often used during pretraining, their reliance on modifying the model's onput-output structure makes them challenging integrate into fine-tuning pipelines.
To overcome these challenges. we propose SCOUT (Stepwise Cognitive Optimization Using Teach- ers), a lightweight fine-tuning framework that equips pretrained L.MMs with Flow CoT capabilities. SCOUT introduces a progressive distillation strategy, where each reasoning step is supervised by a teacher model of matching strength. As illustrated Figure [b] this enables each iteration refine its reasoning based on current capacity, rather than being constrained by uniform, overly strong supervi- sion targets. In addition, we incorporate a non-intrusive retrospective module based on cross- cross-attention. which allows each step to selectively attend to previous outputs while preserving the model's 's original reasoning flow. Together, these components preserve the base model's capabilities while significantly enhancing reasoning through fine-tuning alone. SCOUT thus serves as practical realization of the Flow CoT paradigm, offering a structured and cognitively grounded alternative to conventional recursive reasoning. Experiments across diverse benchmarks confirm consistent improvements in both accuracy and explanation quality, demonstrating the scalability and effectiveness of Flow CoT through fine-tuning alone.
2

---

We summarize our main contributions as follows:
We propose Flow Chain-of-Thought (Flow CoT), paradigm that conceptualizes recursive reasoning as a progressive, depth-aware cognitive trajectory.
We introduce SCOUT, a fine-tuning framework that instills Flow CoT capabilities in pre- trained LLMs via progressive distillation with multiple teacher models and a lightweight retrospective module, enabling recursive reasoning without additional pretraining.
We conduct extensive experiments across eight reasoning benchmarks. SCOUT not only achieves up to 1.8% accuracy gains under fine-tuning, but also consistently improves reasoning coherence and explanation quality.
2
Related Work
Chain-of-Thoughh Reasoning. Large language models (LLMs) have demonstrated remarkable capa- bilities across a wide range of tasks [9] [0 []]. Chain-of-Thouge (CoT) methods elicit intermediate reasoning steps in natural language enhance the performance LLMs. Early work demonstrates that prompting LLMs with exemplar reasoning chains can significantly improve their accuracy on complex tasks [2 Subsequent approaches fine-ture models to generate reasoning steps via su- pervised learning 14 15 or reinforcement learning [16 17 [8]. These methods increase the model's
effective reasoning depth by recursively incorporating generated thoughts into the input context [9] Recent advances focus on improving efficiency, including generating concise reasoning traces 20
deferring decisions until sufficient computation has occurred [21] or performing reasoning entirely in latent space using continuous representations [22 However, most CoT methods require explicitly annotated intermediate steps, limiting scalability and generalizability across domains.
Recursive LLMs models. Recurrent computation has been explored in various transformer designs, such as Universal Transformers [24] Looped Transformers 25 26), and Sparse Recurrent Trans- formers (27] These ideas have recently been adapted to LLMs to support multi-iteration reasoning via recursive applicpplintiinn 28
29 Recursive refinement deepens the reasoning trajectory
by reprocessing the model's own outputs 7 4]. However, most recursive methods rely on simple heuristics such as feeding previous outputs back as inputs g or concatenating them with the original prompt 4 While effective during pretraining, these strategies are often difficult to adapt for fine-tuning without modifying model architecture or training pipelines. addition, these methods typ- ically apply uniform supervision across all iterations, ignoring the evolving representational capacity at different depths and risking misaligned learning signals, particularly early-stage reasoning.
Knowledge Distillation. Knowledge distillation transfers information from a stronger teacher model smaller student, commonly to compress models while retaining performance [30 31
Standard approaches include sequence-level distillation (SeqKD) [33 and distribution matching via Kullback-Leeible divergence [34 35 36). Others align internal representations directly [37. However,
overly strong teachers can destabilize training, especially when student capacity is limited [38
Positioning Our Work. SCOUT provides a practical realization of Flow CoT by enabling multi- iteration reasoning without requiring explicitly annotated intermediate steps. Unlike prior recursive LLMs that apply uniform supervision or require extensive pretraining, SCOUT employs progressive distillation-alinnnnn each iteration with a teacher of matching capacity-and introduces lightweight retrospective module. This allows pretrained LLMs acquire depth- reasoning capabilities directly through fine-tuning.
Method
3.1
Flow Chain-of-Thought
Many recent reasoning methods for large language models (LLMs) rely on recursive self-application, yet they typically treat each iteration as black-box repetition without modeling the evolving nature of cognition. To address this, we introduce Flow Chain-of- Chain-of-Thought (Flow CoT), a general framework that models multi-step reasoning latent trajectory of progressively refined cognitive states.
Inspired by recent iterative reasoning frameworks Sosttonn-whiel we extend and reinterpret to support Flow CoT's stepwise cognitive model-
6. we adopt a standard three-part decom-
3

---

fral
2/30
Retrospaciion Modile
fo
fo
SA
Recuninv
Peachers
fua
fuil
Tall Block
Branch Pocnt
Hidden Stilso Flow
Teachery
Teachery
(a)
[b)
Figure 2: SCOUT Architecture. (a) Overall pipeline: pretrained LLM is decomposed into head block fnead, a recursive block fe. and a tail block fui The model performs T reasoning steps, each producing a latent state 2((), which is decoded by tail block and supervised by a capacity-matehh teacher. Teacher strength increases with é (i.e., Teacher, Teacher,-1) 1), enabling progressive learning. (b) The retrospective module integrates (t-11) via cross-attention (CA), while self-attention (SA) attends to the initial state (0),, enhancing step-to-ster coherence.
cog-consssiing of a head block fneadd a recursive block fs. and a tail block fual. This design enables the head block to encode the input once, the recursive block iteratively refine internal representations, and the tail block to decode the final output.
Given discrete input prompt v; the head block produces initial latent state 20) Shead (z)), which then refined through 7 recursive steps. The first iteration applies the recursive block directly to the initial state:
(z)
while subsequent iterations integrate both the original context and intermediate reasoning traces:
20) = fa (0(=))))((=11)) -((-1)). t= 2..,...,
Here, is history integration function that merges the fixed input context with the evolving internal state. This formulation reflects a key assumption of Flow CoT: effective reasoning requires both retention of prior progress and grounding in the original question. The integration of 2 (0) and (t-1) ensures that each step is guided by both accumulated reasoning and the fixed problem context. After T steps, the final latent state is decoded into discrete prediction y (z))
We define the sequence (2(2), ..... 2(²') the model's cognitive trajectory. Each z(1) functions as an intermediate subgoal-an internal representation that is not merely a step toward the answer, but a meaningful reasoning state that can (and should) be explicitly optimized. This reflects the core principle of Flow CoT: recursive reasoning should not be viewed as black-box repetition, but as structured cognitive evolution-whan each step contributes distinct phase in a progressive latent reasoning trajectory. Unfortunately, existing recursive methods often apply identical supervision (e.g., same output labels) all iterations, ignoring the fact that earlier reasoning stages have limited representational capacity and distinct functional roles. This misalignment can distort latent states and harm downstream refinement an issue address in Section 3.3 through progressive distillation.
3.2 SCOUT: Stepwise Cognitive Optimization Using Teachers
While Flow CoT provides a thcoretical formulation of latent, multi-iteration reasoning, current recursive methods [5] 7 lack training strategies that explicitly pursue this trajectory. In particular, most existing recursive approaches supervise all iterations using identical targets, ignoring the evolving representational capacity of each shep-thus deviating from the core principle of Flow CoT.
To bridge this gap, we propose SCOUT (Stepwise Cognitive Optimization Using Teachers), lightweight fine-tuning framework that equips pretrained LLMs with Flow CoT capabilities through recursive refinement. Building directly on the Flow CoT formulation, SCOUT adopts the same modular decomposition: head block /hend a recursive reasoning block fe. and a tail block fun- As shown in Figure?? 2 the input is encoded once, the latent state iteratively updated over T steps fe. and the final output is decoded from the last state using fual-

---

While this architecture aligns with the Flow CoT formulation, effectively training the model to follow such recursive trajectory introduces unique challenges. First, each reasoning step should be supervised in a way that reflects its current level of abstraction, rather than applying uniform supervision across all iterations. Second, the model must be able incorporate reasoning from earlier steps without disrupting the pretrained architecture's data flow and attention structure. To address these issues, SCOUT introduces two key mechanisms each integrated with minimal architectural modification:
Progressive Distillation: As shown in Figure 2 a), each latent state 2(() is decoded and aligned with teacher of matching capacity. Later steps guided by stronger teachers (e.g.. larger LLMs), enabling step-wise cognitive refinement while avoiding over-regularization in early stages. This is implemented via per-step KL divergence losses (see Section (.3)
As shown in Figure b)) we introduce a cross-attention module that fuses the prior state z((-1) into the current update. The model performs self-attention over the original latent state 20 while attending to previous reasoning traces via cross- attention. This allows each step to selectively reuse earlier thoughts while maintaining input grounding (see Section 3.4.
Retrospective Reasoning:
Together, these mechanisms enable SCOUT realize Flow CoT using only recursive application and fine-tuning-witee requiring annotated intermediate steps or architecture-leve modifications. SCOUT supports step-wise alignment, latent coherence, and efficient deployment within standard LLM pipelines.
3.3 Progressive Distillation
Flow CoT assumes that different reasoning steps reflect dif- ferent levels of abstraction and cognitive complexity. Early steps may produce tentative or exploratory reasoning, while later steps are expected to perform more precise inference. To support this progression, we propose progressive distillation, supervision strategy that aligns each step with a teacher model of matching capacity.
KL Divergence (Mean Std)
3.2
Oierpence
2.0
Unlike prior methods that apply the same supervision target every iteration, we assign each latent state 2+)) soft target distribution q(t) generated by step-specific teacher model. Formally, given an input z, we define:
6.6
150
38
75
145
205
Teacher
(t) Softmax(/fexheer, (2)),
(3)
Figure 3: KL divergence on Dolly dataset.
where freacbor, is pretrained teacher LLM selected match the expected capacity of step é. Early steps use smaller or distilled
models (e.g., 0.5B, 1.5B), while later steps are supervised by increasingly powerful teachers (c.g..
7B, 14B, 32B), as illustrated in Figure a()..
To empirically justify this approach, (Qwen2.5-0.58 (41)) and a range of teacher models of increasing size, using 100 instructions sampled from the Dolly dataset [42). As shown in Figure 3 the KL divergence increases steadily with teacher size. This indicates that larger teachers produce sharper and more complex distributions-coneeee richer learning signals. It also validates our hypothesis that step-specific supervision should be capacity-aware: stronger teachers should guide later steps that have greater representational strength.
measured the KL divergence between student model
For each reasoning step {L,..., T), the model produces student distribution:
(4) Softmax(fuul ()))..
Pa
(4)
which is optimized to match (0) through a distillation loss:
((t) KI. (x()) (1 + Lit)
(5)
where the second term is optional hard-label loss defined as:
c CE(p!) (t) .y"),
(6)
5

---

with y" denoting the ground-trun truth output tokens and CE denoting cross gross-entropy,
Compared to uniform supervision, this strategy allows each step optimize toward learning target that matches its stage of cognitive development. This not only prevents over-regularization in early stages, but also ensures that stronger guidance is applied where the model is ready to absorb it. When access is limited to single large model, one can first self-distill size-stratified set of students and assign them to recursion steps from weakest strongest, thereby preserving capacity matching.
3.4 Retrospective Reasoning Module
Recursive reasoning requires the model to build upon its prior cognitive states while staying grounded in the original input. However, pretrained LLMs are typically optimized for single-pass processing, where self-attention is computed over a fixed input sequence. Simply feeding previous outputs back into the prompt modifying layer behavior often disrupts this pretrained flow-leading leading to suboptimal performance or the need for retraining [4.
To address this, we design a lightweight retrospective reasoning module that enables information flow across steps while minimally altering the model's original computation logic. Our design principle emphasizes simplicity and compatibility: each reasoning step reprocesses the original input via self-attention, while selectively attending to the prior step's ' latent state via cross- tross-attention. This allows the model to treat previous state 2(t-1) as external memory, while maintaining stable computation over the static input embedding =()),
Formally, we implement the history integration function H. as introduced in Eq. 2 where each 2()) updated by fusing 20)) and (?-11) via a two-stream attention mechanism. Note that this mechanism is applied only for 2 2, as the initial step z(1) is computed directly from z(() without retrospective integration, as defined in Eq. (T. Specifically, the model uses self-attention over the original state =(0), and cross-attention over the previous step's output =(t-1), as shown in Figure b). This setup provides two key benefits: (I) it preserves the model's original dataflow over (0)), avoiding destructive interference with pretraining behavior; and (2) it allows flexible, content-based retrieval from the prior step, making the reasoning process both coherent and adaptive.
Compared to alternative fusion methods (e.g., additive mixing gated updates 47),, cross-attention offers a modular, query-driven mechanism. preserves the representational separation between the current task context and past cognitive traces. We refer to this design XAttn in subsequent analysis, to distinguish it from other integration strategies. Unlike methods that modify internal layer behavior [4), our approach introduces only a shallow wrapper-requing requiring neither architectural rewiring nor re- e--intiialization of pretrained weights. This design aims to enable coherent multi-step reasoning while remaining fully compatible with pretrained LLMs.
3.5 Training and Inference
Training. SCOUT is trained via standard fine-tuning, but with supervision applied across T internal reasoning steps rather than only the final output. In our experiments, 7 is treated as fixed hyper- parameter (e.g., T 3), though can be extended to dynamic reasoning schedules in future work. At each step E ( TY. the model refines its latent state z(() using the retrospective module and recursive block (see Section 3.4). Each z() is decoded and supervised using capacity- matched teacher, as described in Section 3.3 The total training loss is the weighted sum of per-step losses:
L
Σ A L
This formulation enables the model to learn step-wise cognitive refinement through end-to-end backpropagation, without architectural changes or pretraining modifications.
Inference. At inference time, the model follows the recursive update described in Eq. initializes 20() via Sheat. and iteratively updates the latent state across T steps. The final state 2(T)
It
decoded by full to produce the output. Optional early stopping can applied based on output entropy inter-step consistency.

---

Efficiency. SCOUT reuses existing model layers, adds only shallow cross- atts-nattnnion modules, and requires recursive passes per input. By preserving the model's original reasoning flow, remains fully compatible with off aff-th LLMs and standard fine-tuning workflows.
Experiments
4.1 Experimental Setup
Model architecture. We use the Qwen2.5 series 43 and select Qwen2.5-0.5B as the student model. Recursive fine-tuning is applied under the SCOUT framework with T = reasoning iterations. For progressive distillation, we assign Qwen2.5-1.55 5-,5B, Qwen2.5-3B, and Qwen2.5-7BB as teacher models for reasoning iterations 1, 2, and 3, respectively. See Appendix A.3 for additional implementation
details.
Training configuration. We fine-tune the model for 2 epochs with learning rate of 2 10-3 using the LlamaFactory framework [44]. We fine-tune the model using a composite instruction- following dataset, which is a mix of five diverse instruction- tunonnntttnnnggnnnnn datasets. These datasets aim to improve the model's ability to follow specific instructions and reason through tasks step-by-step. The mixed dataset includes data from: (1) Alpaca GPT4 [45] and Alpaca CoT [46] (general instruction- following and chain-of-thought reasoning). (2) WikiQA 44 (open-domain question answering), (3)
CodeAlpaca 48 (code generation), and (4) MathInstruct 114 (multi-step mathematical reasoning).
For distillation, we apply the Adaptive Bellback--eibller (AKL) [49] method to align student outputs with teacher distributions. Unless otherwise specified, we set A 1/3 for all t. assigning equal weight to each reasoning iteration. Further implementation details are provided in the Appendix A.
Evaluation benchmarks. To assess the performance of the fine-tuned model, evaluate it using the irvaluation-harnese framework [50] across a broad set of benchmarks, categorized into four areas: (i) commonsense QA, including ARC-easy, ARC-challenge [51], OpenBookQA [52]. and TruthfulQA [53): (ii) mulli-step reasoning, such as GSM8K [54] and MMLU (55); (iii) reading
comprehension and dialogue, with CoQA (56) and GLUE [57]; and (iv) code generation, using
MBPP [58] These evaluation benchmarks are used to measure how well the model performs after fine-tuning, reflecting its reasoning ability across a range of domains.
Baselines. We compare SCOUT against a series basclines to isolate the effect of recursive refinement and supervision strategies. All recursive variants share the same architecture based on simple layer stacking, ensuring that differences arise solely from how supervision is applied across iterations.
SFT (Supervised Fine-Tuning): Standard supervised fine-tuning on the same training data, where the loss is applied only at the final output, with no recursive refinement. DSFT (Distilled-SFT) [49] Fine-tunes the model using soft targets from Qwen2.5-7B, but without any recursive computation.
R-SFT (Recursive Hard-Label Supervision) 77 Fine-tuning within the recursive frame- work, but with hard-label supervision at each step (i.e., each iteration applies standard supervised learning using ground-truth labels, without progressive distillation).
R-Distill-EQ (Recursive Distillation with Equal Weights): Uses the recursive architecture with a fixed 7B teacher for all iterations and applies equal loss weights X X2 = >3 1/3 to provide uniform supervision.
R-Distill-Wi -TT (Recursive Distillation with Weighted Loss) 8]: Similar to R-Distill-EQ. but applies increasing weights at each iteration reflect the growing abstraction: \ 0.2,
A2 = 0.3, Ag = 0.5, with stronger supervision on later reasoning stages.
R-SCOUT (Reversed SCOUT): A control variant where the teacher order is reversed (7B
++ 3B + 1.5B), violating the progressive distillation strategy and providing insights into the impact of teacher progression order.
4.2 Comparative Analysis with Baselines
We analyze the results Table comparing SCOUT with a range of baseline methods, in order to assess the impact of recursive design and supervision strategies.
7

---

Table 1: Experimental results for different methods and iterations. A denotes the improvement relative to SFT; positive values are highlighted blue, negative in red. Iter. denotes the number of recursive steps. All results are averaged across tasks. Both Avg and values are computed
before rounding, and then rounded to two decimal places. Abbreviations: OB OpenBookQA; e
ARC-Easy; - ARC- Challenge; TF TruthfudQA.
Method
Iter.
OB
GSM8K
MBPP
ARC-e
ARC-c
TF
CoQA
GLUE
Avg
A
SFT
23.8
32.07
27.6
66.58
30.72
26.44
45.95
44.54
37.21
DSFT
23.8
33.59
27.4
64.94
27.47
26.81
48.97
51.55
38.06
+0.85
24.6
31.84
27.4
66.08
31.83
25.83
44.62
47.72
37.49
+0.27
R-SFT
25.2
32.22
27.0
64.94
32.08
26.93
44.88
48.61
37.73
+0.52
24.6
32.22
27.0
64.31
32.34
26.81
45.40
47.28
37.49
+0.28
35.10
28.0
65.95
28.24
26.68
47.80
52.02
38,44
+1.23
R-Distill-EU
23.8
33.81
26.2
63.64
28.50
26.81
46.77
47.84
37.17
-0.04
22.2
33.59
27.4
63.30
28.16
26.93
47.85
49.49
37.36
+0.15
24.0
33.59
27.6
65.87
28.50
26.81
48.70
50.77
38.23
+1.01
R-Distill-WT
23.2
32.68
27.6
63.89
27.82
26.68
48.18
51.97
37.75
+0.54
22.8
33.97
27.6
64,39
28.67
26.93
48.48
51.87
38.08
+0.87
24.8
34.87
28.4
66.71
29.44
25.95
48.60
50.75
38.69
+1.47
R-SCOUT
21.4
33.66
28.0
65.07
28.33
27.05
48.60
50.84
37.86
+0.65
20.4
32.60
26.8
62.50
26.96
26.81
49.28
48.95
36.78
-0.42
21.8
33.46
27.6
63.72
27.05
26.07
49.47
50.37
37.44
+0.23
SCOUT
2
25.2
34.39
27.2
63.83
29.44
27.05
48.18
50.81
38,26
+1.05
24.0
35.45
28.2
64.73
30.20
28.56
48.75
52.35
39.03
+1.81
Distillation and recursion individually help, but remain limited. DSFT, which applies soft targets from a strong teacher without recursion, achieves modest improvement over SFT (+0.85), confirming that teacher signals improve calibration. Similarly, R-SFT and R-Distill-BQ add recursive structure by supervising each iteration independently. While these variants show initial gains (e.g.. R-SFT: +0.27 + +0.52 +0.28), performance stagnates or regresses later steps (R-Distill-EO) +1.23 + -0.04 + +0.15). This suggests that without step-specific guidance, recursion alone fails to support progressive refinement and can even introduce noise that hinders later-stage reasoning.
Simple weighting helps-but only partially. To mitigate early over-regularization, R-Distill-WT applies loss weights that increase across iterations (Ag 0.5). This stabilizes later performance compared to R- R-Distill-EQ (e.g., +0.87 vs. +0.15 at step 3), but overall gains remain modest. Since all steps still rely on fixed teacher (7B), the model cannot align supervision with its evolving capacity-hhihlighhing that weighting alone cannot substitute for progressive teacher guidance.
SCOUT achieves coherent, step-wise refinement. SCOUT outperforms all baselines, with mono- tonic improvement across iterations (+0.23 - +1.05 - +1,81), and highest final performance (c.g.. 35.5 GSM8K. 52.4 GLUE). These gains validate the Flow CoT principle: effective multi-step reasoning requires -erratoon-specific supervision and cognitive reuse. In contrast, R-SCOUT, which reverses teacher order (7B - 3B 1.5B), initially achieves strong results (+1.47) but collapses in later steps (-0.42), confirming the necessity of aligning teacher strength with iteration depth.
Our findings reveal three insights: (1) recursion and distillation must be combined to support depth- aware reasoning; (2) fixed or reversed teacher strategies can degrade performance; and (3) SCOUT uniquely enables progressive cognitive refinement through modular architecture, aligned supervision, and cross-step integration.
4.3 Effect of Retrospective Integration Mechanisms
To evaluate the impact of retrospective reasoning design, we compare our cross-attention approach (XAttn) with five common alternatives: initial injection (Init) (5), additive fusion (Add) [4] concat- and-project (CatProj), gate-based control (Gate), and N-style modulated injection (ModInj).
See Appendix B.3 f.3 for detailed implementations.

---

Table 2: Average accuracy across benchmarks using different retrospective modules.
Iter.
Init
Add
CatProj
Gate
ModInj
XAttn (ours)
37.18
37.59
36.36
33.85
34.55
37.49
2
34.10
35.38
27.75
27.94
32.16
37.73
3
27.11
29.09
23.95
27.09
28.98
37,50
Naive fusion strategies such as Init, Add, and CatProj show diminishing returns with deeper iterations, suggesting limited support for long-horizon reasoning. ModInj and Gate partially improve early performance but lack stability. Only our proposed XAttn yields consistent and monotonic improvements across all steps. This confirms the importance of query-driven, modular integration: XAttn enables cognitive coherence across iterations while preserving compatibility with pretrained inference flows-core core to SCOUT's recursive design.
4.4 Qualitative Analysis: Iterative Cognitive Refinement
To better understand how SCOUT progressively refines its reasoning through recursive steps, we present two qualitative visualizations. These illustrate how SCOUT incrementally improves both its answer accuracy and reasoning structure through iteration.
Tricensi mvel Probabi by Quectio: "6-3ee Cornect Anewer:
0.24
5:00
0.53
0.00
5:00
0.02
Question
4.2
librar had 200 books, purchaood 10 new books and removed 15 damaged creas. many boora are library now?
0.22
212
0.22
0.22
0.16
8.16
6.33
0.12
616
6.1
Scout Bar 1 Answer
lobrry now. answor
0.34
1.0
0.53
0.57
0.30
0.05
0.53
0.06
6.96
0.65
The lnrwyy foed 250 tooks intiaaly puechoooe 10 bookss haw 250 10 260 books remmeeed to damagee houios, sn hadd 2800 245 bookoe Therelore, there sew 245 booker in the library
0.24
1:00
0.53
0.53
Scout her Answer
0.24
1:05
0.53
3.04
5.55
0.53
number bocks and
comageed booke. Stort with
the retal number of booker 250 2 the number of bookee 250
10 -oen Suetarr the numbie dhmrgged 2ro0be:780 15 245
0.16
0.22
0.22
0.60
SEOUT
SCOUT
1:22
ka1
Figure 4: Token-level prediction probabilities on the math prompt "6 =". SCOUT gradually shifts probability mass toward the correct answer (3) across iterations, unlike flat pattern in R-Distil--EQ.
Figure 5: Reasoning trace across SCOUT iterations for book-counting question. The model transitions from incorrect shallow reasoning accurate multi- step explanations.
Refining belief through iteration. Figure 4 presents token-level probability heatmap for the math problem "6 - =". The correct answer is 3. In R-Distill-EQ. all iterations assign high probability to token 1, and predictions stagnate. In contrast, SCOUT exhibits distinct shift in attention: the first iteration mirrors R- A-Distill-EO output (1 dominates), but the second iteration begins to assign more weight to 3, and by the third iteration, 3 emerges the top prediction. This trajectory illustrates how SCOUT performs latent search over cognitive space, adjusting its belief toward a more plausible output step-by-step.
Improving reasoning quality across iterations. Figure showcases the evolution of SCOUT's natural language reasoning across three iterations simple arithmetic question. The first response is syntactically valid but mathematically incorrect (computes 250 10 15 235). The second iteration corrects the logic and provides more faithful solution (260- 15 = 245). The third response further improves by structuring the explanation into ordered sub-steps, reflecting improved decomposition and explanation clarity. This illustrates how recursive refinement in SCOUT not only improves accuracy, but also enriches the reasoning granularity and explanatory quality.
These examples confirm that SCOUT's iterative mechanism is not merely repeating inference but actively refining both the belief space (token predictions) and the reasoning space (explanation quality) over time tine-embodying the core principle of Flow COT.

---

5 Conclusion
We present Flow -foooght (Flow CoT), reasoning paradigm that models multi-step inference as progressive trajectory of latent cognitive states. Unlike traditional Chain-of- -fhoughh prompting, which relies on manually curated step-by-step traces, Flow CoT enables reasoning to emerge organically through recursive computation- -oftettriiooofffingggg a scalable and cognitively grounded alternative. To instantiate this paradigm, we introduce SCOUT, a lightweight fine-tuning frame- work that equips pretrained LLMs with Flow CoT capabilities. SCOUT combines progressive distillation- where each reasoning step is supervised by a capacity-matchen teacher-with a cross- attention-based retrospective module that enables selective integration of prior outputs. Importantly, SCOUT requires no additional pretraining. Across cight benchmarks, SCOUT consistently improves both final answer accuracy and explanation quality, validating the effectivvness and practicality of Flow CoT under a fine-tuning regime.
Limitations and Future Work. SCOUT currently uses fixed number of reasoning steps and manu- ally selected teacher models, which provide clarity and training stability but may limit adaptability to tasks of varying complexity. Future work could explore dynamic iteration control or adaptive teacher selection, for example through Sinfforeeeent-learing---- intended as general reasoning framework, it opens opportunities for new architectures and training strategies that support structured, cognitively motivated inference. We hope this work encourages further exploration of principled recursion in language model design.
scoring. More broadly, as Flow CoT
Acknowledgments
This work was supported by the National Key R&D Program of China (2022YFB4701400/47704444 SSTIC Grants IKIZD202309231155 106601, KIZD202309231 1992113449632, GJHZ20240218 13344400088 the National Natural Science Foundation of China under Grant (62202214), and the Guangdong Basic and Applied Basic Research Foundation under Grant (2033A1515012819),
References
[1] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-24832 2022.
[2] Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. Automatic chain of thought prompting in large language models. arXiv preprint 2xXvv22110033993 2022.
[3] Shizhe Diao, Pengcheng Wang, Yong Lin, Rui Pan, Xiang Liu, and Tong Zhang. Active prompting with thain-of-thougle for large language models. arXiv preprint arX1::2302.12246, 2023.
[4] Jonas Geiping, Sean McLeish, Neel Jain, John Kirchenbauer, Siddharth Singh, Brian R Bartold- son, Bhavya Kailkhura, Abhinav Bhatele, and Tom Goldstein. Scaling up test-time compute with latent reasoning: A recurrent depth approach. arXiv preprint arXiv:2502.05177, 2025.
[5] Nikunj Saunshi, Stefani Karp, Shankar Krishnan, Sobhan Miryoosefi, Sashank Jakkam Reddi, and Sanjiv Kumar. On the inductive bias of stacking towards improving reasoning. Advances in Neural Information Processing Systems, 37:71437-71464, 2024.
[6] Guanghao Li, Wenhao Jiang, Li Shen, Ming Tang, and Chun Yuan. Zero token-driven deep thinking in Ims: Unlocking the full potential of existing parameters via cyclic refinement. arXiv preprint arXiv:2502. /22 14, 2025.
[7] Nikunj Saunshi. Nishanth Dikkala, Zhiyuan Li, Sanjiv Kumar, and Sashank J Reddi. Reasoning with latent thoughts: On the power of looped transformers. arXiv preprint arXiv:2502. 17416, 2025.
[8] Sangmin Bae, Adam Fisch, Hrayr Harutyunyan, Ziwei Ji, Seungyeon Kim, and Tal Schuster. Relaxed recursive transformers: Effective parameter sharing with laser-wise lora, arXiv preprint arXiv:2410.2067. 2024.
10

---

[9] Yongxian Wei, Anke Tang, Li Shen, Zixuan Hu. Chun Yuan, and Xiaochun Cao. Modeling multi-task model merging as adaptive projective gradient descent. In serty- International Conference Machine Learning. 2025.
[10] Yongxian Wei, Zixuan Hu, Li Shen, Zhenyi Wang. Chun Yuan, and Dacheng Tao. Open- vocabulary customization from clip via data-free knowledge distillation. In The Thirteenth International Conference on Learning Representations, 2025.
[11] Yongxian Wei, Runxi Cheng, Weike Jin, Enneng Yang. Li Shen, Lu Hou, Sinan Du, Chun Yuan, Xiaochun Cao, and Dacheng Tao. Unifying multimodal large language model capabilities and modalities via model merging. arXiv preprint arXiv:2505. 19892, 2025.
[12] Tushar Khot, Harsh Trivedi, Matthew Finlayson, Yao Fu, Kyle Richardson, Peter Clark, and Ashish Sabharwal. Decomposed prompting: A modular approach for solving complex tasks. arXiv preprint arXiv:2210.02406, 2022.
[13] Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang. Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoe Le, et al. Least-to-most prompting enables complex reasoning in large language models. arXiv preprint arXiv:2205. 10625, 2022.
[14] Xiang Yue, Xingwei Qu, Ge Zhang, Yao Fu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. Mammoth: Building math generalist models through hybrid instruction tuning. arXiv preprint arXiv:2309.05665: 2023.
[15] Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. Metamath: Bootstrap your own mathematical questions for large language models. arXiv preprint arXiv:2309. /2284, 2023.
[16] Peiyi Wang, Lei Li, Zhihong Shao, RX Xu, Damai Dai. Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui. Math-shepherd: Verify and reinforce Ilms step-by-step without human annotations. arXiv preprint 13X.::2312.08935.. 2023.
[17] Fangxu Yu, Lai Jiang, Haoqiang Kang, Shibo Hao, and Lianhui Qin. Flow of reasoning: Efficient training of Ilm policy with divergent thinking. arXiv preprint arXiv:2406.05673 2024.
[18] Alex Havrilla, Yuqing Du, Sharath Chandra Raparthy, Christoforos Nalmpantis, Jane Dwivedi- Yu, Maksym Zhuravinskyi, Eric Hambro, Sainbayar Sukhbaatar, and Roberta Raileanu. Teaching large language models to reason with reinforcement learning. arXiv preprint 0e0:24403.44422 2024.
[19] Guhao Feng. Bohang Zhang, Yuntian Gu, Haotian Ye, Di He, and Liwei Wang. Towards revealing the mystery behind chain of thought: a theoretical perspective. Advances in Neural Information Processing Systems, 36:77757-70798, 2023.
[20] Aman Madaan and Amir Yazdanbakhsh. Text and patterns: For effective chain of thought, takes two to tango. arXiv preprint arXiv:2209.07686 2022.
[21] Eric Zelikman, Georges Harik, Yijia Shao, Varuna Jayasiri, Nick Haber, and Noah D Goodman. Quiet-star: Language models can teach themselves to think before speaking. arXiv preprint arXiv:2403.00622 2024.
[22] Yuntian Deng, Kiran Prasad, Roland Fernandez, Paul Smolensky, Vishrav Chaudhary, and Stuart Shieber. Implicit chain of thought reasoning via knowledge distillation. arXiv preprint arXiv:2311.01460, 2023.
[23] Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. Training large language models to reason in a continuous latent space. arXiv preprint a4Xiv:2412.06769, 2024.
[24] Mostafa Dehghani, Stephan Gouws, Oriol Vinyals, Jakob Uszkoreit, and Lukasz Kaiser. Uni- versal transformers. arXiv preprint arXiv: 038 2018.
[25] Angeliki Giannou, Shashank Rajput, Jy-yong Sohn, Kangwook Lee, Jason D Lee, and Dimitris Papailiopoulos. Looped transformers as programmable computers. International Conference on Machine Learning, pages 11398-11442 PMLR, 2023.
11

---

[26] Ying Fan, Yilun Du Kannan Ramchandran, and Kangwook Lee. Looped transformers for length generalization. arXiv preprint arXiv:2409. 15647. 2024.
[27] Shawn Tan, Yikang Shen, Zhenfang Chen, Aaron Courville, and Chuang Gan. Sparse universal transformer. arXiv preprint 10.00023,0.0709, 2023.
[28] Dahyun Kim, Chanjun Park, Sanghoon Kim, Wonsung Lee, Wonho Song, Yunsu Kim, Hyeon- woo Kim, Yungi Kim, Hyeonju Lee, Jihoo Kim, et al. Solar 10.7 b: Scaling large language models with simple yet effective depth up-scaling. arXiv preprint arXiv:2312.15166, 2023.
[29] Mark Schöne, Babak Rahmani, Heiner Kremer, Fabian Falck, Hitesh Ballani, and Jannes Gladrow. Implicit language models are RNNs: Balancing parallelization and expressivity, In Forty-second International Conference on Machine Learning. 2025.
[30] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv: wtt.r:1033..33.. 2015.
[31] Jianping Gou, Baosheng Yu, Stephen J Maybank, and Dacheng Tao. Knowledge distillation: A survey. International Journal of Computer Vision, (2966):789-1881)) 2021.
[32] Pengguang Chen, Shu Liu, Hengshuang Zhao, and Jiaya Jia. Distilling knowledge via knowledge review. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5008-5011 2021.
[33] Yoon Kim and Alexander M Rush. Sequence-lever knowledge distillation. In Proceedings of the 2016 conference on empirical methods natural language processing, pages 1317-1327, 2016.
[34] Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv: 108, 2019.
[35] Hongzhan Chen, Xiaojun Quan, Hehong Chen, Ming Yan, and Ji Zhang. Knowledge distillation for closed-source language models. arXiv preprint arXiv:240..07013, 2024.
[36] Inar Timiryasov and Jean-Loup Tastet. Baby Ilama: knowledge distillation from an en- semble of teachers trained on small dataset with no performance penalty. arXiv preprint arXiv:2308..m 02019, 2023.
[37] Chen Liang, Simiao Zuo, Qingru Zhang, Pengeheng He, Weizhu Chen, and Tuo Zhao. Less is more: Task-aware layer-wise distillation for language model compression. In International Conference on Machine Learning, pages 20852-20861 20867.. PMLR, 2023.
[38] Jang Hyun Cho and Bharath Hariharan. On the efficacy of knowledge distillation. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4794-4802, 2019.
[39] Seyed Iman Mirzadeh, Mehrdad Farajtabar, Ang Li, Nir Levine, Akihiro Matsukawa, and Hassan Ghasemzadeh. Improved knowledge distillation via teacher assistant. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 5191-5198, 2020.
[40] Dan Busbridge, Amitis Shidani, Floris Weers, Jason Ramapuram, Etai Littwin, and Russ Webb. Distillation scaling laws. arXiv preprint arXiv:2502.08666, 2025.
[41] Qwen Team. Qwen2.5: A party of foundation models, September 2024.
[42] Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Jun Wan, Sam Shah, Ali Ghodsi, Patrick Wendell, Matei Zaharia, and Reynold Xin. Free dolly: Introducing the world's first truly open instruction-tuned 11m, 2023.
[43] An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei. Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng He, Junyang Lin, Kai Dang, Keming Lu, Keqin Chen, Kexin Yang, Mei Li, Mingfeng Xue, Na Ni, Pei Zhang, Peng Wang, Ru Peng, Rui Men, Ruize Gao, Runji Lin, Shijie Wang, Shuai Bai, Sinan Tan, Tianhang Zhu, Tianhao Li, Tianyu Liu, Wenbin Ge, Xiaodong
12

---

Deng, Xiaohuan Zhou, Xingzhang Ren, Xinyu Zhang, Xipin Wei, Xuancheng Ren, Yang Fan, Yang Yao, Yichang Zhang, Yu Wan, Yunfei Chu, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zhihao Fan. Qwen2 technical report. arXiv preprint 2407. 10671, 2024.
[44] Yaowei Zheng, Richong Zhang. Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. Llamafactory: Unified efficient fin-tuning of 100+ language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), Bangkok, Thailand, 2024. Association for Computational Linguistics.
[45] Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. Instruction tuning with gpt-4. arXiv preprint 20Xiv:2304.03377,, 2023.
[46] Zheng Lin Qingyi Si. Alpaca-cot: An instruction fine-tuning platform with instruction data collection and unified large language models interface. htstp:://gittubb.com//obbbbbl alpaca-CoT 2023.
[47] Yi Yang, Wen-tau Yih, and Christopher Meek. WikiQA: A challenge dataset for open-domain question answering. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2013-2018, Lisbon, Portugal, September 2015, Association for Computational Linguistics.
[48] Sahil Chaudhary. Code alpaca: An instructionnfoolloinng llama model for code generation. https://gi https:://githb.... cotb wtt.c////aaa1111/////ooaoaooo coccaaaaccccccaccaaaaaaaaaa 2023.
[49] Taiqiang Wu, Chaofan Tao, Jiahao Wang, Runming Yang. Zhe Zhao, and Ngai Wong. Rethinking kullback-leibler divergence in knowledge distillation for large language models. arXiv preprint arXiv:240..02657, 2024.
[50] Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac'h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. The language model evaluation harness, 07 2024.
[51] Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. ArXiv, abs// 1801 300.055577 2018.
[52] Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can suit of armor conduct electricity? new dataset for open book question answering. In EMNLP, 2018.
[53] Stephanie Lin, Jacob Hilton, and Owain Evans. TruthfulQA: Measuring how models mimic human falsehoods. In Proceedings of the 60th Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Long Papers), pages 3214-3252, Dublin, Ireland, May 2022. Association for Computational Linguistics.
[54] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Jacob Hilton, Reiichiro Nakano, Christo- pher Hesse, and John Schulman. Training verifiers to solve math word problems, 2021.
[55] Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. Proceedings of the International Conference on Learning Representutions (ICLR), 2021.
[56] Siva Reddy, Danqi Chen, and Christopher D Manning. Coqa: A conversational question answering challenge. Transactions of the Association for Computational Linguistics, 7:249-266. 2019.
[57] Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. Glue: A multi-task benchmark and analysis platform for natural language understanding. arXiv preprint arXiv: 07441, 2018
[58] Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. Program synthesis with large language models. arXiv preprint arXiv:2108.077:: 07732, 2021.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction clearly describe Flow CoT as reasoning paradigm and SCOUT as its instantiation. All stated claims (c.g., improvements in reasoning accuracy and explanation quality, no need for pretraining) are supported by the experimental results.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
It is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
The limitations SCOUT are discussed the "Conclusion and Future Work" section, including the use of fixed iteration depth and manually assigned teacher models. These are framed as simplifications to support initial exploration of Flow COT.
Justification:
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory assumptions and proofs
14

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: The paper focuses on method design and empirical evaluation. It does not present formal theoretical theorems or proofs.
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Section Experiments details model configurations, iteration schedules, datasets used, teacher models, and optimization settings. Appendix provide additional architectural and training details.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to model. In general. releasing code and data often one good way to accomplish this, but reproducibilityy can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open- open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the partieular way they provide for reproducibiliity In the case of closed-source models, it may be that access to the model is limited in
15

---

some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [No]
Justification: All datasets used in this work publicly available. Our model training follows prior works, and the detailed implementation and training settings are provided in Section 4 and the Appendix.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips cc/
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
7 //nips.cc/publ::: hwaplli//guideee guwdgs/dd///ee Cttp///ooeeSSeieeeeeeeeeee wwwteeessuuiiiiiiiniiiiiii wwwsssssiiiiyyooo for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Appendix specify all key hyperparameters (learning rate, epochs, loss weights), data composition, teacher assignments, and number of iterations.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Figure presents error bars (standard deviation) for KL divergence across teacher models. For benchmark results (Table 1), we report mean accuracy as computed by the Im-evaluation - framework, which internally averages over multiple prompt variants or input instances per task. While formal confidence intervals multiple training runs were not performed due to resource constraints, the evaluation based on fixed validation splits with standardized metrics, ensuring meth- comparability.
16

---

Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Section Experiments states the number of iterations and training epochs, and further hardware details are included in Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported didn't 'tttt make it into the paper).
paper (e.g., preliminary or failed experiments that
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics htttp/////oaarrpp...cc////llalat
Answer: [Yes]
Justification: The research does not involve any human subjects, private data, or potentially harmful content. All datasets used open-source and widely adopted in academic research.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative socictal impacts of the work performed?
17

---

Answer: [Yes]
Justification: Potential societal impacts-suche as improving the transparency and accuracy of language models-are discussed in the Broader Impact paragraph of the Appendix. We also acknowledge the risk of misuse in automatic reasoning and suggest safeguards.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
The models trained this work are derived from open-source base models (Qwen2.5) and do not pose significant additional misuse risks beyond standard instruction- tuned L.LMs.
Justification:
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All datasets and models used are cited in and carry permissive licenses. References are provided in the bibliography.
18

---

Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswi thcode. com comddatasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: This paper does not introduce any new dataset or pre-trained model checkpoint. It builds on existing models.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/model as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsoureing research involving human participants.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
19

---

Answer:
[NA]
Justification:
This work does not involve human subjects or IRB-relevant procedures.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [Yes]
Justification: This work proposes fine-tuning method for pretrained LLMs (Qwen2.5 series). LLMs are central the methodology and their clearly described in Sections Experiments.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to L.LM policy for what should or should not be described.
20

---

Appendix
This appendix provides full implementation and evaluation details to complement the main paper. We include training configurations, dataset statistics, benchmark setups, model partitioning strategies. and ablation results for various retrospective integration mechanisms.
A
Additional Experimental Details
A.1 Training Settings
Hardware. All experiments conducted on single NVIDIA H20 NVLink GPU (96 GB) attached to dual-socket server with 20 CPU cores (Intel Xcon R Platinum 8457C) and 200 GB RAM. We utilize torch native gradient accumulation emulate global batch size of 128 sequences.
Optimization hyper- paramt-arrmssers. Below detail the learning-rate schedule and other relevant knobs. Fine-tuning lasts for 2 epochs with a learning rate of Ir pre = 2 x 10-5 applied to all pretrained parameters. Newly introduced parameters (e.g.. cross-attention projection, FC adapters, layer-norm gates, etc.) are trained with higher learning rate Irnew 21 to accelerate adaptation while minimizing residual drift. We employ a cosine learning rate schedule with warm-up ratio of 10%. Training is performed using bfl6 precision.
Distillation loss.
Unless stated otherwise, the per-iteration objective is defined as
(t) ( (1) (1)) + hand
(8)
where use a 0.5. This balances soft-target alignment with ground-truth cross-entropy, matching the setting that yields the best validation perplexity in a 3x3 grid search over o E 0.5, 0.7].
A.2
Fine-tuning Data
Table 3 summarizes the five instruction-tuning corpora used for SCOUT. We follow the official splits and concatenate them after normalizing a conversational format compatible with Qwen2.5.
Table Statistics of the composite instruction corpus.
Dataset
Domain
Purpose
Alpaca GPT-4 45
general
single-turn instructions; broad world knowledge curated chain-of-though exemplars to expose stepwise patterns factual span extraction, grounding in Wikipedia context code synthesis completion to exercise symbolic reasoning multi-step numerical reasoning, explicit intermediate variables
Alpaca CoT 46
reasoning
WikiQA -7
open-domain QA
CodeAlpaca
48
code
MathInstruct [14]
maths
A.3 Model Design
Choice of Backbone and Teachers. We adopt the Qwen2.5 family because it provides a size- stratified yet homogeneous suite in both architecture and training recipe (0.5B, 1 1.5B, 3B, 7B..... enabling capacity- matthed progressive distillation while minimizing confounders across teacher capacities. Qwen2.5 also enjoys broad community adoption and strong reproducibility.
Partition strategy. Unless otherwise specified, we adopt the following configuration to as Case 2 in 5
Embedding first 1/2 layers Head block
Remaining 1/2 layers - Recursive block
Dutput projection + Tail block
21

---

This layout preserves deeper semantics in the head and allocates more capacity to recursive reasoning. as later confirmed our ablation results.
For ablation analysis (A (Appendix B.2). we compare this against the following alternative configura- tion-referred to Case 1:
Embedding + first 1/3 layers + Head
Middle 1/3 layers + Recursive
Final 1/3 layers FC - Tail
Cross-attention details. Each recursive block layer is augmented with a causal cross-attention sub-layer that queries the previous iteration's 's output hidden states. We reuse the same causal mask as self-attention, ensuring no access future tokens. We also insert a two-layer MLP projection before the cross-atis cross-attention module to map the current representation into the key/value space of the previous step, which helps stabilize training and improves convergence.
Vocabulary compatibility. The teacher Qwen2.5-7B model uses [V| 152,064 sub-word to- kens, whereas the student 0.5B model has [V] 151,936. The extra 128 symbols are all aoccialladded___ _xxx___>>__>>> placeholders. During distillation, (i) truncate the teacher logits to the student vocabulary size and (ii) renormalize before the softmax, so that the probability mass on absent IDs is redistributed proportionally.
A.4 Retrospective Reasoning Module
To support information flow across reasoning steps without disrupting the structure of pretrained L.Ms,, we introduce lightweight retrospective module that fuses each latent state (?-1)) with the initial state =0)) through a history integration function H(2(0) 2((-1)), producing the contextual input for the current step.
We compare six retrospective integration mechanisms five from prior work and one proposed by us-under the standard R-SFT setting (i.e., hard-label supervision at each recursion step). These include:
Init injection (Init) 77 Reuse the previous latent state directly: H(=(0), ((=((),=((-1)) 2(1-1),
Additive fusion (Add) 44, Combine prior and initial states by addition: H(2(0) 2(t-1)) =
20)) + (t-1).
Gate-based control (Gate) 4 Use a learnable gate to modulate the contribution (((-1)
via clement-wiss, control.
Concat-and-project (CatProj)
4 ((t-1) = MLP((2(0); (t (t-))).
Concatenate the states and apply a projection:
Modulated injection (ModInj) 4 Inject (t-1) into the current computation via AdaLN-
style modulation.
Cross-attention (XAttn, ours), Treat (1-1) external memory and apply cross-attention
using 20) as the query. This allows selective retrieval of prior reasoning while preserving the model's original attention flow.
A.5 Evaluation Benchmarks
We evaluate model performance using In-evalluation-harrees 50 with metrics summarised below for each dataset.
Commonsense QA. ARCeesysscceeeee (5T) and OpenBookQA 52 are multiple-choice datasets; their accuracy are reported. TruthfulAA (53) evaluated with the MCI-single accuracy variant.
Multi-step reasoning. GSM8K 54 measures mathematiially-groundee free-form answers via exact string match.
22

---

Reading comprehension & dialogue. Co is assessed with FI over answer spans. GLUE [57] is the nine-task hanarallaannugguudnnanaae convention and report the unweighted average of dev-set metrics (e.g., MNLI-m accuracy, QQP F1).
suite; follow the harness
Code generation. MBPP [58 is evaluated with the pass- @-1 criterion using harness-buil execution-baad bassd tests.
B More Experimental Results
This appendix extends Section 4 with additional analyses, including: progressive distillation (single- pass, no recursion), ablations, dataset-level breakdowns, and module comparisons to further validate approach. Unless otherwise noted, averages computed before rounding and then rounded to two decimals.
B.1 Progressive Distillation (Single-Pass, No Recursion)
Table 4: One-pass progressive distillation of 0.5B student from 1.5B-3B teachers).
Model
OB
GSM8K
MBPP
ARC-e
ARC-e
TF
CoQA
GLUE
Avg
Progressive
24.4
32.37
27.2
64.73
27.39
29.01
48.28
51.08
38.06
We distill a 0.5B student sequentially from 1.5B 3B - 7B teachers in single pass (no recursive reasoning). This simple baseline attains performance comparable to direct distillation from the 7B teacher. Without iterative reasoning, the student tends to retain primarily the last (largest) teacher's signal, whereas SCOUT improves with each recursion step via stepwise supervision and history- refinement.
B.2 Ablation on structural partitioning
Table 5: Ablation structural partitioning. Case 1 Embedding first 3. layers as head, middle
layers as recursive block, final layers output-projection (FC) as tail; Case 2 Embedding + first 1 layers as head, remaining layers as recursive block, output-projectio as tail. For each iteration, A reports the gain of Case 2 over the corresponding Case 1 (blue positive). All results are obtained under the R-SFT regime, using hard-label supervision each recursive step. OB OpenBookQA; ARC-e ARC-Easy; ARC-c ARC- AhC--Chaleeng;; TF- TruthfulQA.
Method
Iter. OB
GSM8K
MBPP ARC-e
ARC-c
TF
CoQA
GLUE
Avg
A
22.40
16.38
15.80
61.62
28.41
25.21
40.33
41.25
31.43
Init inj. - Case
21.20
16.07
7.40
59.93
29.27
24.24
35.73
44.06
29.74
21.00
8.49
0.60
56.78
28.58
24.24
24.92
46.97
26.45
24.80
30.55
26.20
65.74
31.23
25.83
48.20
44.86
37.18
+5.75
Init inj. - Case
23.40
26.31
26.20
64.18
31.23
27.78
30.40
43.33
34.10
+4.37
25.20
14.86
13.80
59.09
31.06
28.15
2.15
42.55
27.11
+0.66
21.40
1.97
0.20
57.32
27.13
22.15
8.35
43.88
22.80
Add. fusion Case 1
19.20
1.97
0.00
56.36
26.54
23.50
9.93
44.55
22.76
18.00
1.67
54.29
25.43
24.24
11.75
45.94
22.67
25.20
31.16
28.20
66.54
30.29
24.60
47.52
47.23
37.59
+14.79
Add. fusion Case 2
24.60
29.57
24.80
64.23
32.42
26.93
35.97
44.53
35.38
+12.63
23.60
13.80
60.06
33.36
27.78
14.22
42.56
20.20
1.36
0.00
55.09
27.56
23.99
7.33
48.58
23.01
Cross-attn Case
18.40
1.44
0.00
50.88
24.91
24.60
4.77
41.88
20.86
18.20
1.29
0.00
51.64
24.83
24.48
5.93
45.79
21.52
24.60
31.84
27.40
66.08
31.83
25.83
44.62
47.72
37.49
+14.48
Cross-attn Case 2
25.20
32.22
27.00
64.94
32.08
26.93
44.88
48.61
37.73
+16.87
24.60
32.22
27.00
64.31
32.34
26.81
45.40
47.28
37.50
+15.98
23

---

Analysis of Layer-Partition Ablations. To systematically examine the impact of structural par- titioning, we compare two layout strategies (Case 1 vs. Case 2) under three different retrospective modules-Init injection, Additive fusion, and Cross-attention (ours)-withy the R-SFT training regime. That is, each recursive step uses standard supervised learning with ground-truth labels, without progressive distillation. The goal is to isolate the effect of partition strategy while holding training and integration conditions fixed.
Table S presents the results. We interpret them based on two empirical facts: (i) pretrained layer representations are nor uniformly informative early layers encode low-level lexical cues, middle layers gradually disentangle semantics, and late layers specialize in reasoning and decoding: (ii) recursive refinement can only add information that the recursive block itself is capable of modeling.
Larger heads expose richer priors. Case promotes half of the backbone to the head block, giving the very first forward pass direct access to deeper semantic features that are otherwise reached only after recursion in Case Consequently, iteration already outperforms the entire Case-1 trajectory by +14.5-14.8 points.
Recursive depth has diminishing returns when the middle block is shallow. In Case 1 the recursive block consists of merely $ of the layers; its latent states (z(9) carry limited new evidence, so iteration 3 accrues noise faster than signal. Expanding the recursive block to 2 (Case 2) stabilizes iteration 2, but iteration 3 still Brreeaus-uuggettnng that once high-capacity semantics are already present in 2 further passes yield marginal benefit.
Cross-attention maximises structural capacity. The XAttn integration module exploits the richer head features without disrupting the pretrained attention flow. By treating the previous latent state as external memory, it filters useful cues while discarding spurious ones, giving the largest average gain (+16.9 at 2=2) under the same parameter budget.
Why "half-half-0" offers stronger performance. Pretraining concentrates reasoning ability toward later layers; shifting those layers to the head (1/2) exposes them to the recursive block early. The recursive block-now also 1/2-veceives stronger inputs and therefore produces more meaningful refinements, which the lightweight tail (FC) can decode with minimal overhead. In essence, Case 2 turns recursion into iterative polishing of already high-quality embeddings, whereas Case I must first build that quality through shallow recursion, incurring greater noise amplification.
Limitations and future work. Owing to computational constraints, we did not ex- plore finer-grained alternative layer-allocation strategies. Systematically varying the head/recursive/tall ggiitespeccially with dynamic or data-adaptive partitions remains an interesting direction for future research.
Overall, these findings substantiate our design choice: assigning more pretrained depth the head and recursive blocks, coupled with cross-step attention, yields the most faithful instantiation of Flow CoT-achieving faster convergence and higher final accuracy without extra parameters or pretraining.
B.3 Dataset-Level Breakdown of Retrospective Modules
This section complements the integration strategies described in Appendix A.4 by reporting dataset- level performance across recursion depths (L 3)) All results are obtained under the R-SFT training regime [7], where each iteration uses standard hard-label supervision without progressive distillation. T detailed benchmark scores for each retrospective module. All metrics are reported as accuracy (%), except for CoQA (FI) and GLUE (average).
Per-dataset observations.
OpenBookQA. XAttn obtains the highest score (25.2) at t=2, narrowly surpassing Additive fusion (25.2 at (=1) while maintaining stability at deeper steps.
GSM8K (math). XAttn leads on every iteration, peaking at 32.22. CatProj collapses after the first step, revealing difficulty in propagating structured numeric information.
MBPP (code). Additive fusion yields the single best result (28.2) but deteriorates sharply afterwards; XAttn retains > 27 across all iterations, indicating more robust generalisation.
24

---

Table 6: Detailed benchmark results for different retrospective modules.
Module
Iter.
OB
GSM8K
MBPP
ARC-e
ARC-
TF
CoQA
GLUE
Avg
24.8
30.55
26.20
65.74
31.23
25.83
48.20
44.86
37.18
Init
2
23.4
26.31
26.20
64.18
31.23
27.78
30.40
43,33
34,10
25.2
14.86
13,80
59.09
31.06
28.15
2.15
42.55
27.11
1
25.2
31,16
28.20
66.54
30.29
24.60
47.52
47.23
37.59
Add
2
24.6
29.57
24.80
64.23
32.42
26.93
35.97
44.53
35.38
33.36
27.78
29.09
1
23.0
32.37
24.00
63.93
30.38
23.75
45 15.66
47.85
36.36
CatPmij
53.91
27.67
46.89
27.75
20.0
7.96
0.00
49.49
26.11
24.48
19.83
43.69
23.95
1
24.0
27.90
65.61
29.61
25.58
39.42
51 11.08
33.85
Gate
2
17.0
22.52
0.40
56.86
26.45
24.85
25.33
50.08
27.94
3
20.0
15.69
1.20
60.14
27.39
25.83
24.25
42.18
27.09
21.2
28.81
23.00
28.58
25.
40.53
45.79
34.55
ModInj
2
20.8
24.03
16.20
61.91
29.61
25,09
36.78
42.89
32.16
3
20.6
16.07
8.60
59.09
28.16
25.34
29:43
44.57
28.98
1
24.6
31.84
27.40
66.08
31.83
25.83
44.62
47.72
37.49
XAttn (ours)
2
25.2
32.22
27.00
64.94
32.08
26.93
44.88
48.61
37.73
24.6
32.22
27.00
64.31
32.34
26.81
45.40
47.28
37.50
ARC-easy /
ARC-hallengeeee XAttn delivers the top or second-best accuracy on both ARC splits, confirming that cross-step attention is especially beneficial for commonsense and scientific QA.
TruthfulQA. XAttn reaches 26.93, improving over Gate/ModInj by 1.4 pts, which suggests better resistance to hallucination when knowledge is revisited recursively.
CoQA. Additive fusion attains the absolute maximum (47.52), yet performance drops by 33 pts at f=3. XAttn grows steadily to 45,40, demonstrating superior long-horizon dialogue coherence.
GLUE, Gate-based control wins the first iteration (51.08) thanks to its explicit gating, but XAttn overtakes it deeper steps (47.28 vs. 42.18) while keeping higher overall average.
Take-aways. Across eight diverse benchmarks, XAttn is the only mechanism that (i) never collapses with depth and (ii) ranks first second all tasks. Among the integration strategies introduced earlier, cross-attention clearly stands out. The results reinforce our main claim: query- driven rross-atentionn provides the right inductive bias for reusing intermediate cognition, whereas simpler fusions either over-mix (Add) or under-utilize (CatProj) the latent history. Future work could combine the strong first-step performance of Gate with XAttn's 's depth stability-for instance, by conditioning the cross-attention weights on learnalee gating signal.
C Impact Statement
Our work introduces Flow Chain-of-Thougee (Flow CoT) and its practical instantiation SCOUT, enabling pretrained language models to achieve deeper, interpretable reasoning with fine-tuning only. Because SCOUT reuses existing weights and adds only lightweight cross-attention adapters, it lowers the computational and energy cost typically associated with multi-step reasoning systems that require extensive pre- or re-training. This can democratise access to stronger LLM reasoning in research, education, and low-resource settings while reducing the environmental footprint of model development.
As with any technique that amplifies model capability, Flow CoT may inadvertently strengthen harmful content generation, persuasive misinformation, biased reasoning chains. Moreover, progressive distillation leverages larger teacher models whose biases may propagate to students. We encourage follow-up work to pair Flow CoT with robust safety filters, bias audits, and energy-usage tracking. In addition, dynamic early-exit policies could further reduce inference cost when high- confidence answers are reached, aligning computational efficiency with responsible Al principles.
25


Reviews:
[{"review_id": "aU8KObHEcx", "reviewer": "Reviewer_sQsh", "summary": "The paper proposes Flow CoT and SCOUT, a novel reasoning and lightweight training method.  They propose having recursive reasoning stages within an LLM, where each stage learns from the last stage and improves upon it. Experiments show gains over the prior methods. Moreover, they show qualitatively that each reasoning stage learns how to correctly answer progressively difficult queries.", "questions": "Q1: Can you please provide more information on how exactly are the teachers trained, how are they matched for each recursive block?\nQ2: How much resources are required", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strenthgs:\n- Experiments are performed across different reasoning types like commonsense, mathematical, reading comprenhsion, and code generation. \n- Results demonstrate improved performancce compared to standard  SFT/distillation and recursive hard label supervision.\n\nWeaknesses:\n- Writing could be improved, it is not clear when the input is in latent form and when it is in discrete embedding form\n- it is not clear how are teachers trained or matched for each recursive block.\n- The paper does not talk about the training or inference resources used\n- The paper would benefit from an analysis of accuracy on the entire datasets using each recursive block.", "was_revised": false, "final_justification": "", "rebuttal": "**Thank you for recognising the value of our work. Below we detail the additions and clarifications made in the revision, structured around the four points you raised.**\n\n---\n\n### **1. Latent Representation vs. Discrete Embedding**\n\nIn a standard LLM, the model’s internal hidden states are continuous latent representations that no longer correspond directly to any discrete token in the vocabulary, but instead serve as rich features for deep reasoning.[2]  \n\nIn SCOUT, the workflow is as follows:\n\n1. **Input:** discrete tokens (human-readable)  \n2. **Embedding layer:** look up each token’s embedding vector to produce the initial continuous latent representation.  \n3. **Head block & Recursive blocks:**  \n   - A **head block** processes latent representation $z^{(0)}$.  \n   - Followed by T **recursive blocks**, each transforming the latent state $z^{(t-1)}$ to $z^{(t)}$.  \n4. **Tail block:** processes the final latent state $z^{(T)}$.  \n5. **Output layer (fully connected):** maps $z^{(T)}$ back to discrete tokens (human-readable).\n\n **Note:** All intermediate representations ($z^{(0)}$, $z^{(1)}$, $...$, $z^{(T)}$) are continuous hidden states that flow only within the model. Only the initial input tokens and the final output tokens are in discrete form. This entire format transformation mirrors the process described in [1], and the use of continuous latent representation to enhance reasoning is motivated by their ability to capture richer information than discrete representations, as noted in [2].\n\nWe will add these clarifications in the main text to improve our writing and help readers clearly distinguish between “latent representation” and “discrete embedding.”\n\n\n### **2. Teacher Model Selection and Matching Logic**\n\nWe provided a brief discussion on the teacher model selection and matching in Sections 3.3 and 4.1 of the main text. Below is a more detailed explanation:\n\n**Matching Principle**  \nAs the number of iterations \\(t\\) increases, the model’s latent representation capability is enhanced [1][3], so we align each recursion step with a larger-scale/higher-performance teacher model.\n\n**Teacher Configuration**  \nTo ensure reproducibility, we did not train multiple separate teacher models. Instead, we selected models from the same Qwen2.5 series, which span a broad range of sizes (0.5B–72B) and share a unified vocabulary and knowledge distribution without extra training. We chose the 1.5B, 3B, and 7B variants as our teacher models, applied as follows:\n\n| Teacher Model   | Parameters | Iteration Step \\(t\\) | Avg Score ↑ | Description                                                                                         |\n|-----------------|------------|----------------------|-------------|-----------------------------------------------------------------------------------------------------|\n| Qwen2.5-1.5B    | 1.5B      | 1                    | 50.84       | Guides the 1st recursion, providing directional guidance without over-constraining.                 |\n| Qwen2.5-3B      | 3B        | 2                    | 55.60       | Guides the 2nd recursion, providing directional guidance without over-constraining.                 |\n| Qwen2.5-7B      | 7B        | 3                    | 60.90       | Guides the 3rd recursion (final output), providing directional guidance without over-constraining. |\n\n**Alternative Strategy When Few Models Are Available**  \nIf only a single large model is accessible, one can first distill multiple student models of various sizes (e.g., 0.5B, 1B, etc.) and then assign them as teachers for recursion steps 1, 2, and 3 in order from weakest to strongest, preserving the matching principle.\n\nWe will expand on the teacher selection and matching logic in more detail in the main text.\n\n---\n\n### **3. Compute Resources**\n\nIn Appendix A.1 (Training Settings), we provide a comprehensive description of our setup, including:\n\n- **Hardware:** We use a single NVIDIA H20 NVLink GPU (96 GB), a dual-socket Intel® Xeon® Platinum 8457C CPU (20 cores), and 200 GB of RAM. One GPU is sufficient to train each model.  \n- **Training configuration:** We employ gradient accumulation to achieve an effective batch size of 128. Training is performed in bfloat16 (bf16) precision.  \n- **Optimization hyperparameters:** Learning rates, optimizer settings, and additional details are listed in Appendix A.1.  \n\n\n\nWe will add a cross-reference in the main text’s experimental settings section to direct readers to Appendix A.1.  \n\n\n---\n### **4. Dataset-wide Performance Analysis Across Recursive Blocks**\n\nIn Table 1 of the main text, the second column “Iter” corresponds to the average performance after each recursive block:\n\n| Method | Iter | Avg Score |\n| ------ | ---- | ------------ |\n| SCOUT  | 1    | 37.44%       |\n|        | 2    | 38.26%       |\n|        | 3    | 39.03%       |\n\n- The results demonstrate a monotonic increase in performance as the number of iterations grows, validating the effectiveness of progressive distillation. We have discussed this trend in detail in Section 4.2.  \n- To present this more clearly, we will add an annotation in Table 1 clarifying that “Iter represents the number of recursive blocks.”\n\n\n\n\n---\n\nThank you once again for your valuable feedback. \n\n---\n\n>#### References\n>[1] J. Geiping *et al.* “Scaling up test-time compute with latent reasoning: A recurrent depth approach.” *arXiv preprint arXiv:2502.05171*, 2025.\\\n>[2] S. Hao *et al.* “Training large language models to reason in a continuous latent space.” *arXiv preprint arXiv:2412.06769*, 2024.\\\n>[3] N. Saunshi *et al.* “Reasoning with latent thoughts: On the power of looped transformers.” *arXiv preprint arXiv:2502.17416*, 2025."}, {"review_id": "F5qWEqlw53", "reviewer": "Reviewer_rHmT", "summary": "This paper points out a problem in existing recursive reasoning models: recursive reasoning should not be viewed as black-box repetition, but as a structured cognitive evolution. It introduces Flow CoT, a reasoning paradigm that envisions recursive inference in LLMs as a progressive trajectory of cognitive states, moving beyond conventional chain-of-thought, which depends on explicit supervision for intermediate steps. To operationalize this paradigm, the authors propose SCOUT, a fine-tuning framework that combines progressive distillation with a cross-attention-based retrospective reasoning module.  The approach is extensively evaluated on 8 benchmarks (covering commonsense QA, multi-step math, code, etc.), demonstrating improved accuracy and explanation quality versus several recent baselines.", "questions": "The concept of Flow CoT appears to share similarities with diffusion models. Is there a deeper connection between language diffusion models and your work? Could you elaborate on this relationship?\n\nIn Line 131, you state, \"This misalignment can distort latent states and harm downstream refinement.\" This is a strong claim. Do you have any citations or concrete analyses to support this strong claim about existing recursive methods?\n\nSome aspects of the method are still unclear. For instance, how are the head block and tail block explicitly defined when working with LLMs? Could you provide more details about the training and inference processes?\n\nCould you explain why CoQA performance decreases as the number of reasoning steps increases?\n\nBenchmarks such as GLUE, CoQA, and MBPP are generally not considered to require extensive reasoning. As a result, they may not be suitable for evaluating a model’s multi-step reasoning abilities. Can you explain more about the model behavior of these datasets and why they can be benefited from SCOUT?", "limitations": "Yes.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n(1) Novel Concept: The paper introduces the concept of Progressive Distillation and leverages latent variables to enhance performance on reasoning tasks. This is a valuable contribution to the reasoning research community.\n\n(2) Comprehensive Evaluation: The proposed SCOUT method is evaluated on eight benchmarks covering various types of tasks. The paper also explores different training settings and provides insights into the key aspects of the method.\n\n\nWeaknesses:\n(1) Scalability: The method depends on multiple teacher models with different abilities, raising concerns about scalability—especially when applying it to larger models. For very strong models, it may be difficult to find appropriately leveled teacher models for supervision. Additionally, experiments only go up to a 7B teacher model, which limits confidence in the method’s effectiveness at larger scales (e.g., 72B). The approach should be validated on a wider range of model sizes.\n\n(2)Baseline Comparison: An additional baseline should be included where a single model is distilled progressively using teachers of sizes 1.5B to 7B in sequence, with inference performed only once (to distinguish from R-Distill-EQ). Also, the fair comparison for SCOUT should be with CoT-style SFT, not just pure SFT as currently reported. Comparisons with latent CoT-based methods would also strengthen the evaluation. At least the differnet should be discussed thoroughly.\n\n(3) Lack of Efficiency Analysis: Although the method could offer efficiency advantages over traditional explicit reasoning approaches like R1, there is insufficient analysis of wall-clock time and inference speed. In particular, Flow CoT should theoretically be faster than decoding a model three times in a standard CoT setup, but this is not discussed.", "was_revised": true, "final_justification": "The scaling experiments show positive results, and it is convincing to me.", "rebuttal": "**Thank you for your detailed review. We address your concerns as follows:**\n\n---\n\n### **1. Scalability**  \n#### **1.1 Large-Scale Evaluation**  \nTo address the suggestion for larger‐scale evaluation, we replaced the original 7B teacher with 14B and 32B model. The results are as follows:\n\n|Teacher Size|Iteration|OB|GSM8K|MBPP|ARC-e|ARC-c|TF|CoQA|GLUE|**Avg**|\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|**14B** |1|22.6|33.21|27.2|65.49|28.67|26.68|49.63|49.27|37.84|\n||2|22.4|34.27|28.4|65.49|29.18|26.81|47.33|52.68|38.32|\n||3|24.0|34.39|28.4|66.54|30.29|27.42|47.80|54.41|39.16|\n|**32B**|1|23.4|33.74|27.8|66.62| 30.29 | 27.17| 48.32|52.22|38.70|\n||2|23.2|34.27|28.0|66.88|30.63|27.17|47.05|53.58|38.85|\n||3|24.0|34.72|29.0|66.96|30.72|27.66|47.12|54.05|39.28|\n\n We observe that even with a 32B teacher, SCOUT’s average accuracy steadily improves over iterations (38.70→38.85→39.28), demonstrating both the effectiveness and scalability of our method at larger scales.\n \n#### **1.2 Teacher Models** \n\nOur central contribution is the Flow CoT paradigm—treating recursive reasoning as a progressive trajectory of latent states. SCOUT’s multi-teacher design is one concrete instantiation, not a prerequisite of Flow CoT, and our experimental results already demonstrate the effectiveness and scalability of both SCOUT and Flow CoT. \n\nWe have also considered the point you raised.  Consequently, in the Limitations and Future Work section we explicitly state that the present study offers an  proof-of-concept validation of Flow CoT, and we outline alternative directions—such as *reinforcement-learning–based scoring without external teachers*—that can provide per-iteration difficulty rewards and represent promising avenues for future research.\n \n ---\n \n ### **2. Baseline Comparison**  \n Because SCOUT is designed for iterative models, our main experiments primarily compare against other recursive approaches to demonstrate the effectiveness of our method.  The additional baselines you suggested are conventional single-pass fine-tuning techniques that do not involve model recursion; nevertheless, we have included them for completeness.\n\n#### **2.1 Progressive Distillation Baseline**  \nWe added a progressive distillation baseline: a 0.5B student distilled sequentially from 1.5B→3B→7B teachers, yielding:\n\n|Model|OB|GSM8K|MBPP|ARC-e|ARC-c|TF|CoQA|GLUE|**Avg**|\n|-|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|**Progressive**|24.4|32.37|27.2|64.73|27.39|29.01|48.28|51.08|38.06|\n\nWe observe that simple, one-pass progressive distillation achieves a performance comparable to directly distilling with the 7B teacher. Without iterative reasoning, the student model tends to “forget” earlier-teacher knowledge and retain primarily the largest teacher’s knowledge—contrasting with SCOUT’s iterative improvement of reasoning capabilities.\n\n#### **2.2 Comparison with CoT-Style SFT**  \nTo compare against a chain-of-thought supervised fine-tuning (CoT-SFT) baseline, we evaluated both CoT-SFT and SCOUT on the AQUA-RAT math dataset:\n\n|Method|AQUA-RAT|\n|-|:-:|\n|**CoT-SFT**| 33.07    |\n|**SCOUT (Iter 1)**|32.28|\n|**SCOUT (Iter 2)**|33.46|\n|**SCOUT (Iter 3)**|**34.64**|\n\nSCOUT demonstrates consistent iterative gains, ultimately surpassing the CoT-SFT baseline by iteration three.\n\n#### **2.3 Comparison with Latent CoT Methods**  \n- **Latent CoT (e.g., Coconut[1])** performs **horizontal recursion**: it feeds the last‐layer hidden state back into the model for the next step, requiring specialized data pipelines.  \n- **SCOUT / Flow CoT** performs **vertical recursion**, looping hidden states between internal layers for multi-stage reasoning without emitting intermediate tokens, is fully compatible with standard Hugging Face Trainer workflows, and requires no special data preprocessing  modifications.\n\n---\n\n### **3. Efficiency Analysis**  \n\nTo evaluate inference speed, we measured the wall-clock latency for performing three reasoning iterations.\n\n|Method|Latency (s)|\n|-|:-:|\n|**SCOUT**|0.0327|\n|**CoT-SFT**|0.0448|\n\n- **CoT-SFT:** Three full-model decodings (one forward pass per reasoning step), producing three chain-of-thought tokens.  \n- **SCOUT:** Only loops the recursive block three times—without re-running the entire model—hence the lower latency.\n\nSCOUT reduces latency by 27% compared to CoT-SFT (0.0327 s vs. 0.0448 s) for three-step reasoning, demonstrating a clear efficiency advantage.\n\n---\n\n### **4. Relationship to Diffusion Models**  \n\nAlthough both Flow CoT and latent diffusion iteratively refine an internal state, they differ in three key ways:\n\n1. **No Noise Injection**  \n  - **Latent diffusion** (e.g. DDPMs) injects noise into the latent representation during training and then learns to denoise over multiple steps.  \n  - **Flow CoT** relies entirely on the model’s own predictive capabilities—there is no explicit noise schedule or denoising objective.\n\n2. **Autoregressive vs. Masked Generation**  \n- **Language diffusion** (e.g. LLADA[2]) generates a full token sequence, then in each pass masks out low-confidence tokens and re-predicts them in parallel—explicitly refining token distributions in a “one-shot” update loop.  \n- **SCOUT (Flow CoT)** remains fully autoregressive and latent-only: it never re-emits or re-masks tokens during reasoning but instead loops and refines hidden states across layers, then generates exactly one new token per cycle. \n\n---\n\n### **5. Statement Discussion**  \n\nWe first highlight two key ideas:\n\n1. **Recursive Model Strengthening**: Each pass refines its latent state, so intermediate representations become more accurate and powerful over iterations [3][4][5].  \n2. **Teacher–Student Capacity Matching**: Effective distillation requires teacher sizes aligned with the student’s capacity; oversized teachers early on cause a representational mismatch [6].  \n\nWhen early iterations are supervised by an overly strong teacher, the student’s limited capacity is pushed toward goals it can’t yet model, corrupting its latent trajectory and impairing later loops. For example, R-Distill-EQ sees accuracy fall 38.44→37.17→37.36, whereas SCOUT’s staged teachers (0.5B→1.5B→7B) steadily improve accuracy: 37.44→38.26→39.03. Moreover, relaxing the first-iteration loss weight (Table 3 in [7]) similarly boosts downstream performance, further validating our progressive-distillation approach.  \n\n---\n\n### **6. Algorithm Details**  \n\nThe definitions of the head block and tail block are similar to those in [3]. In a typical LLM architecture, we define the first X layers as the head block, the last Y layers as the tail block, and the remaining layers as the recursive block. The specific values of X and Y can either be set manually or determined through ablation experiments to find the most effective configuration. Further details and results from these experiments can be found in Appendix B.1.\n\nSince our framework is compatible with standard frameworks like Hugging Face Trainer, we use the LlamaFactory framework for training, and the lm-evaluation-harness framework for inference. More details about the training and inference processes are provided in Appendix A.\n\n---\n\n### **7.  CoQA Performance**\n\nWe attribute the slight F1 drop over successive reasoning steps to the CoQA evaluation metric rather than a weakening of SCOUT’s reasoning:\n\n- **Metric sensitivity:** CoQA’s F1 score treats each answer as a bag of characters, so any extra tokens reduce the overlap count—even if the answer remains correct.\n- **Longer outputs:** As shown in Figure 5, iterations 2 and 3 produce answers that are on average 15–20 % longer than iteration 1, introducing more non-overlapping characters and thus lowering the computed F1.\n- **Semantic fidelity:** Manual review shows that later-step answers remain semantically equivalent to the first-step output and often include extra reasoning details, enhancing interpretability.\n\n---\n\n### **8. Model Behavior Explanation**\n\nAlthough GLUE, CoQA, and MBPP are not primarily designed as deep, multi-step reasoning benchmarks—GLUE and CoQA emphasize memory and shallow inference, while MBPP targets code generation—they nonetheless contain non-trivial reasoning components; we therefore include them as relevant, widely used evaluations, and SCOUT still delivers consistent gains by leveraging latent-state recursion to:\n\n- **Memory consolidation:** Prior work shows that stacking LLM layers steadily boosts memory performance—likely because key facts and context get split across loops and reinforced each pass [4]. SCOUT mirrors this effect, improving results on GLUE subsets .  \n- **Progressive reasoning:**  Each latent-state loop in SCOUT refines internal representations toward increasingly complex subgoals, boosting final answer quality.\n\nOur results in Table 1, spanning eight benchmarks, show that latent-state recursion consistently improves performance, demonstrating that distributing both memory reinforcement and reasoning refinement across multiple loops benefits a wide spectrum of NLP tasks.\n\n---\nWe hope this clarifies your concerns.\n\n---\n\n>#### References\n>1. Hao, S. *et al.* “Training Large Language Models to Reason in a Continuous Latent Space.” *arXiv preprint arXiv:2412.06769*, 2024.\n>2. Nie, S. *et al.* “Large Language Diffusion Models.” *arXiv preprint arXiv:2502.09992*, 2025. \n>3. Geiping, J. *et al.* “Scaling Up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach.” *arXiv preprint arXiv:2502.05171*, 2025. \n>4. Saunshi, N. *et al.* “Reasoning with Latent Thoughts: On the Power of Looped Transformers.” *arXiv preprint arXiv:2502.17416*, 2025. \n>5. Saunshi, N. *et al.* “On the Inductive Bias of Stacking Towards Improving Reasoning.” *arXiv preprint arXiv:2409.19044*, 2024.\n>6. Busbridge, D. *et al.* “Distillation Scaling Laws.” *arXiv preprint arXiv:2502.08606*, 2025.\n>7. Bae, S. *et al.* “Relaxed Recursive Transformers: Effective Parameter Sharing with Layer-Wise LoRA.” *arXiv preprint arXiv:2410.20672*, 2024."}, {"review_id": "DQwWdItiDX", "reviewer": "Reviewer_Z32A", "summary": "The paper proposes a flow CoT for recursive reasoning paradigm with depth-aware trajectory of latent cognitive states.  The authors develop a light-weight fine-tuning framework with a step wise optimization technique using teachers.  The introduction of a progressive distillation strategy enables the reasoning to be supervised by a teacher. The paper also presents a non-intrusive retrospective module using cross-attention that ensures the original reasoning flow. The authors empirically show the improvements in the performance on multiple benchmarks.", "questions": "1. The authors should clearly mention the reason for the model preference.", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The major strength of the paper lies in its capabilities of multi-step reasoning without requiring explicit annotations at each step. The paper attempts to train the model through progressive distillation using teachers with matching capacity unlike distillation requiring strong uniform supervision. The proposed retrospective module enhances depth-aware reasoning capabilities in pre-trained LLMs. The authors present a comparative study of the performance of the proposed model with different baselines. The token level probabilities heatmap shows a clear progressive trajectory towards the correct answer using the approach.\n\nIt is not clear why Qwen 2.5 model series is chosen for the task. Similarly if Qwen is used, the reason behind not using an update Qwen model is also missing.", "was_revised": true, "final_justification": "Yes the authors have addressed all the questions raised by me and I have given the scores accordingly.", "rebuttal": "We thank you for the careful evaluation. Below we clarify our choice of the Qwen 2.5 series:\n\n1. **Homogeneous Multi-Scale Model Family**  \n   Qwen 2.5 provides a spectrum of models—from 0.5B, 1.5B, 3B, 7B up to 14B, 32B, and 72B—all trained with the same recipe. This architectural and training-data homogeneity allows us to perform progressive distillation on a single student model using increasingly stronger teachers, without introducing confounding variables.\n\n2. **Community Adoption & Experimental Timeline**  \n   Qwen 2.5 has been widely adopted in recent large-model research (e.g., [1], [2], [3]), demonstrating strong community support and reproducibility.  Although Qwen 3.0 was released at the end of April 2025, our experimental pipeline had already been finalized and the core experiments were nearly complete by then, making it infeasible to re-run the full study before the submission deadline.\n\n We will include this rationale in the final manuscript. We hope this explanation clarifies our choice of the Qwen 2.5 series and appreciate your valuable feedback.\n\n---\n\n>### References\n>1. He, Yichen *et al.* “PaSa: An LLM Agent for Comprehensive Academic Paper Search.” *Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)*, 2025.\n>2. Hong, Jiwoo *et al.* “On the Robustness of Reward Models for Language Model Alignment.” *arXiv preprint arXiv:2505.07271*, 2025.\n>3. Yang, Shiming *et al.* “Demystifying Long Chain-of-Thought Reasoning.” *Proceedings of the Forty-second International Conference on Machine Learning*, 2025."}]

Author rebuttal:

Meta-review: This paper proposed a framework called SCOUT for reasoning model fine-tuning where a progressive distillation method and a retrospective reasoning module are adopted. In the progressive distillation, the authors use a series of QWen models with different model sizes as teacher models to provide supervise signals for the intermediate steps in a recursive process. At the same time, the retrospective module consists of both self-attention and cross-attention components to incorporate the original latent information and the intermediate reasoning results. 

Reviewers agreed that the proposed methodology is novel and the experimental evaluation is well designed. 
However, some of them also pointed out the paper is not well written and need to be improved. Using a series of QWen models as teachers to train a small model to improve its reasoning capability. It seems that such a method is not scalable in practice. In addition, this paper only uses Qwen models in the experiments, and the conclusion might not be able to general applicable to other model families because there are papers pointing out that some strange behavior of Qwen models.
Decision: Accept (poster)