Topology of Reasoning: Understanding Large Reasoning Models through Reasoning Graph Properties
Authors: Gouki Minegishi
Hiroki Furuta
Takeshi Kojima
Yusuke Iwasawa
Yutaka Matsuo
Venue: NeurIPS 2025 poster ()
Keywords: Interpretability
Reasoning
LLM
Primary area: deep_learning
TL;DR: This study reveals that reasoning models achieve superior performance by forming distinct reasoning graphs, characterized by greater cyclicity, larger diameters, and strong small-world properties.

Abstract: Recent large-scale reasoning models have achieved state-of-the-art performance on challenging mathematical benchmarks, yet the internal mechanisms underlying their success remain poorly understood. In this work, we introduce the notion of a reasoning graph, extracted by clustering hidden‐state representations at each reasoning step, and systematically analyze three key graph-theoretic properties: cyclicity, diameter, and small-world index, across multiple tasks (GSM8K, MATH500, AIME~2024). Our findings reveal that distilled reasoning models (e.g., DeepSeek-R1-Distill-Qwen-32B) exhibit significantly more recurrent cycles (about 5 per sample), substantially larger graph diameters, and pronounced small-world characteristics (about 6x) compared to their base counterparts. 
Notably, these structural advantages grow with task difficulty and model capacity, with cycle detection peaking at the 14B scale and exploration diameter maximized in the 32B variant, correlating positively with accuracy.  
Furthermore, we show that supervised fine-tuning on an improved dataset systematically expands reasoning graph diameters in tandem with performance gains, offering concrete guidelines for dataset design aimed at boosting reasoning capabilities. By bridging theoretical insights into reasoning graph structures with practical recommendations for data construction, our work advances both the interpretability and the efficacy of large reasoning models.

Topology of Reasoning: Understanding Large Reasoning Models through Reasoning Graph Properties
Gouki Minegishi' Hiroki Furuta Takeshi Kojima' Yusuke Iwasawa' Yutaka Matsuo
'he University of Tokyo, Google DeepMind
{minegishi, furuta, furrut,, t. kojoimm, iwasawa, (weasuooooebbbbbb........ ttt u.u-okkyo.ac.
Abstract
Recent large-scale reasoning models have achieved state-of-the-art performance challenging mathematical benchmarks, yet the internal mechanisms underlying their success remain poorly understood. In this work, we introduce the notion of a reasoning graph, extracted by clustering hidden-state representations each rea- soning step, and systematically analyze three key graph-theoretic properties: cyclic- ity, diameter, and small-world index, across multiple tasks (GSM8K, MATH500, AIME 2024). Our findings reveal that distilled reasoning models (e.g., DeepSeek- RI-Distill-Qwen Qwen-32B) exhibit significantly more recurrent cycles (about per sample), substantially larger graph diameters, and pronounced small-world charac- teristies (about 6x) compared their base counterparts. Notably, these structural ad- vantages grow with task difficulty and model capacity, with cycle detection peaking at the 14B scale and exploration diameter maximized the 32B variant, correlating positively with accuracy. Furthermore, we show that supervised fine-tuning on an improved dataset systematically expands reasoning graph diameters in tandem with performance gains, offering concrete guidelines for dataset design aimed at boost- ing reasoning capabilities. By bridging theoretical insights into reasoning graph structures with practical recommendations for data construction, both the interpretability and the efficacy of large reasoning models. Implementation available here: https: :tap:::///ttiie //github. com/ gouki510/Topoolo Tooplloooooloooooooooooooo Bacoooyyoooo_ooooooee RReasoning
work advances
Introduction
Recent advances in large reasoning models, such OpenAl-ol families [47], extended thinking mode Gemini [29], Claude [2], Grok [66], and DeepSeek-RI [10], have achieved striking performance gains pushing the frontier across expert-level coding, competitive math, and PhD-level science questions. These recent reasoning models are characterized with to think and reason for longer before responding. Inspired by the breakthroughs in reasoning capabilities, novel methods to imitate reasoning abilities with smaller models have been developed such as supervised fine-tuning (SFT) techniques [44, 69] and distillation [10]. However, despite these notable successes, the internal mechanisms enabling their remarkable reasoning capabilities remain unclear, particularly in comparison traditional, non-reasoning models.
To understand the key factors behind the success of recent reasoning models, we introduce the concept of reasoning graph (Figure I). In mathematical tasks, for example, a reasoning graph can be defined as the path through simple computational states (e.g., addition or subtraction) toward the final answer, where each state corresponds to node in the graph. Prior to recent breakthroughs in reasoning models, some previous works [48, 63) have empirically and theoretically shown that L.LMs
Work done as an advisory role only.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Reasoning Graph
Question
Joy can read 8 pages of a book in 20 minutes. How many hours will take her to read 120 pages?
Q
11
Q
V1
122
1/2
Generated Answer
Base Model
Reasoning Model
1/5
V/5
in bear thare are asts rninutes.
3/3
3/3
1n:1 heur there and aass of eiautes.
pages 200 reluuddd pagenstes 120 pages pageotheent
174
1/4
120 peggehhhiiettceeee hww.ett-ch0..okkoee
pages 20 miovit pagasrmiiuss
260 silett 60 enlost ood
16
1/6
varty
puger
A
The assee' Ebouas
pages propo//eeatiio 320 viliutes -0 aduutacchorr hours.
Incorrect
Correct!
Base Model
Reasoning Model
Figure 1: Illustration of the concept of reasoning graphs, comparing base models and large reasoning models. Nodes represent simple computational states (c.g., calculation steps shown the lcft), with paths leading to the final answer constituting the reasoning graph. We analyze graph-theoretic properties of reasoning graphs, including cyelic structures, diameter, and small-world characteristics. Examining these structural distinctions enables us to better understand and recent performance improvements in challenging mathematical tasks.
employing chain-of-thought prompting achieve higher accuracy by traversing this graph step-by-step. In this work, analyze the reasoning graphs of large reasoning models from teph-thco perspective, aiming to identify unique structural properties that contribute recent breakthroughs in reasoning performance.
First, we extract reasoning graph nodes by clustering hidden states of LLMs using kmeans during reasoning tasks. Then, for each reasoning task sample, we construct a reasoning graph by connecting the nodes visited by the model during inference and analyze its properties. Visualizing the reasoning graph with t-SNE clearly demonstrates that reasoning graphs from large reasoning models include cycles and have a broader exploration range compared to base models. Quantitatively assessing these cyclic properties of reasoning graph confirms that large reasoning models exhibit significantly more cycles than base models. Furthermore, the proportion of reasoning graphs containing cycles increases progressively with task difficulty, observed across datasets GSM8K [7] (casier), MATH500 [23] (intermediate). and AIME 2024 [11] (more challenging). To quantitatively examine whether large reasoning models explore a broader range of nodes during inference, compared the diameters of their reasoning graphs. We observed notably larger diameters for large reasoning models, suggesting that they explore a wider variety of reasoning states, potentially enabling more sophisticated inference strategies. To gain deeper insights into the structural underpinnings of reasoning ability, we assess the small-world characteristics of these graphs, revealing that large reasoning models construct graphs exhibit significantly higher small-world properties. This small-world structures indicate that reasoning graph of large reasoning model have dense local clustering structures which likely contribute to improved reasoning performance. These graph-theoretic properties become more pronounced with increasing model size, suggesting that greater model capacity facilitates the formation of graph structures beneficial for reasoning. These distinct graph-theoretic characteristies provide critical insights into the mechanisms underlying enhanced reasoning performance.
Furthermore, to connect insights derived from reasoning graph properties directly to practical LLM training, we examine reasoning graphs through the lens of FT, particularly analyzing the s1 method [44]. Training the base model using the sl dataset leads to increased graph diameters, and notably, training on the improved sl-v1.1 dataset, which achieves higher performance, results in even larger graph diameters. These results indicate that more effective SFT data for reasoning can be characterized by distinctive graph properties, suggesting practical guidance for designing better data construction methods.
In summary, contributions are below:
Graph-Theoretic Analysis of Enhanced Reasoning (Section 4): Through empirical analyses across multiple datasets, we identify distinctive graph properties of large reasoning models, including (1) increased cyclic behavior, (2) larger graph diameters, and (3) heightened small- world characteristics. These structural patterns offer key insights into the mechanisms behind recent breakthroughs in the reasoning performance of L.LMs, highlighting how advanced models explore broader range of reasoning states and transition between them more effectively.

---

Insights for Effective SFT Data Construction (Section 5) By examining how SFT datasets influence reasoning graph characteristics, we reveal that refined datasets lead to larger graph diameters and better reasoning performance. These insights provide actionable guidelines for designing training data to explicitly enhance reasoning capabilities.
2 Related Works
Approaches to Enhanced Reasoning Recent methods for enhancing reasoning in language models include: (1) search-based methods that utilize additional inference-time computation iterative self-improvement, and (2) reinforcement learning (RL)-based fine-tuning, which has notably driven significant performance breakthroughs. Search-based methods involve external computation of inference time such as parallel sampling [34, 5], sophisticated verifier-based searches [38, 70, 62], or internal in-context refinement and self-correction strategies [17, 18, 71, 33, 27]. Despite their effectiveness, these methods often require careful design or redundant computation. RL methods autonomously discover effective reasoning strategies, encompassing both off-policy [74, 24] and on-policy techniques [75, 30, 9]. Recent advances like DeepSeek-RI [10], leveraging algorithms such as PPO [50] and GRPO [51], demonstrate significant improvements through structured reasoning traces [35, 73]. A notable phenomenon is the emergence of the "aha moment", " characterized by models spontaneously revising their reasoning strategies, inspiring new training approaches such as s1 [44] and Think-DPO [69].
Motivated by these findings, this research elucidates reasoning improvements by analyzing underlying reasoning graph structures.
Analytical Studies Reasoning Capabilities Earlier studies prior to recent breakthroughs in RL-based reasoning emphasized reasoning graphs to explain LLM capabilities. Previous work [48] theoretically and empirically demonstrated that reasoning capabilities emerge due the locality property inherent in natural language data; specifically, models achieve better accuracy by traversing intermediate variables frequently co-occurring during training. Additionally, Wang al. [63] pro- posed extracting reasoning graphs by clustering internal model states using K K-means, hypothesizing that pre-training data enables random walk over these graph nodes. Other studies [13, 67, 6, 59] have created simple toy tasks with explicit reasoning graphs to better understand the mechanisms un- derlying reasoning abilities language models. More recently, DeepSeek R1 [10] have demonstrated significant improvements through RL without explicit reasoning supervision. Analyses following this advancement have explored steering vectors [58], cognitive behaviors [19], and anthropomorphic expressions [68].
However, while recent reasoning models are featured with enhanced reasoning traces, studies have analyzed them through reasoning graph perspective. Addressing this gap is essential for understanding current advancements.
3 Graph Properties of LLM's Reasoning Process
3.1 Extracting Reasoning Graph from LLM's Representations
Let D (sx, W be our evaluation set of N questions. For each question x E D. we prompt the where each re is delimited by newline character ('An') and thus represents one reasoning step. We denote the total number of reasoning steps per question as T,, and the token length of a segment as Lt- Let hf, E Rd be the hidden state at Transformer layer for the μ-th token of segment r; (illustrated in 1 Σ/ hf
model to generate sequence of intermediate reasoning steps R =
Figure 2-(a)). and define the segment representation as the mean: sf
Node Definition Following the previous work [63], we aggregate segment representations S= {ff 1 T, z E D} and run K-means (default K = 200) to obtain clusters {C*) 1K with centroids {cx). Each centroid corresponds to node in the reasoning graph:
V = fvi,..., "x], d(u;, U2) = ||: e, 2l2-
The distance d(v,, vy)) between nodes n and v is defined as the Fuclidean distance between their corresponding centroids e; and cy. Figure 2-(b) presents representative nodes obtained from clustering
3

---

Layer
MG
Node
Generated Reasoning Step
First leg: 10 30 300.
Multiplicative (Node 4)
Now multiply that by 120 x 1.157625.
Layor2
Total chairs: 22 + 66 = 88.
Layert
Additive (Node 21)
Total sofas: 20 + 64 = 84.
Wait, but need add this Wait, no. Let me clarify...
Question
Ans
Wait (Node 1.5)
Reoooring Step?
(a) Extract reasoning graphs from LLMs
(b) Representative Nodes from Reasoning Graph
Figure 2: (a) Illustration of the methodology used to extract reasoning graphs from LLMs. (b) Representative nodes obtained from clustering the DeepSeck-FR RI-Distill- Distill-Qwen-- using GSM8K dataset.
the DeepSeek-R --iistill- tristil--wwe-32BB [10] using GSM8K dataset [7]. Each clustered node corresponds to simple computations encountered within tasks. As characteristic of reasoning models, some nodes include the term "wait", indicative of an "aha moment" where the model rechecks its outputs. A more fine- areegrained analysis, leveraging LLM-as-judge characterize the semantics of these clusters, is provided in Appendix B.
Edge Construction Informally, the edges in the reasoning graph represent the sequential path of nodes visited by the model for each question during inference. Formally, for each question æ, let (i1, i2, ........ (.), where i, = arg ming ||s cxl12 assigns segment to its nearest centroid. We then define the directed-edge set
E = { + ) it=
yielding the reasoning graph G (V, E") for that question. The reasoning graph properties (cycle density, diameter, small-world index) are then computed over G.
3.2 Measuring Graph Properties
Having extracted reasoning graphs from LLM representations, we evaluate their structural properties from three perspectives: (1) Cycles, (2) Diameter, and (3) Small-Words index. Simple implementa- tions of each method are provided in Appendix C.
Cycles We detect cycles in the reasoning graph, defined repeated visits to the same node, excluding self-loops and adjacent duplicates. This is because repetitive behaviors frequently observed in large reasoning models do represent meaningful cycles [68]. We define the cycle detection ratio as the proportion of reasoning graphs containing at least one cycle across all samples. Additionally, measure the cycle count of each reasoning graph as the maximum number of repeated visits to any single node (excluding self-loops, which repeat the same sentence).
Diameter To compute the reasoning graph diameter, defined as the maximum shortest path distance between any two reachable nodes, we run Dijkstra's is algorithm [12] from each node u. We record its distance map d(u, v),, and define: diameter = max,, max, d(u, v). A large diameter indicates that the reasoning graph explores a wider range of potential reasoning nodes during inference.
Small-Worle index Small-world organisation is robust network feature that has been observed in diverse domains-sociall networks [57, 65], biological and neural systems [28, 53], ecological webs [43], and technological networks such the World-Wide Web [1]. While the graph diameter represents the maximum geodesic length, it says nothing about local connectivity. We therefore evaluate the small-world index. Following Humphries and Gurney [26]. we first symmetrise the directed reasoning graph to obtain an undirected neighbour set N(()) for each node i. With n, [i)),,, Ne
K( n; 2 2}}, and Nr. >..>. Evfa 1 reachable from u). we define
C. # (edges among neighbors of i} (n - 1
Σ C, L- E. Eogu d(u, im,22
C
Nc
NL
where d(u, v) is the shortest-path distance from node u to node v. Letting be the total number of nodes and K the mean degree of the undirected graph, approximate the corresponding

---

Qwen2.5-32B
sample #5
sample #21
sample #45
sample #211
sample #689
DeepSock-Rl-Distill RI-Distill -wennl32B
sample #5
sample #21
sample #45
sample #211
sample #689
Figure 3: Visualization of reasoning graphs GSM8K dataset using t-SNE embeddings. The upper shows graphs from base model (Qwen2.5-322) while the lower row represents those from the large reasoning model (DeepSeek- --ikk-RR--Disssse Diis-iilll-ee Owen-32B). Compared to the base model, the reasoning model exhibits qualitatively broader exploration with notably more cycles in its reasoning graphs.
Cycle Detertion Ratio on GSMBK
Cycle Detection Ratio on MATH-500
Cycle Detection Ratio en AIME2024
Cycle Detection Ratio
1.9
1.0
Cycle Detection Ratio
3.0
8.8
0.80
Owe12.5-328
8.a
Owen2.3-328
0.8
8.50
0.61
Detectioon
0.6
0.6
Deeg Sart--i--ati--eeeeeeee
9.4
0.4
6.4
9.2
0:15
0.11
1.06
3.00
0.2
0.
Target
Ratio
0.0
Target Layer Ratio
6.0
3.3
Target Ratio
0.5
(a) GSM8k
(b) MATH500
(c) AIME 2024
Figure 4: Comparison of cycle detection ratios across different layers in the large reasoning model (DeepSeek- Ri-Distill-Qwen-
and the base model (Qwen2.5-32BB evaluated three tasks: GSM8K, (b) MATH500, and (e) AIME 2024. Results consistently show that large reasoning model exhibits significantly higher cycle detection ratios than the base model at all layer ratios and tasks. Additionally, trend emerges, indicating that the cycle detection ratio increases as task difficulty escalates from GSM8K through MATH500 to AIME 2024.
In N and define the small- In K L/Lraa The clustering coefficient describes the tendency of nodes to form tightly interconnected groups, while the average path length indicates how efficiently information propagates through the network. The small-world index combines these characteristics, highlighting graph's ability maintain local cohesion while supporting rapid global connectivity.
K Rénčs-Rényi random-graph baseline values [4] as Crand N-T Lrand
world index by S C/Crand
Analyzing Enhanced Reasoning through Graph-Theored Properties
We utilize the Qwen2.5 family distilled from DeepSeek-R1 [10] as large reasoning models, available in sizes of 1.5B, 7B. 14B, and 32B parameters. The corresponding base models and their details are provided in Appendix D. By default,
the highest- 32B variant. Unless otherwise specified, we extract reasoning graphs from the hidden layer positioned at 90% depth (e.g., layer 58 in the 64-layer 32B model). We employ the GSM8K [7], MATH500 [23], and AIME 2024 [11] datasets for constructing the reasoning graphs. Additional analyses on non- mathematical tasks, including Strategy QA [22] and LogicalDeduction from BIG-Bench [54], are provided in Appendix E.
4.1 Visualization and Quantification of Cycles in Reasoning Graphs
To intuitively capture the characteristics of reasoning graphs in large reasoning models, we first visualize reasoning graphs for some samples from the GSM8K dataset using 3-dimensional t-SNE
5

---

Cycle Distribution
Diameter Distribution
DeepSeek--R R1-Distill ----still---eerte Qwen-328
DeepSeek- R1 -il-Diiil-QQenneee---
Owen2.5-928
8000
Count
20
DiaimiL
6000
4000
2000
0.1
0.3
0.5 Target Layer Ratio
0.7 0.9
0.1
0.3
0.5 Target Layer Ratio
0.7
0.9
(a) Cycle Counts
(b) Diameter
Figure 5: (a) Distribution of cycle counts for the large reasoning model (DeepSeek RI Distill-Qwent 32B) and the base model (aven2.5-3BB) across various hidden layer depths. The reasoning model exhibits significandl higher eycle counts. (b) Distribution of reasoning graph diameters across various hidden layer depths. The diameter of reasoning graphs increases progressively with deeper layers. The reasoning model demonstrates significantly larger graph diameters, indicating a broader exploration space compared to the base model.
embedding, as depicted in Figure In the visualization, reasoning graphs are represented as directed arrows connecting nodes visited during inference. The base model demonstrates relatively simple and predominantly acyclic reasoning graphs. contrast, the large reasoning model exhibits more complex structures, characterized by frequent cyclic patterns and broader node coverage.
To quantitatively validate these qualitative observations, we employed the cycle detection method introduced in Section 3.2. Figure 4 shows cycle detection rates for GSM8K, MATH500, and AIME 2024 datasets, comparing the large reasoning model (blue) with the base model (orange). The horizontal axis denotes different relative depths of hidden layers (0.1, 0.3, 0.5, 0.7, and 0.9), Across
corresponding respectively to layers 6, 19, 32, 45, and 58 in the 64-layer Qwen2.5-322 all layers, the large reasoning model consistently exhibited a notably higher frequency of cyclic reasoning graphs compared to the base model. Additionally, we observed higher cycle detection rates at the earlier and later layers, with lower detection rates in intermediate layers. This pattern suggests that intermediate layers compress token representations, making less cycle detection difficult, whereas layers closer to input output exhibit clearer cyclic behaviors. The results for varying the hyperparameter & of the -means clustering are provided Appendix F, showing consistent trends across all tested values of k. Furthermore, another consistent trend emerges in which cycle detection ratios increase with the increasing complexity of tasks, progressing from GSMSK through MATH500 to AIME 2024. These findings reinforce the hypothesis that cycles within reasoning graphs contribute to the enhanced reasoning capabilities observed in large reasoning models.
Figure 5-(a) illustrates the distribution of cycle counts per sample. The large reasoning model consistently exhibits higher cycle counts. indicates that the large reasoning model not only exhibits higher proportion of samples containing cycles but also features higher average number of cycles per sample, approximately five cycles on average. These findings also emphasize the importance of eyclic structures in reasoning graphs as critical characteristic that improves reasoning performance.
4.2 Analyzing Reasoning Exploration through Graph Diameter
To better understand exploratory behaviors within reasoning graphs, we analyzed the distribution of graph diameters for both the large reasoning model and the base model using the GSM8K dataset. As shown in Figure 5-(b), the large reasoning model consistently demonstrates larger graph diameters across all examined layers compared to the base model. This indicates that the large reasoning model explores wider range of reasoning nodes during inference, likely contributing to its superior reasoning performance.
Moreover, we observed clear trend of progressively increasing graph diameters in deeper hidden layers, suggesting that richer contextual representations at deeper layers correspond to broader exploration scopes. These observations imply similarities between expanding the number of output tokens (thus enlarging the exploration scope) and increasing model depth from the perspective of reasoning graph diameters. Our findings suggest unified explanation based on reasoning graph diameters, which aligns closely with recent studies that emphasize improved reasoning through iterative deep-layer processing [42, 21]. Additional results for the MATH500 and AIME 2024 datasets can be found in Appendix G, demonstrating the same trends across all tasks.

---

Length Clastering 101C
Average Small-World Index (S)
2.5
R1 A1--isttll--ooe owen
0.35
5-322255326
Average Simall-Worid Index
0.30
0.29
Qwen2.5-328
Clustrira Confliient (C|C
2.0
0.25
0,20
0.20
0.15
0.15
0.05
0.04
0.01
0.02
0.0
Path Length lowe)
Target Layer Ratio
(a) Path Length & Clustering Coefficient
(b) Small- index
Figure 6: (a) Distribution of average path lengths and clustering coefficients in the large reasoning model -aeis--R---iisil--e-e-eeee
and the base model (Qwen2.5-32B)) Reasoning models exhibit larger cluster- ing coefficients and longer path lengths, indicating densely clustered yet widely separated reasoning nodes. (b) Comparison of small-world index computed from clustering coefficients and average path lengths. Across all layers, the reasoning model consistently exhibits higher small-world characteristics compared to the base model.
4.3 Small-Worte Structure and Enhanced Reasoning Efficiency
To gain deeper insights into the graph characteristics underlying reasoning graphs, examine their small-world properties. Specifically, using the AIME 2024 dataset, we examine clustering coefficients (C), average path lengths (I.), and their relationship through the Small- World Index (S).
Figure 6-(a) depicts the distributions clustering coefficients and average path lengths for the large reasoning model and the base model at a layer ratio of 0.9. The large reasoning model clearly demonstrates notably higher clustering coefficients alongside longer average path lengths. This combination indicates that reasoning graphs in large reasoning models form densely interconnected local clusters while also having some nodes connected by relatively long-range paths. Such a structure allows quick access via short paths arbitrary nodes within local clusters, facilitating easier recovery from incorrect reasoning pathways and potentially enhancing reasoning performance. This structural pattern aligns with recent theoretical findings [31] that model reasoning processes Markov chains, comprising densely connected nodes (representing simple reasoning steps) and sparsely connected critical transitions (representing complex reasoning steps). Additional results for other layer ratios are provided in Appendix H, consistently showing similar trends across all layers.
In Figure 6-(b), we further present the Small-World Index (S) computed across different hidden layer depths. The large reasoning model consistently shows higher values compared to the base model across all layers. Interestingly, reflecting the previously discussed cyclic reasoning behaviors. Collectively, these findings underscore the essential contribution of small- world graph characteristics to adveneed reasoning performance and suggest valuable avenues for future theoretical and empirical exploration.
observe declining trend in S near intermediate layers,
4.4 Impact of Model Size on Reasoning Graph Properties
To clarify how model size influences reasoning graph properties, analyzed the relationship between cycle detection ratios, reasoning graph diameters, cycle counts, and task accuracy on the AIME 2024 dataset. The results of the small-world index relative to model size are detailed in Appendix I.
Figure 7-(a) demonstrates that the cycle detection ratio generally increases with model size, peak- ing at 100% cycle detection rate in the 14B model. Interestingly, our largest 32B model, which achieves the highest task accuracy, exhibits a lower cycle detection ratio than the 14B model. To better understand the reason for this trend, we compares the outputs generated by 14B and 32B models in Figure We find that the 14B model experiences language mix- ing, a phenomenon recently reported [68] where language model switches and repeat differ- ent languages during the reasoning process. Such undesirable cyclicity likely explains why the 14B model, despite its higher cycle detection ratio, underperforms relative to the 32B model.

---

Cycle Detection Rarlo S 224 Acouracy
Cycle Count AIME2024 Accurscy
Diameter Distrbution S. AIME2024 Acouracy
0.38
AIME Accuracy
Accuracy
Accorscy
10
OP
Indo
Dentil Ovent
Qownn 4
Dinntll Qpenr 348
Disutl Simn-Za
Dunti OwendJa Lag
biasil Qrenn
Bioelt
Bloagi queee 928
Dixtil Ohean-
piuui
Diatili Qwee--14 148
qwen-220
(a) Cycle Detection
(h) Cycle Count
Diameter
Figure 7: (a) Relationship between cycle detection ratio and AIME 2024 accuracy across different model sizes. The cycle detection ratio generally increases with model size up to 14B, which achieves a 100% cycle detection ratio. (b) Relationship between cycle count and accuracy across different model sizes. Larger models demonstrate increased cycle counts, with the 32B model, which achieves the highest accuracy, exhibiting the greatest number cycles. (c) Relationship between reasoning graph diameter and accuracy across different model sizes. The 32B model, achieving the highest accuracy, also exhibits the largest graph diameter.
These findings indicate that although cyclic reasoning generally enhances reasoning effectiveness, certain types of cycles, such as language mixing, do not positively contribute to performance.
Figure 7-(b) illustrates that cycle counts progressively rise with model size, reaching the maximum in the 32B model. The observed positive correlation suggests that iterative revisitation of reasoning nodes fosters deeper refinement, thereby substantially enhancing performance in complex math tasks. Figure 7-(c) highlights that the reasoning graph diameter expands with model size, with the largest (32B) model consistently showing the most significant diameters alongside the highest accuracy. This suggests that broader exploration and complex reasoning paths are crucial for superior reasoning performance.
GT Answer: 809
14B
32B lat'
Ckay Mcewtty Sgua aut
10ving
Wet ta SeT
Tosa in wandla 4p (1M
Figure 8: Comparison of reasoning outputs from the 14B and 32B models. The 14B model exhibits language mixing [68], switch- ing languages through the response, while the 32B model maintains consistent language.
Collectively, these results imply that larger model capac- ities enable greater cyclicity and wider exploration within reasoning graphs, aligning with prior analyses showing smaller models struggle more in reasoning tasks [36].
5 Evolution of Reasoning Graph Properties during Supervised Fine- Tine-Tuning
To bridge the graph-theoretic properties of large reasoning models observed in Section 4 with practical improvements in reasoning performance, we conducted SFT using the sl dataset [44], which significantly enhances the reasoning capabilities of the 5-3B2.5-32B-- Insttuuctt We analyzed the evolution of reasoning graph properties across training steps to elucidate how these characteristics emerge through training. Detailed training parameters provided in Appendix J. Our experiments utilized two versions of the dataset: the original version (s1-v1.0 !) and an updated version (s1-v1.1 2), each consisting of 1000 training samples. Performance metrics on benchmark datasets indicated higher efficacy for the updated dataset, with v1.1 achieving 94.4% accuracy compared to 92.6% for v1.0 MATH500, and 56.7% versus 50.0% accuracy on AIME 2024, respectively [44].
Figure 9-(a) illustrates the differences in reasoning graph diameters between s1-v1.0 1.0 and sl-v1. at 200 training steps, and Figure 9-(b) at 400 training steps across various layers on AIME 2024. It can be observed that, on average, sl-v 1.1 consistently produces larger diameters across all layers. Furthermore, there more samples exhibiting larger diameters at 400 steps compared to 200 steps. These findings suggest that the diameter of reasoning graphs is amplified by SFT, and notably, superior SFT data such as sl-v1.1 enhances the reasoning graph diameter, effectively expanding the exploration space. The results for other checkpoints are provided in Appendix K.
8

---

Diameter Distribution at 200 steps
Diameter Distribution 400 steps
4000
Dataset
4000
Dataset
3500
s1 v1.0
3500
$1 v1.0
51 v1.1
s1 v1.1
2500
2500
1500
1500
1000
1000
0.5 Target Layer Ratio
Target Layer Ratio
(a) 200 steps
(b) 400 steps
Figure 9: Comparison of reasoning graph diameter distributions between datasets s1-v1.0 and s1-v1.1 across different hidden layers (a) 200 training steps and (b) training steps. Dataset s1-v1.1 consistently yields larger graph diameters compared to s1-v1.0, and graph diameters increase as training progresses from 200 to 400 steps, indicating enhanced exploration capacity facilitated by superior SFT data.
6 Discussion
In this work, we conducted an extensive analysis of reasoning graphs derived from large reasoning models, uncovering key structural properties that correlate with their enhanced performance. Our main findings highlight that large reasoning models consistently exhibit (1) greater cyclicity, (2) broader exploratory behaviors (larger diameters), and (3) pronounced small-worke characteristics compared to base models. These insights suggest sophisticated structures in reasoning graphs as critical factor driving reasoning improvements. Our results connect several observed behaviors in large reasoning models and offer implications for constructing more effective training datasets.
Aha Moment Models trained via RL have been reported to exhibit an intriguing phenomenon known as the "aha moment," where the model reconsidered its intermediate answers during reasoning [10, 68]. From the perspective of our reasoning graph analysis, this phenomenon aligns consistently with the observed cyclic structures (as illustrated in Figure 1). Although the "aha moment" was initially identified this behavior through the cycle properties of reasoning graphs, thereby contributing to a deeper mechanistic understanding of the "aha momen!" from the internal states of LLMs.
phenomenon at the generated token level, our study quantitatively measures
Overthinking and Underthinking Recent studies have highlighted specific reasoning inefficiencies in large reasoning models. Overthinking, characterized by redundant or excessively long reasoning processes, has been frequently observed, particularly in agent-based tasks [32, 55, 8, 14]. Conversely, models in the ol family display underthinking, rapidly switching thoughts without adequately ex- ploring potentially valuable reasoning paths [64]. These phenomena align closely with the graph properties have analyzed: redundant cyclic structures (discussed in Section 4.4) explain overthink- ing, while overly extensive exploratory behaviors (reflected in larger graph diameters, discussed in Section 4.2) may account for underthinking. Thus, our research clarifies these unique behaviors of large reasoning models through the lens of reasoning graph characteristics.
Implications for Reasoning SFT-Data Construction Some studies have significantly improved reasoning performance through SFT or DPO [49, 15] with limited data [72, 44, 69]. While these studies typically create datasets based on qualitative criteria such as difficulty, quality, and diversity, which are inherently challenging quantify, our proposed metric based on reasoning graph charac- teristics extracted from hidden states provide novel insights for dataset construction. For example, the high high-quality dataset s1-v1.1 demonstrated notably larger graph diameters, suggesting its structural properties are indicative of superior reasoning potential. Furthermore, even when comparing the sl-v1.0 against LIMO, the s1 dataset consistently yields reasoning graphs with larger diameters and more cycles across all layer depths. This suggests that higher-quality SFT data induces more exploratory and reflective reasoning behavior, further supporting the use of graph- traph-thooettic metrics indicators of data effectiveness.
shown in Appendix L,

---

Acknowledgements
We thank Heiga Zen and Yinlam Chow for their support and reviewing the draft of this paper. We also appreciate the funding support from Google Japan.
References
[1] Réka Albert, Hawoong Jeong, and Albert-LászI6 Barabási. Diameter of the world-wide web. Nature, 401 September 1999. ISSN 1476-4687. doi: 10.1038/4360. URL
http://dx. doi org/ 10w 1038/43601.
[2] Anthropic. Claude's extended thinking, February 2025. URL
wrrissleeeeetnddee triredddttiiiiigg.. Accessed: 2025-05-11.
[3] Leonard Bereska and Stratis Gavves. Mechanistic interpretability for AI safety re-
view. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL https:
/ //pppeeeeeee...t www.:/netttoooooott/eeeeeeee
Survey Certification, Expert Certification.
[4] Béla Bollobás. Random Graphs, volume 73 of Cambridge Studies in Advanced Mathematics. Cambridge University Press, Cambridge, 2 edition, 2001. ISBN 97805217972211
[5] Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoe V. Le, Christopher Ré, and Azalia Mirhoseini. Large language monkeys: Scaling inference compute with repeated sampling. 2024. URL https://arxiv /awxiv.vvvvv////////eee//aaaavva hetg/abs//aee 21787.
[6] Vivien Cabannes, Charles Arnal, Wassim Bouaziz, Xingyu Alice Yang, Francois Charton, and Julia Kempe. Iteration head: A mechanistic study of chain-of-thought. In The Thirty- eighth Annual Conference on Neural Information Processing Systems, 2024. URL https: //pppprerieer..tefffrrmmiiccc.. 2=-oorrkm2210000000t505550
[7] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reicchrro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.1416. 141668 2021.
[8] Alejandro Cuadron, Dacheng Li, Wenjie Ma, Xingyao Wang, Yichuan Wang, Siyuan Zhuang, Shu Liu, Luis Gaspar Schroeder, Tian Xia, Huanzhi Mao, Nicholas Thumiger, Aditya Desai, lon Stoica, Ana Klimovic, Graham Neubig, and Joseph E. Gonzalez. The danger of overthinking: Examining the reasoning-action dilemma in agentic tasks, 2025. URL https: abs/2502. /t2///5522.00055..
org/
[9] Ganqu Cui, Lifan Yuan, Zefan Wang, Hanbin Wang, Wendi Li, Bingxiang He, Yuchen Fan, Tianyu Yu, Qixin Xu, Weize Chen, al. Process reinforcement through implicit rewards. arXiv preprint arXiv:2502.01455
2025.
[10] DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang. Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue. Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Honghui Ding. Huajian Xin, Huazuo Gao, Hui Qu, Hui Li. Jianzhong Guo. Jiashi Li, Jiawei Wang, Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, J. L. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang, Lecong Zhang, Liang Zhao, Litong Wang, Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Meng Li, Miaojun Wang. Mingming Li, Ning Tian, Panpan Huang, Peng Zhang, Qiancheng Wang, Qinyu Chen, Qiushi Du, Ruiqi Ge. Ruisong Zhang, Ruizhe Pan. Runji Wang, R. Chen, R. Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang. Shuiping Yu, Shunfeng Zhou, Shuting Pan, S. Li, Shuang Zhou, Shaoqing Wu, Shengfeng Ye, Tao Yun, Tian Pei, Tianyu Sun, T. Wang, Wangding Zeng, Wanjia Zhao, Wen Liu, Wenfeng Liang, Wenjun Gao, Wengin Yu,
10

---

Wentao Zhang, W. L. Xiao. Wei An, Xiaodong Liu, Xiaohan Wang. Xiaokang Chen, Xiaotao Nie, Xin Cheng. Xin Liu, Xin Xie, Xingchao Liu, Xinyu Yang, Xinyuan Li, Xuecheng Su, Xuheng Lin, X. Q. Li, Xiangyue Jin, Xiaojin Shen, Xiaosha Chen, Xiaowen Sun, Xiaoxiang Wang, Xinnan Song. Xinyi Zhou, Xianzu Wang, Xinxia Shan, Y. K. Li, Y. Q. Wang, Y. X. Wei, Yang Zhang, Yanhong Xu, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Wang, Yi Yu, Yichao Zhang, Yifan Shi, Yiliang Xiong, Ying He, Yishi Piao, Yisong Wang, Yixuan Tan, Yiyang Ma, Yiyuan Liu, Yongqiang Guo. Yuan Ou, Yuduan Wang, Yue Gong, Yuheng Zou, Yujia He, Yunfan Xiong. Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Y. X. Zhu, Yanhong Xu, Yanping Huang, Yaohui Li, Yi Zheng, Yuchen Zhu, Yunxian Ma, Ying Tang. Yukun Zha, Yuting Yan, Z. Z. Ren, Zehui Ren. Zhangli Sha, Zhe Fu, Zhean Xu, Zhenda Xie, Zhengyan Zhang, Zhewen Hao, Zhicheng Ma, Zhigang Yan, Zhiyu Wu, Zihui Gu, Zijia Zhu, Zijun Liu, Zilin Li, Ziwei Xie, Ziyang Song, Zizheng Pan, Zhen Huang, Zhipeng Xu, Zhongyu Zhang, and Zhen Zhang. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement 12948.
learning, 2025. URL httttt//p//axiiii www.riiii.......ioeeeoooee org/abs
[11] Di Zhang. di di-zhang-fde fuoeggg-duuAAA AAEEE 190___999__2222
lime_1983_2022 2025.
URL hatps://huggingteee http://ngaace.c//daaatttstte co/datasets/
[12] Edsger W Dijkstra. A note on two problems in connexion with graphs. Numerische mathematik, 1(1):269-271, 1959.
[13] Subhabrata Dutta, Joykirat Singh, Soumen Chakrabarti, and Tanmoy Chakraborty. How to think step-by-step: A mechanistic understanding of thain-of-thought reasoning. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL https //opnr/eee///ereeee..eeeee. net/
forum?id-uHLDiQVES
[14] Chenrui Fan, Ming Li, Lichao Sun, and Tianyi Zhou. Missing premise exacerbates overthinking: Are reasoning models losing critical thinking skill?, 2025. URL atttps://arxiv. org/ abs/ 2504. 065144.
[15] Hiroki Furuta, Kuang-Huei Lee, Shixiang Shane Gu, Yutaka Matsuo, Aleksandra Faust, Heiga 7en, and Izzeddin Gur. Geometric-averaged preference optimization for soft preference labels. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
[16] Hiroki Furuta, Gouki Minegishi, Yusuke Iwasawa, and Yutaka Matsuo. Towards empirical interpretation of internal circuits and properties in grokked transformers on modular poly- nomials. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL https://openreviess
[17] Kanishk Gandhi, Dorsa Sadigh, and Noah D Goodman. Strategic reasoning with language models. arXiv preprint ar 1355 2023.
[18] Kanishk Gandhi, Denise HJ Lee, Gabriel Grand, Muxin Liu, Winson Cheng, Archit Sharma, and Noah Goodman. Stream of search (sos): Learning to search in language. In First Conference on Language Modeling, 2024.
[19] Kanishk Gandhi, Ayush Chakravarthy, Anikait Singh, Nathan Lile, and Noah D. Goodman. Cognitive behaviors that enable self-improving reasoners, or, four habits of highly effective stars, 2025. URL https: aaxiiv.iiiv..iii//aaaeaaaavv org/ ///aabb/22500 /20p//250..00 01307.
[20] Leo Gao. Tom Dupre la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. Scaling and evaluating sparse autoencoders. In The Thirteenth International Conference on Learning Representations, 2025. URL https:// openreview. wwwparrvieennet/tfeeee httt:/forumidd=tssttee
[21] Jonas Geiping, Sean McLeish, Neel Jain, John Kirchenbauer, Siddharth Singh, Brian R. Bartoldson, Bhavya Kailkhura, Abhinav Bhatele, and Tom Goldstein. Scaling up test- time compute with latent reasoning: A recurrent depth approach, 2025. URL https:
//arxiv. argg/bs//5022
[22] Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant. Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies. Transactions of the Association for Computational Linguistics (TACL), 2021.
11

---

[23] Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. NeurlPS, 2021.
[24] Matthew Douglas Hoffman, Du Phan, David Dohan, Sholto Douglas, Tuan Anh Le, Aaron Parisi, Pavel Sountsov, Charles Sutton, Sharad Vikram, and Rif A Saurous. Training chain-of-thought via latent-variable inference. In NeurlPS, 2023.
[25] Robert Huben, Hoagy Cunningham, Logan Riggs Smith, Aidan Ewart, and Lee Sharkey. Sparse autoencoders find highly interpretable features in language models. In The Twelfth International Conference on Learning Representations, 2024, URL id=F76bWRSLeK.
[26] Mark Humphries and Kevin Gurney, Network 'small-world-ness:: A quantitative method for determining canonical network equivalence. PloS one, 3:e0002051, 02 2008.
[27] Hyconbin Hwang, Doyoung Kim, Seungone Kim, Seonghyeon Ye. and Minjoon Seo. Self- explore: Enhancing mathematical reasoning in language models with fine-grained rewards. arXiv preprint wrXiv:2404.10346, 2024.
[28] H. Jeong, B. Tombor, R. Albert. Z. N. Oltvai, and A.-L. Barabási. The large-scale organization of metabolic networks. Nature, 407(68044) 1-15551-6564 654, October 2000. ISSN 1476-4687. doi: 10.1038/35036627. URL http://dx:: doi. org/ 10. 1008//3503600
[29] Koray Kavukcuoglu. 2025.
Gemini 2.5: Our most intelligent ai model, March URL https://blog wwwgle/technoloooo hoaemoooogggooool/oooo coogleeddeppinde gemini- Sieiiiimodel-thiniie wwwhiinkinnuuddttess--iaae updates maraatees-arch..e
Accessed: 2025-05-11.
[30]
Amirhossein Kazemnejad, Milad Aghajohari, Eva Portelance, Alessandro Sordoni, Siva Reddy, Aaron Courville, and Nicolas Le Roux. Vineppo: Unlocking rl potential for Ilm reasoning through refined credit assignment. arXiv preprint arXiv:2410.01679, 2024.
[31] Juno Kim, Denny Wu, Jason Lee, and Taiji Suzuki. Metastable dynamics of thain-of- reasoning: Provable benefits of search, rl and distillation, 2025. URL https://////x org/ abs/2502.01694
[32] Abhinav Kumar, Jaechul Roh, Ali Naseh, Marzena Karpinska, Mohit lyyer, Amir Houmansadr, and Eugene Bagdasarian. Overthink: Slowdown attacks on reasoning 1lms, 2025. URL
https://arxix: http://axvvvirg///////// /bt///2/.//2...////////// 02542.
[33] Aviral Kumar, Vincent Zhuang, Rishabh Agarwal, Yi Su, John D Co-Reyes, Avi Singh, Kate Baumli, Shariq Iqbal, Colton Bishop, Rebecca Roelofs, et al. Training language models to self-correct via reinforcement learning. arXiv preprint (tt:::2409.2991111 2024.
[34] Chen Li, Weiqi Wang, Jingeheng Hu, Yixuan Wei, Nanning Zheng, Han Hu, Zheng Zhang, and Houwen Peng. Common 7b language models already possess strong math capabilities. arXiv preprint 240.:::20033470666 2024.
[35] Dacheng Li, Shiyi Cao, Tyler Griggs, Shu Liu, Xiangxi Mo, Shishir G Patil, Matei Zaharia, Joseph E Gonzalez, and Ion Stoica. Lims can easily learn to reason from demonstrations structure, not content, is what matters! arXiv preprint arX1v:2502.07377, 2025.
[36] Yuetai Li, Xiang Yue, Zhangehen Xu, Fengqing Jiang, Luyao Niu, Bill Yuchen Lin, Bhaskar Ra- masubramanian, and Radha Poovendran. Small models struggle to learn from strong reasoners, 2025. URL http:: //tp://aa./////// http:/awwv.ogg///a///////// wogg/aa//////0 abs/ 12143.
[37] Tom Lieberum. Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Nicolas Sonnerat, Vikrant Varma, János Kramár, Anca Dragan, Rohin Shah, and Neel Nanda. Gemma scope: Open sparse autoencoders everywhere all once on gemma 2, 2024. URL htttt://aaaaxii org/abs 2etaa////0.... 051477.
[38] Hunter Lightman. Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, llya Sutskever, and Karl Cobbe. Let's verify step by step. In The Twelfth International Conference Learning Representations, 2023.
12

---

[39] Gouki Minegishi, Hiroki Furuta, Yusuke Iwasawa, and Yutaka Matsuo. Rethinking evaluation of sparse autoencoders through the representation of polysemous words. The Thirteenth International Conference on Learning Representations, 2025. URL https://oppnneviex.. net/forum? 1d=
[40] Gouki Minegishi, Hiroki Furuta, Shohei Taniguchi, Yusuke Iwasawa, and Yutaka Mat- suo. Beyond induction heads: In-context meta learning induces multi-phase circuit emer- gence. In Forty-second International Conference on Machine Learning, 2025. URL http:///ppeeaeeeeeeeeeeeerrrrrrr
[41] Gouki Minegishi, Yusuke Iwasawa, and Yutaka Matsuo. Bridging lottery ticket and grokking: Understanding grokking from inner structure of networks. Transactions on Machine Learn- ing Research, 2025. ISSN 2835-8856. eQeYyupltm.
URL attps://openreet net/ meetfformmidd
[42] Takeru Miyato, Sindy Löwe, Andreas Geiger, and Max Welling. Artificial kuramoto oscillatory neurons. In The Thirteenth International Conference on Learning Representations, 2025, URL https://openreviet: wwt./eoemmiddddddeeiee.....
[43] JOSE M. MONTOYA and RICARD V. SOLÉ. Small world patterns in food webs. Journal of Theoretical Biology, 214(3):405-412, 2002. ISSN 0022-5193. doi: https://doi.or/// (006/jtbi.2001..222 2460. URL attps://wwx. www.wwwddeeicc.rrct....mmm
$0022519801926609.
[44] Niklas Muennighoff, Zitong Yang. Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannanch Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candes, and Tatsunori Hashimoto. sl: Simple test-time scaling. In Workshop on Reasoning and Planning for Large Language Models, 2025. net aeae//oooommn--ddorrggllll
URL
[45]
Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Dawn Drain, Deep Ganguli, Zac Batfield-Dodde Danny Hernandez, Scott Johnston, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah. In-context learning and induction heads. Transformer Circuits Thread, 2022. https//teansformerciiccisss wwtp20020//-ooteereeaainnnaaonno heads/index.htm.
[46] OpenAI, Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, Aleksander Mądry, Alex Baker-Whitcomb, Alex Beutel, Alex Borzunov, Alex Carney, Alex Chow, Alex Kirillov, Alex Nichol, Alex Paino, Alex Renzin, Alex Tachard Passos, Alexander Kirillov, Alexi Christakis, Alexis Conneau, Ali Kamali, Allan Jabri, Allison Moyer, Allison Tam, Amadou Crookes, Amin Tootoochian, Amin Tootoonchian, Ananya Kumar, Andrea Vallone, Andrej Karpathy, Andrew Braunstein, Andrew Cann, Andrew Codispoti, Andrew Galu, Andrew Kondrich, Andrew Tul- loch, Andrey Mishchenko, Angela Baek, Angela Jiang, Antoine Pelisse, Antonia Woodford, Anuj Gosalia, Arka Dhar, Ashley Pantuliano, Avi Nayak, Avital Oliver, Barret Zoph, Behrooz Ghorbani, Ben Leimberger, Ben Rossen, Ben Sokolowsky, Ben Wang, Benjamin Zweig, Beth Hoover, Blake Samic. Bob McGrew, Bobby Spero, Bogo Giertler, Bowen Cheng, Brad Lightcap, Brandon Walkin, Brendan Quinn, Brian Guarraci, Brian Hsu, Bright Kellogg, Brydon Eastman, Camillo Lugaresi, Carroll Wainwright, Cary Bassin, Cary Hudson, Casey Chu, Chad Nelson, Chak Li, Chan Jun Shern, Channing Conger, Charlotte Barette, Chelsea Voss, Chen Ding, Cheng Lu, Chong Zhang, Chris Beaumont, Chris Hallacy, Chris Koch, Christian Gibson, Christina Kim, Christine Choi, Christine McLeavey, Christopher Hesse, Claudia Fischer, Clemens Winter, Coley Czarnecki, Colin Jarvis, Colin Wei, Constantin Koumouzelis, Dane Sherburn, Daniel Kappler, Daniel Levin, Daniel Levy, David Carr. David Farhi, David Mely, David Robinson, David Sasaki, Denny Jin, Dev Valladares, Dimitris Tsipras, Doug Li, Duc Phong Nguyen, Duncan Findlay. Edede Oiwoh, Edmund Wong, Ehsan Asdar, Elizabeth Prochl, Elizabeth Yang. Eric Antonow, Eric Kramer, Eric Peterson, Eric Sigler, Eric Wallace, Bugene Brevdo, Evan Mays, Farzad Khorasani, Felipe Petroski Such, Filippo Raso, Francis Zhang, Fred von Lohmann, Freddie Sulit, Gabriel Goh, Gene Oden, Geoff Salmon, Giulio Starace, Greg Brockman, Hadi Salman, Haiming Bao, Haitang Hu, Hannah Wong, Haoyu Wang, Heather Schmidt, Heather
13

---

Whitney, Heewoo Jun, Hendrik Kirchner, Henrique Ponde de Oliveira Pinto, Hongyu Ren, Hui- wen Chang, Hyung Won Chung, Ian Kivlichan, Ian O'Connell, Ian O' O'Connell, Ian Osband, Ian Silber, Ian Sohl, Ibrahim Okuyucu, Ikai Lan, llya Kostrikov, llya Sutskever, Ingmar Kanitschei- der, Ishaan Guirajani, Jacob Coxon, Jacob Menick, Jakub Pachocki, James Aung, James Betker, James Crooks, James Lemnon, Jamie Kiros, Jan Leike, Jane Park, Jason Kwon, Jason Phang, Jason Teplitz, Jason Wei, Jason Wolfe, Jay Chen, Jeff Harris, Jenia Varavva, Jessica Gan Lee, Jessica Shich, Ji Lin, Jiahui Yu, Jiayi Weng, Jie Tang, Jieqi Yu, Joanne Jang, Joaquin Quinonero Candela, Joe Beutler, Joe Landers, Joel Parish, Johannes Heidecke, John Schulman, Jonathan Lachman, Jonathan McKay, Jonathan Uesato, Jonathan Ward, Jong Wook Kim, Joost Huizinga, Jordan Sitkin, Jos Kraaijeveld, Josh Gross, Josh Kaplan, Josh Snyder, Joshua Achiam, Joy Jiao, Joyce Lee, Juntang Zhuang, Justyn Harriman, Kai Fricke, Kai Hayashi, Karan Singhal, Katy Shi, Kavin Karthik, Kayla Wood, Kendra Rimbach, Kenny Hsu, Kenny Nguyen, Keren Gu-Lemberg, Kevin Button, Kevin Liu, Kiel Howe, Krithika Muthukumar, Kyle Luther, Lama Ahmad, Larry Kai, Lauren Itow, I Lauren Workman, Leher Pathak, Leo Chen, Li Jing, Lia Guy, Liam Fedus, Liang Zhou, Lien Mamitsuka, Lilian Weng, Lindsay McCallum, Lindsey Held, Long Ouyang, Louis Feuvrier, Lu Zhang, Lukas Kondraciuk, Lukasz Kaiser, Luke Hewitt, Luke Metz, Lyric Doshi, Mada Aflak, Maddie Simens, Madelaine Boyd, Madeleine Thompson, Marat Dukhan, Mark Chen, Mark Gray, Mark Hudnall, Marvin Zhang, Marwan Aljubeh, Mateusz Litwin, Matthew Zeng, Max Johnson, Maya Shetty, Mayank Gupta, Meghan Shah, Mehmet Yatbaz, Meng Jia Yang. Mengchao Zhong, Mia Glaese Mianna Chen, Michael Janner, Michael Lampe, Michael Petrov, Michael Wu, Michele Wang. Michelle Fradin, Michelle Pokrass, Miguel Castro, Miguel Oom Temudo de Castro, Mikhail Pavlov, Miles Brundage, Miles Wang, Minal Khan, Mira Murati, Mo Bavarian, Molly Lin, Murat Yesildal, Nacho Soto, Natalia Gimelshein, Natalie Cone, Natalie Staudacher, Natalie Summers, Natan LaFontaine, Neil Chowdhury, Nick Ryder, Nick Stathas, Nick Turley, Nik Tezak, Niko Felix, Nithanth Kudige, Nitish Keskar, Noah Deutsch, Noel Bundick, Nora Puckett, Ofir Nachum, Ola Okelola, Oleg Boiko, Oleg Murk, Oliver Jaffe, Olivia Watkins, Olivier Godement, Owen Campbell-Moor. Patrick Chao, Paul McMillan, Pavel Belov, Peng Su, Peter Bak, Peter Bakkum, Peter Deng, Peter Dolan, Peter Hoeschele, Peter Welinder, Phil Tillet, Philip Pronin, Philippe Tillet, Prafulla Dhariwal, Qiming Yuan, Rachel Dias, Rachel Lim, Rahul Arora, Rajan Troll, Randall Lin, Rapha Gontijo Lopes, Raul Puri, Reah Miyara, Reimar Leike, Renaud Gaubert, Reza Zamani, Ricky Wang. Rob Donnelly, Rob Honsby, Rocky Smith, Rohan Sahai, Rohit Ranchandani, Romain Huet, Rory Carmichael, Rowan Zellers, Roy Chen, Ruby Chen, Ruslan Nigmatullin, Ryan Cheu, Saachi Jain, Sam Altman, Sam Schoenholz, Sam Toizer, Samuel Miserendino, Sandhini Agarwal, Sara Culver, Scott Ethersmith, Scott Gray, Sean Grove, Sean Metzger, Shamez Hermani, Shantanu Jain, Shengjia Zhao, Sherwin Wu, Shino Jomoto, Shirong Wu, Shuaiqi, Xia, Sonia Phene, Spencer Papay, Srinivas Narayanan, Steve Coffey, Steve Lee, Stewart Hall, Suchir Balaji, Tal Broda, Tal Stramer, Tao Xu, Tarun Gogineni, Taya Christianson, Ted Sanders, Tejal Patwardhan, Thomas Cunninghman, Thomas Degry. Thomas Dimson, Thomas Raoux, Thomas Shadwell, Tianhao Zheng, Todd Underwood, Todor Markov, Toki Sherbakov, Tom Rubin, Tom Stasi, Tomer Kaftan, Tristan Heywood, Troy Peterson, Tyce Walters, Tyna Eloundou, Valerie Qi, Veit Moeller, Vinnie Monaco, Vishal Kuo, Vlad Fomenko, Wayne Chang, Weiyi Zheng, Wenda Zhou, Wesam Manassra, Will Sheu, Wojeiech Zaremba, Yash Patil, Yilei Qian. Yongjik Kim, Youlong Cheng, Yu Zhang, Yuchen He, Yuchen Zhang, Yujia Jin, Yunxing Dai, and Yury Malkov. Gpt-4o system card, 2024. URL Sttttt::.// /tapaii.////vviaaaaa wwwp:/org/aa/////000 21276.
[47] OpenAl, ;, Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed EI- El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Camey, Alex Iftimie, Alex Karpenko, Alex Tachard Passos, Alexander Neitz, Alexander Prokofiev, Alexander Wei, Allison Tam, Ally Bennett, Ananya Kumar, Andre Saraiva, Andrea Vallone, Andrew Duberstein, Andrew Kondrich, Andrey Mishchenko, Andy Applebaum. Angela Jiang. Ashvin Nair, Barret Zoph, Behrooz Ghorbani, Ben Rossen, Benjamin Sokolowsky, Boaz Barak, Bob McGrew, Borys Minaiev, Botao Hao, Bowen Baker, Brandon Houghton, Brandon McKinzie, Brydon Eastman, Camillo Lugaresi, Cary Bassin, Cary Hudson, Chak Ming Li, Charles de Bourcy, Chelsea Voss. Chen Shen, Chong Zhang, Chris Koch, Chris Orsinger, Christopher Hesse, Claudia Fischer, Clive Chan, Dan Roberts, Daniel Kappler, Daniel Levy, Daniel Selsam, David Dohan, David Farhi, David Mely, David Robinson, Dimitris Tsipras, Doug Li, Dragos Oprica, Eben Freeman, Eddie Zhang, Edmund Wong, Elizabeth Proehi, Enoch Cheung, Eric Mitchell, Eric Wallace, Erik Ritter, Evan Mays, Fan Wang, Felipe Petroski Such, Filippo Raso, Florencia
14

---

Leoni, Foivos Tsimpourlas, Francis Song, Fred von Lohmann, Freddie Sulit, Geoff Salmon, Giambattista Parascandolo, Gildas Chabot, Grace Zhao, Greg Brockman. Guillaume Leclere, Hadi Salman, Haiming Bao, Hao Sheng, Hart Andrin, Hessam Bagherinezhad, Hongyu Ren, Hunter Lightman, Hyung Won Chung, Ian Kivlichan, Ian O' Connelll Ian Osband, Ignasi Clavera Gilaberte, Ilge Akkaya, llya Kostrikov, llya Sutskever, Irina Kofman, Jakub Pachocki, James Lennon, Jason Wei, Jean Harb, Jerry Twore, Jiacheng Feng, Jiahui Yu, Jiayi Weng, Jie Tang, Jieqi Yu, Joaquin Quiñonero Candela, Joe Palermo, Joel Parish, Johannes Heidecke, John Hallman, John Rizzo, Jonathan Gordon, Jonathan Uesato, Jonathan Ward, Joost Huizinga, Julie Wang, Kai Chen, Kai Xiao, Karan Singhal, Karina Nguyen, Karl Cobbe, Katy Shi, Kayla Wood, Kendra Rimbach, Keren Gu-Lemberg, Kevin Liu, Kevin Lu, Kevin Stone, Kevin Yu, Lama Ahmad, Lauren Yang, Leo Liu, Leon Maksin, Leyton Ho, Liam Fedus, Lilian Weng. Linden Li, Lindsay McCallum, Lindsey Held, Lorenz. Kuhn, Lukas Kondraciuk, Lukasz Kaiser, Luke Metz, Madelaine Boyd, Maja Trebacz, Manas Joglekar, Mark Chen, Marko Tintor, Mason Meyer, Matt Jones, Matt Kaufer, Max Schwarzer, Meghan Shah, Mehmet Yathaz, Melody Y. Guan, Mengyuan Xu, Mengyuan Yan, Mia Glaese, Mianna Chen, Michael Lampe, Michael Malek, Michele Wang, Michelle Fradin, Mike McClay, Mikhail Pavlov, Miles Wang, Mingxuan Wang, Mira Murati, Mo Bavarian, Mostafa Rohaninejad, Nat McAleese, Neil Chowdhury, Neil Chowdhury, Nick Ryder, Nikolas Tezak, Noam Brown, Ofir Nachum, Oleg Boiko, Oleg Murk, Olivia Watkins, Patrick Chao, Paul Ashbourne, Pavel Izmailov, Peter Zhokhov, Rachel Dias, Rahul Arora, Randall Lin, Rapha Gontijo Lopes, Raz Gaon, Reah Miyara, Reimar Leike, Renny Hwang. Rhythm Garg, Robin Brown, Roshan James, Rui Shu, Ryan Cheu, Ryan Greene, Saachi Jain, Sam Altman, Sam Toizer, Sam Toyer, Samuel Miserendino, Sandhini Agarwal, Santiago Hernandez, Sasha Baker, Scott McKinney, Scottie Yan, Shengjia Zhao, Shengli Hu, Shibani Santurkar, Shraman Ray Chaudhuri, Shuyuan Zhang. Siyuan Fu, Spencer Papay, Steph Lin, Suchir Balaji, Suvansh Sanjeev, Szymon Sidor, Tal Broda, Aidan Clark, Tao Wang. Taylor Gordon, Ted Sanders, Tejal Patwardhan, Thibault Sottiaux, Thomas Degry, Thomas Dimson, Tianhao Zheng. Timur Garipov, Tom Stasi, Trapit Bansal, Trevor Creech, Troy Peterson, Tyna Eloundou, Valerie Qi, Vineet Kosaraju, Vinnie Monaco, Vitchyr Pong. Vlad Fomenko, Weiyi Zheng. Wenda Zhou, Wes McCabe, Wojeiech Zaremba, Yann Dubois, Yinghai Lu, Yining Chen, Young Cha. Yu Bai, Yuchen He, Yuchen Zhang, Yunyun Wang, Zheng Shao, and Zhuohan Li. Openai ol system card, 2024. URL https: //tpx//aaii...//aa weaeziiiorgg/e abs///////// aa////bs//222// 227122162..
[48] Ben Prystawski, Michael Y. Li, and Noah Goodman. Why think step by step? reasoning emerges from the locality of experience. In seirth Conference on Neural Information Processing Systems, 2023. URL /ttttp:://ppnreeetee net/ wttt:/forum?tan wwwrruniddccxxUFVV..
[49] Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
[50] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal
policy optimization algorithms, 2017. URL https: // /ttp///aavvvvv////////e///a.aava www.ii..../aaaaaaaaaaae abs 1177. 06347.
[51] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang. Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024. URL https://arxit org/abs/ 2402. 03300.
[52] Lee Sharkey, Bilal Chughtai, Joshua Batson, Jack Lindsey, Jeffrey Wu, Lucius Bushnaq, Nicholas Goldowsky-Dill, Stefan Heimersheim, Alejandro Ortega, Joseph Isaac Bloom, Stella Biderman, Adrià Garriga-Alonso, Arthur Conmy, Neel Nanda, Jessica Mary Rumbelow, Martin Wattenberg, Nandi Schoots, Joseph Miller, William Saunders, Eric J Michaud, Stephen Casper, Max Tegmark, David Bau, Eric Todd, Atticus Geiger, Mor Geva, Jesse Hoogland, Daniel Murfet, and Thomas McGrath. Open problems in mechanistic interpretability. Transactions on Machine Learning Research, 2025. ISSN 2835-8856, URL https: //operreeee//eeeieeeee /nttte/rrveeeeee///eeeeee net/ forum?id--1177mm22225 Survey Certification.
[53] Olaf Sporns and Jonathan Zwi The small world of the cerebral cortex. Neuroinformatics, 2: 145 145-62,00 02 2004. doi: 10. 13:1//N1.::221155
[54] Aarohi Srivastava et al. Beyond the imitation game: Quantifying and extrapolating the capabili- ties of language models, 2022. URL https: / /rwxivv.vvvaa//////aa//aaa org/ abs/a//b...2//////////////// 04615.
15

---

[55] Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Shaochen Zhong, Hanjie Chen, and Xia Hu. Stop overthinking: A survey on efficient reasoning for large language models, 2025. URL https: /ttpxxxxvvvvo/////e/e/. www.iv.orggabb//5000 16419.
[56] Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunningham, Nicholas Turner, Callum McDougall, Monte MacDiarmid, C. Daniel Freeman, Theodore R. Sumers, Edward Rees, Joshua Batson, Adam Jermyn, Shan Carter, Chris Olah, and Tom Henighan. Scaling monosemanticity: Extracting interpretable features from claude 3 sonnet. Transformer Circuits Thread, 2024. URL https:: http://tttaaoofmrrrioooooo scaling www.smmmntiiiit..yooooooomaaa wtw.iiiittiiiiiiniiiii html.
pub/ pub/2024/
[57] JEFFREY TRAVERS and STANLEY MILGRAM. An experimental study of the small world problem**the study was carried out while both authors were at harvard university, and was financed by grants from the milton fund and from the harvard laboratory of so- cial relations. mr. joseph gerver provided invaluable assistance in summarizing and criti- cizing the mathematical work discussed this paper. In Samuel Leinhardt, editor, So- cial Networks, pages 179-197, Academic Press, 1977. ISBN 978-0- 12-442550-0 doi: URL https: /wwp://wwwww../o sciencedirect.
[58] Constantin Venhoff, Iván Arcuschin, Philip Torr, Arthur Conmy, and Neel Nanda. Understanding reasoning in thinking language models via steering vectors. Workshop on Reasoning and Planning for Large Language Models, 2025. URL https: httt:::///penreview.eo www.wwwww.e.o...... fortt/ffrmmidd- OshVWNOBc2.
[59] Boshi Wang, Xiang Yue, Yu Su, and Huan Sun. Grokking of implicit reasoning in trans- formers: A mechanistic journey to the edge of generalization. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors, Advances in Neu- ral Information Processing Systems, volume 37, pages 55238-95266 Curran Associates, Inc., 2024. URL https://procceedings newriis cc/ http/ppper_fiiiie wwl.ffflesfppppe /ppprr/0024//ile/ 0021700077ec771bbf4466666616
Paper-Conf
Conf erence. pdf.
[60] Boshi Wang, Xiang Yue, Yu Su, and Huan Sun. Grokking of implicit reasoning in transformers: mechanistic journey to the edge of generalization. In The ehirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https://openrevien. wtw.pppewrreeeeeeeeettorrrrr id=D4QgSWxi0b.
[61] Kevin Ro Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. Interpretability in the wild: circuit for indirect object identification in GPT-2 small. In The Eleventh International Conference on Learning Representations, 2023. URL. https:
//openreview wet.//orummiaaagsssseo.....
[62] Peiyi Wang, Lei Li, Zhihong Shao, RX Xu, Damai Dai. Yifei Li. Deli Chen, Yu Wu, and Zhifang Sui. shehh-shepherd: Verify and reinforce llms step-by-step without human annotations. arXiv preprint arXiv:2312.08993 2023.
[63] Xinyi Wang. Alfonso Amayuelas, Kexun Zhang, Liangming Pan, Wenhu Chen, and William Yang Wang. Understanding the reasoning ability of language models from the perspec- tive of reasoning paths aggregation. In ICLR 2024 Workshop: How Far Are We From AGI, 2024. URL https:///peenrrries www.ettooormdmdddonnnaaaa 11100.14hW1...
[64] Yue Wang, Qiuzhi Liu, Jiahao Xu, Tian Liang, Xingyu Chen, Zhiwei He, Linfeng Song, Dian Yu, Juntao Li, Zhuosheng Zhang, Rui Wang, Zhaopeng Tu, Haitao Mi, and Dong Yu. Thoughts are all over the place: On the underthinking of ol-liee Ilms, 2025. URL https: //arxiv. arg/abs/25011 18585.
[65] Duncan J. Watts and Steven H. Strogatz. Collective dynamics of 'soldl-worldd networks. Nature, 393:4400442, 1998. URL attps:://aa semanticscholat orgg/orppsID:333466.
[66] xAI. Grok beta - the age of reasoning agents, February 2025. URL https: //x/ ai/news/ grok-3. 3. Accessed: 2025-05-11.
16

---

[67] Violet Xiang, Charlie Snell, Kanishk Gandhi, Alon Albalak, Anikait Singh, Chase Blagden, Duy Phung, Rafael Rafailov, Nathan Lile, Dakota Mahan, et al. Towards system 2 reasoning in llms: Learning how to think with meta chain-of- thain-of-though. arXiv preprint arXiv:250.004682 2025.
[68] Shu Yang. Junchao Wu, Xin Chen, Yunze Xiao, Xinyi Yang, Derek F. Wong, and Di Wang. Understanding aha moments: from external observations to internal mechanisms, 2025. URL https://arxiv. org/ aba/ 02956.
[69] Wang Yang, Hongye Jin, Jingfeng Yang, Vipin Chaudhary, and Xiaotian Han. Thinking preference optimization, 2025. URL wwtth::/t//aaae org/ abs/2502...... 13173.
[70] Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Ad- vances in neural information processing systems, 36:11809-11888 11822, 2023.
[71] Tian Ye, Zicheng Xu, Yuanzhi Li, and Zeyuan Allen-Zhu. Physies of language models: Part 2.2, how to learn from mistakes grade-school math problems. arXiv preprint arXiv:2408.16299 16293, 2024.
[72] Yixin Ye, Zhen Huang. Yang Xiao, Ethan Chern, Shijie Xia, and Pengfei Liu. Limo: Less is more for reasoning, 2025. URL https://arxxii
chain-of-thought reasoning in llms, 2025. URL httgg/bbs/2002 03387.
[73] Edward Yeo, Yuxuan Tong, Morry Niu, Graham Neubig, and Xiang Yue. Demystifying long wwtp://g/.aaa/......./// 03373.
[74] Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Goodman. Star: Bootstrapping reasoning with reasoning. Advances in Neural Information Processing Systems, 35:15476-15488, 2022.
[75] Eric Zelikman, Georges Raif Harik, Yijia Shao, Varuna Jayasiri, Nick Haber, and Noah Good- man. Quiet-star: Language models can teach themselves to think before speaking. In First Conference on Language Modeling, 2024.
17

---

A Broader Impacts
This research sheds light on the underlying mechanisms responsible for improved reasoning per- formance in LLMs, potentially impacting various fields within artificial intelligence and machine learning. From an interpretability perspective, our findings offer explanations for performance gains in previously opaque (black-box) large reasoning models and provide insights toward the construction of better reasoning architectures.
B Automatic Node Labeling with LLMs
To better understand what kinds of reasoning patterns represented by the centroids when A = 200, we conducted an automatic labeling experiment using a large language model. Specifically, we used the GPT-4o 4o-mmmmm [46] API assign theme to each centroid, based on the reasoning steps associated with it in the reasoning graph of DeepSeck-R1-Diste -i---isiilll---e--2222 on the GSMSK dataset.
We provided the following system prompt to GPT, and then input multiple reasoning steps corre- sponding to each centroid as the user prompt:
You are data analyst. The following an output from a LLM.
Your task theme in 1-3 English words.
carefully read the text and summarize its main
Table presents the assigned theme for each centroid, together with example reasoning steps that were mapped to that centroid. This analysis revealed that many centroids align with interpretable reasoning patterns. In addition to previously reported in Figure 2 such as Add, Multiply, and Wair, we identified centroids associated with higher-level computations (e-g.. Calculations Totals, Averages), semantics-bound reasoning (e.g., Age Calculations, Cost Calculations), and structural elements (e.g.. Answer Formatting, Placeholder Tags).
We also observed centroids linked to reasoning behaviors, such as Planning, which reflects the model's initial steps when approaching math problem. Moreover, instances of Wait appear to split into multiple subtypes. For example, centroids like Calculation Correctness and Reevaluation capture the model's 's tendency reassess or double-check its output. A distinct centroid labeled Inconsistencies highlights cases where the model detects contradictions in its reasoning and attempts to revise its calculations.
Overall, these results indicate that the centroids discovered by the reasoning graph clustering proce- dure correspond wide variety of meaningful and interpretable reasoning techniques.
18

---

Table I: Examples of automatically identified themes and corresponding reasoning steps.
Theme (Node id)
Examples of Reasoning Steps
Calculations - Totals (node 83)
5 10 = 15. Then, 15 = 24. Finally, 24 =7.
Let add them step by step. 500 + 1500 2000. Then, 2000 125 2125.
Calculations Average (node 119)
Average = (Sum of all values) / (Number values). Average Speed = Total Distance / Total Time 250 miles / hours 50 mph.
Calculations - Division (node 41)
125,000 20 6.250.
120 pieces (15 pieces per pack) packs. 80 / 10 8 weeks.
Age Calculations (node 147)
Sum of their ages in two years = (B + 2) (2B + =
C 2 x (James's age in 8 years) 5.
Cost Calculations (node 168)
$47.00 x 5 $235.00.
Keenan's 's weekly cost $160 4 $40.
8 x 8 $64.00.
Answer Format (node 137)
The answer is: 50.
The answer is: 8.
Placeholder Tags (node 26)
</think>
Planning (node 163)
Okay, so need to figure how much Leah has spent her kitten far. Let break down step by step. Hmm, let's break it down step by step.
Calculation Correctness (node 52)
Wait, that seems straighttorward, but let me double-check... Wait, let me double-check my calculations to make sure...
Reevaluation (node 100)
Wait, maybe I made mistake in the equations. Wait, maybe I made mistake in the equations. Let try model it again...
Inconsistencies (node 110)
Wait, that's problem. 15 23, which is more than total time 20.
Wait, perhaps the shows part of the 30%, and the other activities are part of the remaining 70%. But that doesn't 't make sense...
19

---

C Measuring the Graph Property Implementation
Here is a sketch of Python code for detecting cycles and computing the diameter of reasoning graphs:
from collections import defaultdict. import heapq
deque
def analyze grr.p....pppp.... simple path, distances):
adj defaultdiet list)
for u, v. w in zpahh [ath[1:], distances):
adj [uj[u]] I.ul..ppee J.ppendd ((v. w))
Cyale detection seen has_loop set(). False loop_count =
entry_node None
for 1. node in enumerate(path
node
has las_p True
entry node mode
loop_con loop_count path count(node) break add
Diameter and Arg Length
def dijkstra (u):
dist [u:
heap [(0. u>l
while heap:
d. node heapq heappop heap)
for neighbor. weight in adjinode
new new_dist
weight
if neighhor not in dist or new nist diethheih dist neighbbr] new_dist
heapq. heappush (heap. (new_dist. neighbor))
relurn disl
all_ diddattassss dijjkttra (node) for node in adjl
diameter max( (max dists. values ()) dists all all_distances), default=0) avg_path_ Jength
sum (sum dists. ()) for dists in all_ distances) sum(len dists)-1 for dists all_ distances)
Clustering Coefficien: undirected defaultdict(set)) for u, neighbors in adj ittms
v. neighhors: undirected [u]. undirected [V] add
clustering sum, count_ cc 0, 0 for node. nbrs undirected it:ms
if len len(nhrs) < 2:
continue
actual_edges- sum(1 for in ubrs for In abrs If and in (vdirecteddv)) clustering sum += actual_edges / (Jen(nbrs) (len (nbrs)-1) count_ec
avg_clustering clustering_sum / count_ee count_ce else
Swall-World Index
(ndireettd))
K sum numelen ubrs) for ubrs in undirected, values ()) N if N else C_rand = K (N 1) if N elxe 0 I__and math moth / math. (K) N and else float('inf')
clustering_non norm cvg_ctusteiing C_rand if C_rand else faah_length_norrg _orm avg_path__ wyg_pat_leength / I_rand if L_rand else 0 wooll_world_index = clustering_norm path_length_norm
lath_length_norm else
return has loop. loop_rount, diameler, avg clusterrnn avg_path eag_path_length small, world index
20

---

D Details of Large Reasoning Model and Base Model
For the large reasoning models, we utilize distilled models from the DeepSeck-RI series, which derived from Qwen-based models. Correspondingly. their base models are the original Qwen and Llama models prior to distillation. detailed list of these models provided in Table 2.
Table 2: Comparison of large reasoning models and corresponding base models.
Base Model
Large Reasoning Model
Qwen2.5-Mathh 1.5B|
1.522
Swen2.5-Math-
DeepSeck-RI- -iieies-----isiii-------
alama-3.1-8B"
DeepSeck-Rl-Distil--
Qwen2.5-14BB
Qwen2.5-32B'
E Extension to Non-Math Reasoning Tasks
To examine whether the observed properties of reasoning graphs generalize beyond mathematical reasoning, we extended our evaluation to include two additional tasks inspired by prior work [63]] (i) StrategyQA [22], multi-hop question answering dataset, and (ii) LogicalDeduction, a logical reasoning dataset from BIG-Bench [54]. In Figure 10, we compare reasoning graph properties between DeepSeek-RI. R1-ikk--R--isiillll Qwen- owen-32B (reasoning model) and Qwen2.5-32B (base model) on these datasets. Similar to the math tasks reported in the main paper, the reasoning model exhibits markedly different graph characteristics, including higher cycle rates, larger diameters, and stronger small-world properties. These consistent patterns across multiple domains provide additional evidence that the reasoning graph framework captures structural properties general manner.
Cycle Detection StrategyQA
Biamater Bleribution StreteppOA
World indes Bkbriburien on mneegyOs
0.02 0.52
0.10
Deepdieak Outil Qwen-
Deeptionk-13 Diaill-Qwen- 328 Qwen2.5--2 5-320
10000
Qween2.22 5-320
4000
0,0000
Layer
Duger Liger
Torper Lyper
(a) StrategyQA: Cycles
(b) StrategyQA: Diameter
(c) Strategy QA: Small-world index
Cydle Detestipe Rigg on Cogical Dequction
Diamster Distiiuzion Logical
Logical
Rado
12000
DuepSeeb--11 Queen Qwen-328
DeepSeei-R33 Quett-Qwen--t2 Qweet So-2
Gweed 2-:2
8000
Spall Woer
24
a0co
Target
Turgrt
Torper Layer
(d) LogicalDeduation: Cycles
(c) LogicalDeduution: Diameter
(f) LogicalDeddution. Small-world index
Figure 10: Comparison of reasoning graph properties between Distill-Qwen-32B and Qwen2.5-32B two non-math reasoning datasets: (top row) Strategy and (bottom row) LogicalDeduuttion Columns correspond Cycles, Diameter, and Small world index.
Small-world
-at 1.
14B
21

---

Experiments with Different K Values in K-Means Clustering
We report differences in cycle detection ratios when varying the number of clusters in K- K-means clustering, specifically for the GSM8K dataset, shown in Figure 11. As expected, decreasing results in fewer clusters and a higher ratio of detected cycles. Across all values, the large reasoning model consistently exhibits a higher cycle ratio compared to the base model.
Cycle Detection Ratio GSM8K
Cycle Detection Ratio GSMBK
Cycle Delection Ratio GSMBK
Gycle Detection Rabio
DkapSeekk4k1-oati Qwen-328
Cycle Dittcctio Ratio
DaapSesk-R3 Dutili--ees-322e
0.23
0.65
Detection
0.57
0.61
0.18
0:12
1.08
0u
Target Layer
0.1
0.1
0.8
Target Layer
9ul
Target Layer
9.4
0,7
6
(a) k=50
(b) k=100
(c) k=200
Figure Comparison of cycle detection ratios across different layers for the large reasoning model (DeepSeek- RI RI--istill-QUee Sailll--een-2225 and the base model (Qwen2.5-328) evaluated on three tasks: (a) A- = 50, (b) l 100, and (c) k = 200. Results consistently show that the large reasoning model exhibits significantly higher cycle detection ratios than the base model at all layer ratios and k.
G Diameter Analysis MATH500 and AIME 2024
Figure 12 compares reasoning graph diameters for the MATH500 and AIME 2024 datasets. The large reasoning model consistently exhibits greater diameters than the base model, with clear trend of increasing diameter in deeper hidden layers, aligning with observations from the GSM8K dataset.
Diameter Distribution
Diameter Distribution
7000
DeepSeell-1--Dieill-OOo
5000
DeepSeek-R1-Disttl--Ooe
6000
Qwen2.5-328
4000
Qwen2.5-326
3000
3000
2000
2000
1000
Target Layer Ratio
Target Layer Ratio
(a) MATH500
(b) AIME 2024
Figure 12: Diameter of Reasoning Graph in DeepSeck-Ri RI-Distill --Distill-Qwen--e Owen-32B and Qwen2.5-32B.
22

---

H Liyer-wise Clustering Coefficient and Average Path Length
Figure 13 shows the clustering coefficient and average path length for each layer. The large reasoning model consistently exhibits higher clustering coefficients at all layer ratios, contributing to its enhanced small-world characteristics.
Despleatd Qward222
329
Owant 5-300
Oewen2 5-300
(a) layer ratio=0.1
(b) layer ratio=0.3
(c) layer ratio=0.5
(d) layer ratio=0.7
Figure 13: The clustering coefficient and average path length for each layer.
I Model Size and Small-World Index
Figure 14 presents the relationship between model size and the Small-World Index. The results suggest that the Small-Wort property becomes more pronounced as the model size increases.
Small-Worid World index on AIME2024
Diameter
Accuracy
0.7
Distill qwen-1.8 5B
Distili Owen-78 B
Distill Owen-14p
Distill Qwen-320
Figure 14: Relationship between model size and the Small- World Index.
23

---

32768 tokens Training Details
We conducted supervised fine-tuning (SFT) experiments using the Qwen2.5-32B-Inss Innttrcctt model as our base model. The training was executed on computing node equipped with 8 NVIDIA H200 GPUs for training and single NVIDIA H200 GPU for inference.
The detailed training configuration is summarized in Table 3.
Table 3: Detailed training configuration for the SFT experiments.
Parameter
Value
Base Model
Qwen2.5-32B 3----2-B--Inneeee
Dataset
simplescalingssIK simplescaling/51K-LL
Number of Epochs
5
Learning Rate
1 x 10
Learning Rate Scheduler
Cosine (minimum LR: 0)
Batch Size
(Effective: GPUs micro-batch size
Gradient Accumulation Steps
Weight Decay
X 10
Optimizer
AdamW (31 0.9, $2 0..5)
Warmup Ratio
0.05
Precision
bf16
Gradient Checkpointing
Enabled
FSDP Configuration
Full Shard Data Parallel (auto-wrap)
Block Size
Inference conducted using single NVIDIA H200 GPU evaluate trained models and generate results presented in the paper.
K Diameter of other s1 checkpoints
Figure compares the diameters reasoning graphs at 100 and 500 training steps using the dataset [44]. In both cases, version s1-v1.1 demonstrates larger diameters compared to s1-v1.0, 10, and diameters tend to increase with further training steps.
Diameter Distribution at 100 steps
Diameter Distribution 500 steps
4500
4500
4000
Dataset
4000
Dataset
s1 v1.0
s1 v1.0
3000
s1 v1.1
3000
v1.1
2000
2000
1500
1500
500
500
0.1
0.3
0.5
0.7
0.9
0.1
0.3
0.5
0.7
0.9
Target Layer Ratio
Target Layer Ratio
(a) 200 steps
(b) 400 steps
Figure 15: Cycle Graph Detection Ratio in DeepSeck-RI RI-Distill Qwen-32B and Qwen2.5-32B.
24

---

L Impact of SFT Data Quality on Reasoning Graph Structure.
To further investigate the relationship between data quality and reasoning-gran graggh properties,
compared two supervised fine-tuning (SFT) datasets: LIMO [72] and s1 v1.0 [44]. The s dataset has previously been recognized for its strong performance in enhancing reasoning abilities [44].
We constructed reasoning graphs from the hidden states of Qwen2.5-32B-Insmmeee prompted with data from sl and LIMO (note that did fine-tune Qwen2.5-32B-nnst
on these datasets).
As shown in Figure 16, reasoning graphs derived from s consistently exhibit larger diameters and higher cycle counts across all examined layer depths. This indicates that the sl dataset inherently induces exploration of a broader range of latent reasoning states, resulting in more iterative reasoning. In contrast, graphs derived from LIMO show narrower reasoning trajectories with fewer cycles, suggesting more linear and potentially shallow reasoning processes.
These findings suggest that higher-qualin SFT data possess richer reasoning -aannnnggggahhhhhh characteristics (such increased cycles and larger diameters), which in turn contribute improved performance when used for fine-tuning. Thus, reasoning grapig analysis from hidden states offers novel perspec- tive and practical guidance for creating better SFT datasets.
Loop Count by Target Layer Ratio Dataset $1 v1.0 vs LIMO
Diameter Distribution by Target Layer Ratio Dataset s1 v1.0 LIMO
6000
Dataset
100
Dataset LIMO
5000
Diameter
v1.0
Loop Count
s1 v1.0
2000
40
2000
1000
Target Layer Ratio
Target Ratio
(a) Diameters
(b) Cycle counts
Figure 16: We constructed reasoning graphs from the hidden states of 1-nn2.5--2B--nseee data from sl and LIMO (note that we did fine-tune Qwen2.5-32B. 1et.2.5-3BB-nntttuee Instruct these datasets). The sl regareedd
prompted with
yields larger diameters and higher cycle counts, indicating broader exploration and more reflective reasoning behavior.
M Limitations
This study introduces the concept of reasoning graphs as a tool identify distinctive graph-theoretic properties that may explain recent breakthroughs in the reasoning performance of large language models (LLMs). While our findings provide a novel explanatory perspective on the reasoning capabilities of advanced models, concrete guidelines constructing models with superior reasoning performance remain insufficient. Although we experimentally examine the relationship between graph properties and reasoning-SPP in Section 5, using these insights first step toward building more effective reasoning models is left for future work.
Our analysis focuses transitions in the context direction of hidden states, but it does not provide feature-level [25, 56, 20, 37, 39] or circuit-level analyses [45, 61, 40] as commonly studied in mechanistic interpretability [3, 52]. Furthermore, how the distinctive graph properties observed in reasoning models emerge during training dynamics [16, 60, 41] remains an open question, which we leave an important direction for future work.
25

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The abstract and introduction clearly state that analyzing reasoning graph properties provides insights into the capabilities of large reasoning models, aligning well with the paper's empirical results and scope.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer:
[Yes]
Justification: Limitations are explicitly discussed in Appendix M, covering assumptions and empirical scope.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "I "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
26

---

Answer:
[NA]
Justification:
This paper is empirical and does not present theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: The appendix provides detailed Python code snippets illustrating methods for measuring graph properties, ensuring reproducibility.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
27

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
The code is provided in single submission zip file with supplementary
material, facilitating full reproduction.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cm
public/guin (oooli//uuides/Codeeutiiii hwtribimsssiooooolliiioooooooooe
onPolicy) icy for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: / //niss wwtppclcccc/////cccccccccccc..lt wttpc//gg//eeeeeeecceoeecesseees
wwwsssssssiigiyooo for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Training details for supervised fine-tuning (SFT) are comprehensively pre- sented in the Appendix J.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: The Appendix clearly describes training details and includes distribution analyses of sample results, supporting statistical significance.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
28

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπτог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Detailed descriptions of computational resources used for experiments are provided in the Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https //neurip///uuiss.... httpppll/ttci/suiiieeeep..ooppee
Answer: [Yes]
Justification: The research adheres fully the ethical guidelines, involving no violation of ethical standards in methodology or reporting.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., . if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Broader societal impacts are explicitly discussed in the ??.
Guidelines:
The answer NA means that there is no societal impact of the work performed.

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification: Safeguards for the responsible release and use of models and data are described in the Appendix A.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All used assets, including datasets and pre-existing models, are properly cited, and their respective licenses and terms of use are explicitly mentioned.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
30

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswi thcode com comddatasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The Appendix includes thorough documentation for all newly introduced code and datasets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question:
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This research does involve crowdsourcing human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
This research does not involve crowdsourcing or human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
31

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [Yes]
Justification: The Appendix clearly documents the large language models used in experi- ments and mentions LLM usage in writing and formatting.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (https : hhe.ripp.cc/Coofeeeeeeeeee wwt//c/////o//cceeccee /ccc/oofeeeeecc//225///... for what should or should not be described.
32


Reviews:
[{"review_id": "SagxyszeQN", "reviewer": "Reviewer_dTpT", "summary": "This paper introduces an interpretability method that extracts reasoning graphs from reasoning traces generated by large language models when solving complex tasks. These graphs are created by clustering the hidden state representations of models at each intermediate reasoning step when solving problems across an evaluation set. The centroids of these clusters are nodes, and transitions between reasoning steps for a given solution define the directed edges in the graph. The authors compare graph-theoretic properties of graphs constructed from traces taken from reasoning models and their non-reasoning, “base” counterparts to show how these metrics correlate with reasoning ability. They claim that this method illuminates how reasoning models approach complex problems, and they argue that these metrics can be used to build better reasoning datasets.", "questions": "(See questions above)\n\n1. Figure 2 lists a small set of nodes taken from the reasoning graph as well as representative example reasoning steps for each node. The three nodes included are labelled “multiplicative,” “additive,” and “wait.” Are the nodes in the reasoning graphs studied in the paper always directly tied to concrete reasoning techniques? If we set K=200, then are all 200 nodes clearly associated with these techniques?", "limitations": "yes", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n* The authors test their method across a wide range of model sizes and evaluation datasets. They sweep across different hyperparameters like k-values for clustering and the layer at which hidden state representations are extracted, showing that relationships presented are not artifacts of cherry picked graph construction. Additionally, the authors demonstrate how these metrics change at different depths within the model.\n* The experiments are comprehensive and thoroughly demonstrate how different graph properties correlate with reasoning performance at different model sizes and target layers.  The examination of the nodes in the graphs, along with example reasoning steps, was very helpful for understanding what the reasoning graph might look like.\n* One question while reading the paper is whether increases in these metrics are merely correlated with superior reasoning or whether reasoning models actually learn to leverage relevant techniques that are measured by these graph properties. The intervention study of training on s1 and iteratively measuring diameter distribution helps answer this question. Additionally, the comparison between s1 v1 and s1 v1.1 demonstrates that graph metrics not only increase with supervised finetuning on reasoning data but that they increase more on better data, supporting the authors’ claim that this method can be used for dataset curation.\n\nWeaknesses:\n* One possible confound is that these graph metrics may increase with longer reasoning chains in general. An experiment controlling for token length/flops is needed.\n* The reasoning steps are segmented at newlines. Some models may have a propensity to generate longer sequences. Is comparing the graphs generated from these models fair?\n* Figures 7b and 7c show a massive increase in AIME24 accuracy between Distill Qwen-7B and Qwen-14B. However, the cycle counts are similar between the two, and graph diameter is noticeably larger for Qwen-7B. Is there any explanation for why this massive increase in accuracy is not correlated with these metrics?\n* Figure 3 demonstrates how DeepSeek-R1-Distill Qwen 32B exhibits more exploration and cyclicity in its reasoning graph compared to the corresponding base model. Were the examples presented in this figure solved by both models? If not, are all the nodes explored by the reasoning model essential to solving the answer. Recent papers like [1] claim that reinforcement learning with verifiable rewards improves pass@k at small k but not at large k, suggesting that RLVR does not expand the coverage of the base model. Do you think that overlaying graphs from multiple attempts from a base model would results in similar graph coverage to that of the reasoning model? Would studying the trajectory of the reasoning model in these problems help confirm or disprove the claim above?\n* Additional insight into what the cycles accomplish would be helpful. The discussion around figure 7a claims that cyclic reasoning generally enhances reasoning effectiveness except in cases like language mixing. The authors later state in section 6 that “redundant cyclic structures may explain overthinking.” Is there any way to differentiate between trivial and non-trivial cycles that harm or help reasoning performance? In addition, if the nodes are tied to general math operations as suggested in figure 2, would something relatively simple like adding and multiplying numbers in sequence manifest as a cycle?\n\n\n[1] [Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?\n](https://arxiv.org/abs/2504.13837)", "was_revised": true, "final_justification": "The authors addressed my concerns and questions - I am sticking with my original accept rating.  The paper adds a novel contribution to this field, and I believe their methodology and experiments are quite thorough.", "rebuttal": "We thank the reviewer for their thoughtful comments and valuable suggestions, which helped us strengthen our analysis and clarify key aspects of our work.\n\n**>W1&W2**\n\n>One possible confound is that these graph metrics may increase with longer reasoning chains in general. \n\n> The reasoning steps are segmented at newlines. Some models may have a propensity to generate longer sequences. Is comparing the graphs generated from these models fair?\n\nFirst, we note that some of the key graph metrics we use—such as the small-world coefficient—are normalized by definition (line 154). Specifically, small-worldness is computed by dividing the graph’s clustering coefficient and average path length by those of a random graph with the same number of nodes and edges. This normalization is designed to enable fair comparisons even when graphs vary in size, such as when models differ in the number of reasoning segments. \n\nSecond, our comparisons are not limited to base vs. reasoning models. We also analyze multiple reasoning models of different parameter sizes, all of which tend to produce long outputs. In those comparisons as well, we observe consistent trends: models with stronger reasoning performance tend to exhibit more structured graph properties, such as more cycles and higher small-worldness. This supports the claim that the graph metrics reflect reasoning capability rather than just output length or compute.\n\nFinally, and most directly addressing your suggestion, we did attempt experiments to control for output length by engineering prompts that elicit structured, similar-length responses across models. However, we found that **constraining the output in this way significantly degraded the performance of the reasoning models**. Therefore, in the main paper, we report results using unconstrained prompts, which better reflect the models’ natural and effective operating regimes.\n\nFor these reasons, we believe the graph properties we observe reflect real differences in how the models reason, not just differences in output length or how much they compute.\n\n**>W3**\n>Figures 7b and 7c show a massive increase in AIME24 accuracy between Distill Qwen-7B and Qwen-14B. However, the cycle counts are similar between the two, and graph diameter is noticeably larger for Qwen-7B. \n\nFigures 7 only present the cycle ratio, cycle count, and diameter metrics, but **Appendix G** includes results for the small-world index, which shows that the 14B model exhibits higher small-worldness than the 7B model.\nThe table below summarizes the average values of all four graph properties—cycle ratio, cycle count, diameter, and small-world index—based on the figures in the paper. As shown, the 14B model demonstrates better graph properties than the 7B model on **three out of four** metrics． \n\nThis suggests that, when considering the full set of graph metrics, there is a meaningful relationship between stronger graph properties and better model performance.\n| Metric / Model-size      | 7B  | 14B  | \n|---|---|---|\n| Cycles | 0.93 | **1.00** |  \n| Cycle counts | 3.56 |**4.10**| \n| Diameter | **2215.39** |1674.35 | \n| Small world index| 0.85 |**0.95** | \n\n**>Q1**\n>Are the nodes in the reasoning graphs studied in the paper always directly tied to concrete reasoning techniques? \n\nTo answer your question, we conducted an automatic labeling experiment using an LLM to investigate whether the nodes in the reasoning graph correspond to meaningful reasoning techniques. \n\nSpecifically, we used the `GPT-4o-mini` API to assign a theme to each node, based on the reasoning steps associated with nodes in the reasoning graph of Distill-Qwen-32B on the GSM8K dataset.\nWe provided the following system prompt to GPT, and then input multiple reasoning steps corresponding to each node as the user prompt.\n```\nYou are a data analyst. The following is an output from a LLM.\nYour task is to carefully read the text and summarize its main theme in 1–3 English words.\n```\nThe table below presents some of the identified themes along with reasoning steps corresponding to each node. In addition to previously reported categories like Add, Multiply, and Wait, we discovered centroids associated with:\n- Higher-level computations (Calculations Totals, Averages, Divisions)\n- Semantics-bound reasoning (Age Calculations, Cost Calculations)\n- Structural elements (Answer Formatting, Placeholder Tags)\n- Reasoning behaviors such as Planning\n- Variants of \"Wait\" behavior, including Reevaluation, Inconsistencies, and Calculation Correctness\n\n| **Themes** |**Examples**  |\n|---|---|\n| **Calculations Totals (node 83)**     | 5 + 10 is 15. Then, 15 + 9 is 24. Finally, 24 + 3 is 27|\n| |Let me add them step by step. 500 + 1500 is 2000. Then, 2000 + 125 is 2125. |\n| **Calculations  Average (node 119)**    | \\text{Average} = \\frac{\\text{Sum of all values}}{\\text{Number of values}}\n|  |\\text{Average Speed} = \\frac{\\text{Total Distance}}{\\text{Total Time}} = \\frac{250 \\, \\text{miles}}{5 \\, \\text{hours}} = 50 \\, \\text{miles per hour}|\n| **Calculations Division (node 41)**   | \\frac{125,000}{20} = 6,250|\n|  | \\frac{120 \\, \\text{pieces}}{15 \\, \\text{pieces/pack}} = 8 \\, \\text{packs}|\n| | \\frac{80}{10} = 8 \\text{ weeks}|\n| **Age Calculations (node 147)**  | Sum of their ages in two years = (B + 2) + (2B + 2) = 28|\n||Brooke's age = B|\n|  |C = 2 * (James’s age in 8 years) - 5|\n| **Cost Calculations (node 168)**       | $47.00 × 5 = $235.00|\n| |Keenan’s weekly cost = $160 ÷ 4 = $40|\n| |\\( 8 \\times 8 = \\$64.00 \\).|\n| **Answers Format (node 137)** | The answer is: 50|\n|  |The answer is: 48|\n|  |The answer is: 8|\n| **Placeholder Tags (node 26)**|<  /think>|\n|   |< /think>|\n| |< /think>|\n| **Planning (node 163)**| Okay, so I need to figure out how much Leah has spent on her new kitten so far. Let me break down the problem step by step.|\n| | Hmm, let's break it down step by step.|\n| **Calculation Correctness (node 52)** |Wait, that seems straightforward, but let me double-check…|\n||Wait, let me double-check my calculations to make sure…|\n| **Reevaluation (node 100)**|Wait, maybe I made a mistake in the equations.|\n| |Wait, maybe I made a mistake in the equations. Let me try to model it again,…|\n| **Inconsistencies (node 110)** |Wait, that's a problem. 15 + 8 = 23, which is more than total time 20.|\n|  |Wait, perhaps the shows are part of the 30%, and the other activities are part of the remaining 70%. But that doesn't make sense…|\n\nThese findings suggest that many nodes align with concrete reasoning techniques, supporting the validity of the reasoning graph.\n\nWhile not all 200 nodes are directly tied to concrete reasoning operations—some, like `</think>`, reflect structural or formatting artifacts commonly produced by the model—the majority of nodes correspond to interpretable reasoning patterns.  This demonstrates that the reasoning graph captures valid reasoning techniques. We believe this supports the soundness of our approach.\n\nTo improve clarity, we will include the full list of labeled node themes and examples in the Appendix of the revised version.\n\n**>W5**\n> Additional insight into what the cycles accomplish would be helpful. \n\nAs you point out, not all cycles may contribute equally to reasoning quality, and differentiating between helpful and redundant cycles is a key challenge.\nTo provide further insight, we conducted an additional analysis comparing the reasoning graphs of samples that were answered **correctly** (n = 936) and **incorrectly** (n = 64) by the reasoning model (Distill-Qwen-32B). The results are summarized below:\n\n| Metric | Correct | Incorrect |\n|----|----|----|\n| Cycle Ratio | 0.67 | 0.92 |\n| Diameter (mean ± var) | 5970.50 ± 45.10 | 3666.34 ± 34.93 |\n\nInterestingly, incorrect samples exhibit a higher proportion of cycles, yet have smaller graph diameters. This suggests that not all cycles are beneficial—specifically, cycles that do not significantly increase the diameter may represent shallow or localized loops that fail to expand the reasoning space meaningfully. \n\nThis observation supports our interpretation in the main paper (e.g., discussion around language mixing and overthinking) that redundant cyclic structures may harm performance, and only cycles that meaningfully contribute to graph expansion (i.e., diameter) are potentially beneficial.\n\nThis highlights the difference between trivial and non-trivial cycles and suggests new directions for reasoning graph analysis.\n\n\n**>W4**\n> Were the examples presented in this figure solved by both models? If not, are all the nodes explored by the reasoning model essential to solving the answer. \n\n> Do you think that overlaying graphs from multiple attempts from a base model would results in similar graph coverage to that of the reasoning model? \n\n\nAs discussed in our response to **W5**, we found that cyclic patterns that do not contribute to increased diameter tend to be less helpful for reaching the correct answer.\n\nYou also raise an interesting hypothesis: whether aggregating reasoning graphs from multiple base model attempts (e.g., pass@k) could resemble the graph produced by a single pass of a reasoning model. While this is a valuable line of inquiry, we emphasize a key distinction:\nIn our setting, the reasoning graph is built from a single, coherent forward pass, where all reasoning steps are connected. We quantified the coverage of the graph by measuring diameter among the connected graph nodes. By contrast, pass@k consists of multiple disjoint generations, which may not form a connected graph under our definition of diameter.\nAs a result, even if the node coverage is similar between the aggregated pass@k graph and the reasoning model's graph, their diameters need not align, since disconnected components are excluded from the diameter computation.\n\nHowever, we think this is a promising research direction, and exploring pass@k phenomena through the lens of graph theory could yield deeper insights into model exploration."}, {"review_id": "bL6hqtkOdq", "reviewer": "Reviewer_8fWp", "summary": "The paper introduces the concept of reasoning graphs to analyze the internal mechanisms of large reasoning models. The paper systematically analyzes key graph-theoretic properties such as cyclicity, diameter, and small-world index across multiple tasks to provide insights into the success of reasoning models. The empirical findings suggest that large reasoning models exhibit unique structural properties in their reasoning graphs, which correlate with enhanced performance. The paper also reviews related works and discusses the implications of these findings for understanding large reasoning models and improving SFT-Data construction.", "questions": "1. Can you provide more details on the construction of reasoning graphs, especially regarding how the nodes are constructed and whether they are biased towards specific large reasoning models?\n2. How do you justify the validity of the reasoning graphs in reflecting the true reasoning process of large reasoning models?\n3. Can you elaborate on the implications for reasoning SFT-Data construction and how the proposed method can be used to directly judge the quality of SFT-Data?", "limitations": "NA", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Paper Strength\n1. Innovative Methodology: The introduction of reasoning graphs to analyze large reasoning models is innovative and provides a new perspective on understanding their internal mechanisms.\n2. Insightful Empirical Findings: The paper offers insightful empirical findings on the structural properties of reasoning graphs in large reasoning models and how they correlate with enhanced performance.\n3. Comprehensive Analysis: The paper systematically analyzes key graph-theoretic properties across multiple tasks, providing a comprehensive understanding of the reasoning processes in large reasoning models.\n\n## Paper Weakness\n1. Lack of Implementation Details: The construction of reasoning graphs needs further clarification. Are the nodes in the graphs constructed separately for each evaluated LLMs? or are they pre-constructed from the outputs of a single LLM such as Qwen2.5-32B-Instruct? If so, would it lead to a bias towards similar large reasoning models, as they tend to generate similar outputs? Besides, when evaluating the reasoning graphs in different layers, would the nodes in the graphs be constructed from the outputs of the same layer of the LLMs? These details are crucial for judging the validity of the findings in the paper.\n\n2. Concerns regarding the reasoning graphs: Are the reasoning graphs really reflecting the true reasoning process of large reasoning models or just showing the statistics for characteristics of the output tokens? Because the reasoning step is broken down by empirically checking the `\\n` and grouping the hidden states of output tokens. Each node in the reasoning graph is a composition of reasoning steps with similar semantics. **Would this really reflect the internal reasoning process of large reasoning models or just a statistical analysis of the output tokens?** For example, the paper mentions that the reasoning graphs exhibit cycles, which could be related to the repetition of the `wait` token in the LLMs outputs. \n\nConsidering the recent works [1] argue that the intermediate tokens in the reasoning process are just some statistically generated tokens and lack any real semantic content or algorithmic meaning (e.g., language mixing in DeepSeek-R1-zero). They could not be treated as reasoning traces of LLMs. This further raises my concerns about the validity of the reasoning graphs in the paper. \n\n3. Implications for Reasoning SFT-Data Construction: The implications for reasoning SFT-Data construction are not well justified. The experiments in Section 5 only show that the model trained with the s1-v1.1 data can generate larger diameter reasoning graphs. This just reflects the reasoning patterns of the finetuned LLMs and is not a direct indication of the quality of the SFT-Data. Can we apply the proposed method to construct the reasoning graph offline and then analyze the graph characteristics to directly judge the quality of the SFT-Data? A naive solution I can think of is to feed the SFT-Data into the forward process of the LLMs in a teaching-forcing manner and extract the hidden states to construct the reasoning graph. By analyzing the cycles and the diameter of the reasoning graph, we can judge the quality of the SFT-Data to see if it is suitable for training the LLMs (i.e., more cycles and a larger diameter are better).\n\n[1] Kambhampati, Subbarao, et al. \"Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces!.\" arXiv preprint arXiv:2504.09762 (2025).", "was_revised": true, "final_justification": "I appreciate the interesting paradigm and findings presented in the paper, which could offer new insights and directions for the community. However, the proposed method relies on unresolved assumptions and appears to lack robustness in handling certain edge cases. Therefore, I maintain my initial score as weak accept.", "rebuttal": "Thank you for your thoughtful and constructive feedback. We carefully addressed each of your questions below and made several clarifications and extensions to strengthen the paper.\n\n**>W1&Q1**\n>Lack of Implementation Details\n\n>Can you provide more details on the construction of reasoning graphs, especially regarding how the nodes are constructed and whether they are biased towards specific large reasoning models?\n\nTo clarify, the nodes in the reasoning graph are constructed independently for **each  model**. Therefore, there is no risk of bias arising from using pre-constructed nodes generated by a single model such as Qwen2.5-32B-Instruct. Each model produces its own reasoning steps, which are then segmented and embedded to form the graph.\n\nSimilarly, when analyzing different layers, we construct segment representations separately for **each layer of each model**, as described in Lines 114–116 of the paper. Thus, the reasoning graph at each layer is based on the outputs from that specific layer.\n\nTo improve clarity, we will revise the relevant sections in the paper to explicitly explain these details. We appreciate your feedback for helping us identify this point of confusion.\n\n**>W2&Q2**\n>Would this really reflect the internal reasoning process of large reasoning models or just a statistical analysis of the output tokens?\n\n>How do you justify the validity of the reasoning graphs in reflecting the true reasoning process of large reasoning models?\n\nTo address this concern about the validity of reasoning graphs, we conducted an automatic labeling experiment using an LLM to examine whether the nodes (centroids) in the reasoning graph correspond to meaningful reasoning behaviors, rather than simply grouping surface-level token patterns.\nSpecifically, we used the `GPT-4o-mini` API to assign semantic themes to each centroid in the reasoning graph of DeepSeek-R1-Distill-Qwen-32B, constructed from its reasoning traces on the GSM8K dataset. For each centroid, we provided the associated reasoning steps and prompted the model with the following instruction:\n```\nYou are a data analyst. The following is an output from a LLM.\nYour task is to carefully read the text and summarize its main theme in 1–3 English words.\n```\n\nThe following table presents the assigned theme for each centroid, along with example reasoning steps that were mapped to that centroid.\n\nThis analysis revealed that centroids often align with meaningful reasoning patterns. In addition to previously reported examples such as Add, Multiply, and Wait, we discovered centroids associated with higher-level computations (e.g., *Calculations Totals*, *Averages, Divisions*), semantics-bound reasoning (e.g., *Age Calculations*, *Cost Calculations*), and structural elements (e.g., *Answer Formatting*, *Placeholder Tags*).\n\nInterestingly, we also identified centroids linked to reasoning behaviors, such as *Planning*, which reflects the model’s initial steps when approaching a math problem.\nMoreover, nodes like `Wait` appear to encompass diverse subtypes. For instance, we found centroids such as *Calculation Correctness* and *Reevaluation*, which reflect the model's tendency to reassess or double-check its own output.\nWe also identified a distinct centroid labeled *Inconsistencies*, which captures moments where the model recognizes contradictions in its own reasoning and attempts to recalculate accordingly.\n\n\n| **Themes** |**Examples**  |\n|----|----|\n| **Calculations Totals (node 83)**     | 5 + 10 is 15. Then, 15 + 9 is 24. Finally, 24 + 3 is 27|\n|                             |Let me add them step by step. 500 + 1500 is 2000. Then, 2000 + 125 is 2125. |\n| **Calculations  Average (node 119)**    | \\text{Average} = \\frac{\\text{Sum of all values}}{\\text{Number of values}}\n|                              |\\text{Average Speed} = \\frac{\\text{Total Distance}}{\\text{Total Time}} = \\frac{250 \\, \\text{miles}}{5 \\, \\text{hours}} = 50 \\, \\text{miles per hour}|\n| **Calculations Division (node 41)**   | \\frac{125,000}{20} = 6,250|\n|                             | \\frac{120 \\, \\text{pieces}}{15 \\, \\text{pieces/pack}} = 8 \\, \\text{packs}|\n|                             | \\frac{80}{10} = 8 \\text{ weeks}|\n| **Age Calculations (node 147)**          | Sum of their ages in two years = (B + 2) + (2B + 2) = 28|\n|                             |Brooke's age = B|\n|                             |C = 2 * (James’s age in 8 years) - 5|\n| **Cost Calculations (node 168)**       | $47.00 × 5 = $235.00|\n|                             |Keenan’s weekly cost = $160 ÷ 4 = $40|\n|                             |\\( 8 \\times 8 = \\$64.00 \\).|\n| **Answers Format (node 137)**        | The answer is: 50|\n|                             |The answer is: 48|\n|                             |The answer is: 8|\n| **Placeholder Tags (node 26)**        |<  /think>|\n|                             |< /think>|\n|                             |< /think>|\n| **Planning (node 163)**          | Okay, so I need to figure out how much Leah has spent on her new kitten so far. Let me break down the problem step by step.|\n|                             | Hmm, let's break it down step by step.|\n| **Calculation Correctness (node 52)** |Wait, that seems straightforward, but let me double-check…|\n|                              |Wait, let me double-check my calculations to make sure…|\n| **Reevaluation (node 100)**            |Wait, maybe I made a mistake in the equations.|\n|                             |Wait, maybe I made a mistake in the equations. Let me try to model it again,…|\n| **Inconsistencies (node 110)**         |Wait, that's a problem. 15 + 8 = 23, which is more than total time 20.|\n|                             |Wait, perhaps the shows are part of the 30%, and the other activities are part of the remaining 70%. But that doesn't make sense…|\n\nThis analysis supports the claim that the reasoning graph provides a meaningful abstraction of the model’s reasoning process, rather than merely capturing surface-level token patterns. The presence of consistent and semantically interpretable themes across centroids—such as arithmetic operations, planning behaviors, and self-correction—indicates that the graph structure reflects underlying reasoning behaviors.\n\nTo improve clarity, we will include the full list of labeled centroid themes and examples in the Appendix of the revised version.\n\n\n**>W3**\n> Can we apply the proposed method to construct the reasoning graph offline and then analyze the graph characteristics to directly judge the quality of the SFT-Data? A naive solution I can think of is to feed the SFT-Data into the forward process of the LLMs in a teaching-forcing manner and extract the hidden states to construct the reasoning graph.\n\nYou are right in pointing out that the original version of our paper focused on analyzing reasoning graphs constructed post hoc—i.e., after training LLMs on different SFT datasets. While this analysis revealed interesting trends, it required training models before assessing the quality of the datasets.\n\nTo improve its practicality for dataset construction with your suggestion, we extended our approach to evaluate reasoning graphs during the forward pass of a frozen model, without any retraining. Specifically, **we input various SFT datasets into a frozen LLM (Qwen2.5-32B-Instruct, the same model used in S1), and constructed reasoning graphs by extracting hidden states** using teacher forcing. This allowed us to evaluate the structural properties of the datasets prior to training.\n\nWe applied this method to two datasets: S1-v1.0, and S1-v1.1, where it is known that models trained on S1-v1.1 achieve better downstream reasoning performance (e.g., higher accuracy on MATH500 and AIME2024). The table below compares the diameters of reasoning graphs induced by these datasets.\n\nWe find that S1-v1.1 consistently yields larger graph diameters, even before any training has occurred, suggesting that reasoning graph properties can serve as a useful signal for evaluation of SFT data quality before training.\n\nThank you again for the valuable suggestion. We will include this result in the revised version as a more direct method for evaluating SFT data quality.\n\n**Diameter** (mean±var)\n| Dataset / Layer Ratio     | 0.1       | 0.3       | 0.5   | 0.7     | 0.9    | MATH500 | AIME2024 | \n|----|----|----|----|----|----|----|----|\n| S1-v1.0           |131.24±5.18        |  360.36 ± 9.407   |   691.68 ± 11.52  | 1057.64±16.36    | 2006.44  ± 20.10   | 92.6% |50.0% |\n| S1-v1.1       | 140.54 ± 8.85         | 392.33 ± 9.32     | 783.55 ± 12.24       | 1100.68±17.07      | 2036.03  ± 21.97      | **94.4%** |  **56.7%**|"}, {"review_id": "ERVDEIfukF", "reviewer": "Reviewer_mySX", "summary": "The paper analyzes the reasoning steps of CoT LLMs as graph, and find interesting correlations between graph properties and the reasoning capability of the LLMs.", "questions": "None", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 1, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strength:\n1. This is a interesting perspective to view each type of reasoning steps as node and analyze the graph patterns between reasoning node.   The graph properties shows strong correlation to the reasoning capability of the Qwen model family.\n\nWeakness:\n1. The paper is a bit hard to read with some grammar issues. For example in the first paragraph \"These recent reasoning models are characterized with to think and reason for longer before responding\". Also the choice of math notations are also not helping the readers to better understand the paper. For example, the author used \"T\" to define the number of reasoning steps, then the readers will naturally expect  to use \"t\" to index the step, but then the author used \"t\" to index the token within each step. Same for \"l\" and \"L\". In general I find much can be improved in the writing.\n2. For the clustering of segments, there are many ad hoc choices that are not explained well. For example, the choice of K=200. Also the authors only showed 3 representative centroids among the 200. Can the authors add the others in the Appendix? I am curious if the other centroid make sense.\n3. It seems the experimental results are restricted to the Qwen family of models. The results will be stronger if more model families can be included.", "was_revised": true, "final_justification": "The authors responded to my concerns and the paper does have some novelty in its interesting perspective to view the reasoning LLM as a graph.", "rebuttal": "We sincerely thank the reviewers for their thoughtful and constructive feedback. We conducted additional analyses and clarifications to address your concerns, as detailed below.\n\n**>W1**\n>The paper is a bit hard to read with some grammar issues. For example in the first paragraph \"These recent reasoning models are characterized with to think and reason for longer before responding\". Also the choice of math notations are also not helping the readers to better understand the paper. For example, the author used \"T\" to define the number of reasoning steps, then the readers will naturally expect to use \"t\" to index the step, but then the author used \"t\" to index the token within each step. Same for \"l\" and \"L\". In general I find much can be improved in the writing.\n\nThank you for pointing this out. Regarding the sentence     \n`These recent reasoning models are characterized with to think and reason for longer before responding`    \nwe have corrected it to:     \n`These recent reasoning models are characterized by their ability to think and reason for longer before responding`\n\nAs for the notations, we agree that the original usage was potentially confusing. To improve clarity:\n- We now use $t$ to index the reasoning step,\n- and use $m$ to index the token within each step.\n- The layer index previously denoted by $\\ell$ is now replaced by $\\mu$.\n\nIf there are any other specific points that you found difficult to follow or unclear, we would greatly appreciate further feedback so we can improve the paper’s readability.\n\n**>W2**\n> For the clustering of segments, there are many ad hoc choices that are not explained well. For example, the choice of K=200. Also the authors only showed 3 representative centroids among the 200. Can the authors add the others in the Appendix? I am curious if the other centroid make sense.\n\n\nAs noted in lines 181–183 of the paper, we already included an analysis of different values of $k$ in **Appendix D**. Specifically, we tested $k = 50, 100$, and $200$, and found that the trends discussed in the main paper were consistent across these $k$-values. We invite you to refer to **Appendix D** for full details.\n\nFurthermore, to address your concern about the other centroids when $k = 200$, we conducted an automatic labeling experiment using an LLM to explore what kinds of reasoning patterns these centroids represent. Specifically, we used the `GPT-4o-mini` API to assign a theme to each centroid, based on the reasoning steps associated with it in the reasoning graph of DeepSeek-R1-Distill-Qwen-32B on the GSM8K dataset.\nWe provided the following system prompt to GPT, and then input multiple reasoning steps corresponding to each centroid as the user prompt:\n```\nYou are a data analyst. The following is an output from a LLM.\nYour task is to carefully read the text and summarize its main theme in 1–3 English words.\n```\n\nThe following table presents the assigned **theme** for each centroid, along with **example** reasoning steps that were mapped to that centroid.\n\n| **Themes** |**Examples**  |\n|----|----|\n| **Calculations Totals (node 83)**     | 5 + 10 is 15. Then, 15 + 9 is 24. Finally, 24 + 3 is 27|\n|                             |Let me add them step by step. 500 + 1500 is 2000. Then, 2000 + 125 is 2125. |\n| **Calculations  Average (node 119)**    | \\text{Average} = \\frac{\\text{Sum of all values}}{\\text{Number of values}}\n|                              |\\text{Average Speed} = \\frac{\\text{Total Distance}}{\\text{Total Time}} = \\frac{250 \\, \\text{miles}}{5 \\, \\text{hours}} = 50 \\, \\text{miles per hour}|\n| **Calculations Division (node 41)**   | \\frac{125,000}{20} = 6,250|\n|                             | \\frac{120 \\, \\text{pieces}}{15 \\, \\text{pieces/pack}} = 8 \\, \\text{packs}|\n|                             | \\frac{80}{10} = 8 \\text{ weeks}|\n| **Age Calculations (node 147)**          | Sum of their ages in two years = (B + 2) + (2B + 2) = 28|\n|                             |Brooke's age = B|\n|                             |C = 2 * (James’s age in 8 years) - 5|\n| **Cost Calculations (node 168)**       | $47.00 × 5 = $235.00|\n|                             |Keenan’s weekly cost = $160 ÷ 4 = $40|\n|                             |\\( 8 \\times 8 = \\$64.00 \\).|\n| **Answers Format (node 137)**        | The answer is: 50|\n|                             |The answer is: 48|\n|                             |The answer is: 8|\n| **Placeholder Tags (node 26)**        |<  /think>|\n|                             |< /think>|\n|                             |< /think>|\n| **Planning (node 163)**          | Okay, so I need to figure out how much Leah has spent on her new kitten so far. Let me break down the problem step by step.|\n|                             | Hmm, let's break it down step by step.|\n| **Calculation Correctness (node 52)** |Wait, that seems straightforward, but let me double-check…|\n|                              |Wait, let me double-check my calculations to make sure…|\n| **Reevaluation (node 100)**            |Wait, maybe I made a mistake in the equations.|\n|                             |Wait, maybe I made a mistake in the equations. Let me try to model it again,…|\n| **Inconsistencies (node 110)**         |Wait, that's a problem. 15 + 8 = 23, which is more than total time 20.|\n|                             |Wait, perhaps the shows are part of the 30%, and the other activities are part of the remaining 70%. But that doesn't make sense…|\n\nThis analysis revealed that centroids often align with meaningful reasoning patterns. In addition to previously reported examples such as *Add*, *Multiply*, and *Wait*, we discovered centroids associated with higher-level computations (e.g., *Calculations Totals*, *Averages*), semantics-bound reasoning (e.g., *Age Calculations*, *Cost Calculations*), and structural elements (e.g., *Answer Formatting*, *Placeholder Tags*).\n\nInterestingly, we also identified centroids linked to reasoning behaviors, such as *Planning*, which reflects the model’s initial steps when approaching a math problem.\nMoreover, nodes like `Wait` appear to encompass diverse subtypes. For instance, we found centroids such as *Calculation Correctness* and *Reevaluation*, which reflects the model's tendency to reassess or double-check its own output.\nWe also identified a distinct centroid labeled *Inconsistencies*, which captures moments where the model recognizes contradictions in its own reasoning and attempts to recalculate accordingly.\n\n**We believe that our additional analysis effectively addresses your concern regarding whether each centroid makes sense.**\n\nTo improve clarity and better support the validity of our analysis, we will include the complete list of centroid themes with illustrative examples in the revised Appendix.\n\n\n**>W3**\n> It seems the experimental results are restricted to the Qwen family of models. The results will be stronger if more model families can be included.\n\nRecent studies on reasoning [1,2] have focused on the Qwen family. In line with this trend, we chose Qwen models for our main experiments due to their widespread use in recent reasoning research.\n\nTo further test the generality of our findings, and following your suggestion, we conducted the same reasoning graph analysis using models from the LLaMA family. Specifically, we compared **LLaMA-3.1-8B (base model)** and **DeepSeek-R1-Distill-LLaMA-8B (reasoning model)** on the GSM8K dataset.\n\nThe tables below present results for cycle ratio, graph diameter, and small world index. As with the Qwen models, we observe that the reasoning model exhibits more cycles, larger diameters, and larger small-worldness than the base model across all layer ratios. **This consistency supports the generality of our findings in another series of models.**\n\nWe will include these results in the revised version of the paper. We will be happy to address if the reviewer has further suggestions on open-source reasoning and base model pairs, which need to be included.\n\n---\n**Cycles**\n| Model / Layer Ratio                      | 0.1  | 0.3  | 0.5  | 0.7  | 0.9  |\n|----|----|----|----|----|----|\n| DeepSeek-R1-Distill-Llama-8B | 0.92 | 0.92 | 0.90 | 0.92 | 0.94 |\n| Llama-3.1-8B                  | 0.51 | 0.61 | 0.71 | 0.69 | 0.70 |\n\n---\n**Diameter** (mean±var)\n| Model / Layer Ratio                          | 0.1        | 0.3       | 0.5   | 0.7 | 0.9  | \n|----|----|----|----|----|----|\n| DeepSeek-R1-Distill-Llama-8B   | 5.90 ± 1.37           | 26.68 ± 2.76          | 57.77 ± 4.31          | 105.87 ± 6.11         | 153.80 ± 7.56         | \n| Llama-3.1-8B                   | 1.67 ± 1.05           | 11.10 ± 2.29          | 19.35 ± 2.97          | 33.07 ± 3.93          | 46.80 ± 4.66          | \n\n---\n**Small world index**\n| Model / Layer Ratio                              | 0.1    | 0.3  | 0.5    | 0.7    | 0.9     |\n|---------|----------|---------|----------|-----------|--------|\n| DeepSeek-R1-Distill-Llama-8B       | 0.70        | 0.73       | 0.72         | 0.73        | 0.78     |\n| Llama-3.1-8B                        | 0.24  | 0.36        | 0.44        | 0.45    | 0.47       |\n\n[1] Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?    \n[2] The Invisible Leash: Why RLVR May Not Escape Its Origin"}, {"review_id": "XJ1cBhrFFH", "reviewer": "Reviewer_EMKy", "summary": "This paper introduces reasoning graphs, which are constructed by clustering hidden states of LLMs during reasoning tasks. It analyzes their graph-theoretic properties (cyclicity, diameter, small-world index), demonstrating that reasoning models exhibit significantly more cycles, larger exploration diameters, and stronger small-world characteristics compared to base models. The work provides new insights into the structural mechanisms behind advanced reasoning capabilities and practical guidance for data design.", "questions": "1. The authors should conduct additional experiments beyond mathematical reasoning datasets to demonstrate the broader applicability of their findings. (W1)\n2. The authors must resolve the issues stated in W2-W4.\n3. The authors should explain why they chose the hidden states from the layer of 90% depth as the default rather than the last layer. The hidden states of the final layer are used to generate responses, which seems significantly more reasonable.\n4. The authors could demonstrate the statistics (e.g., average number of nodes and edges) of the reasoning graphs for each dataset to help readers better understand the changes in graph properties.", "limitations": "yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n1. This paper is well-organized and easy to follow.\n2. The idea of extracting reasoning graphs from LLM and exploring their properties is interesting, offering a new perspective for understanding the model's reasoning mechanisms\n\n**Weaknesses:**\n1. The conclusions of this paper are only validated by three math reasoning datasets, leaving their generalization to a broader domain (such as logical reasoning, code reasoning, and general tasks) uncertain.\n2. The paper emphasizes that the proposed method can guide the design of training data to enhance reasoning capabilities, but it fails to explicitly specify how to construct such data in practice. Moreover, this approach resembles a post-hoc analysis that incurs prohibitive computational costs—since it requires inference across multiple datasets before yielding actionable insights—severely limiting its real-world applicability.\n3.  I am not convinced that the approach proposed in the paper—selecting better datasets solely based on the diameter of the reasoning graph—is practically effective. Taking the same scenario as presented in the [S1 paper](https://arxiv.org/abs/2501.19393) as an example: when a large amount of supervised data has been collected, how can the authors' method be applied to select samples for constructing an optimal dataset?\n4. The logic underlying the paper’s proposed method for constructing SFT data is flawed. The paper only empirically shows that SFT data with more cycles and larger diameters in the reasoning graph tend to yield better performance. However, the reverse does not necessarily hold: SFT data with more cycles and larger diameters do not inherently lead to better performance. Intuitively, data that reflects overthinking or unnecessarily long reasoning paths may result in graphs with more cycles and larger diameters, but such data does not guarantee improved outcomes. As noted in the S1 paper, selecting data with the longest reasoning traces did not lead to optimal performance.\n5. The attribution of redundant cycles to overthinking (Lines 296-298) conflicts with the empirical analysis in Section 4.4, which conclusively traces their origin to language-mixing effects.\n6. The paper has many minor flaws that may affect the reader's impression:\n    * There is a mismatch between the question and the corresponding response in Figure 1, that is, the generated answers do not address the displayed question.\n    * There is a mixed use of notations, such as $K$ in lines 118, 152, and 182, which may lead to reader confusion/misinterpretation.\n    * The quality of Figure 3 may hinder proper interpretation, as it is not provided in vector format, resulting in potential pixelation or blurring when scaled.\n    * There is an inconsistency between the textual description (Line 641) and Figure 14's caption: The text states the model was fine-tuned using \"100 and 500 steps\". However, Figure 14's sublabels show results for \"200 steps\" and \"400 steps\".\n    * There appears to be an issue with invalid citations in Lines 856 and 886, where the references are displayed as question marks (\"?\").", "was_revised": true, "final_justification": "The authors have addressed some of my concerns; however, the issue regarding the practical usability of the proposed method has not been sufficiently resolved. Therefore, I will maintain my score.", "rebuttal": "We sincerely thank the reviewer for their thorough and constructive feedback, which significantly helped us strengthen both the empirical scope and clarity of our paper.\n\n**>W1&Q1**\n> The conclusions of this paper are only validated by three math reasoning datasets\n\nTo address your concern, we have extended our evaluation to include two additional tasks beyond math, inspired by prior work such as [1]: (i) **StrategyQA**, a multi-hop question answering dataset, and (ii) **LogicalDeduction**, a logical reasoning dataset from BIG-Bench.\n\nBelow, we report the comparison of reasoning graph properties between DeepSeek-R1-Distill-Qwen-32B (reasoning model) and Qwen2.5-32B (base model) on these datasets. \nAs with the results reported on math tasks in the main paper, the reasoning model exhibits markedly different graph characteristics, such as significantly higher cycle rates, larger diameters, and stronger small-world properties.\n\nThese consistent patterns across several domains provide further evidence for the generality of our findings. We appreciate the reviewer’s suggestion, which helped us strengthen the empirical validation of our claims.\n\n---\n**(i) StrategyQA**\n\n---\n**Cycles**\n| Model / Layer Ratio | 0.1| 0.3| 0.5| 0.7| 0.9| \n|---|---|---|---|---|---|\n|Distill-Qwen-32B| 0.89| 0.89| 0.86| 0.84| 0.88|\n|Qwen2.5-32B|0.07|0.07|0.04|0.06|0.06|\n\n---\n**Diameter** (mean±var) \n|Model / Layer Ratio|0.1|0.3|0.5|0.7|0.9| \n|---|---|---|---|---|---|\n|Distill-Qwen-32B|517.60 ± 14.56|1161.89 ± 21.35|1627.66 ± 24.71|2110.36 ± 28.09|5037.20 ± 43.71|\n|Qwen2.5-32B|114.58 ± 8.91|256.53 ± 13.12|335.62 ± 15.06|385.42 ± 16.26|1023.93 ± 25.95|\n\n---\n**Small world index**\n|Model / Layer Ratio|0.1|0.3|0.5|0.7|0.9| \n|---|---|---|---|---|---|\n|Distill-Qwen-32B|0.94|0.97|0.95|0.91|0.94 | \n|Qwen2.5-32B|0.02|0.02|0.02| 0.01|0.01|| \n\n---\n**(ii) LogicalDeduction**\n\n---\n**Cycle**\n|Model / Layer Ratio|0.1|0.3|0.5|0.7|0.9| \n|---|---|---|---|---|---|\n|Distill-Qwen-32B|1.00|0.99|0.97|0.99|0.99|\n|Qwen2.5-32B|0.07|0.07| 0.04|0.06|0.06|\n\n---\n**Diameter** (mean±var)\n| Model / Layer Ratio|0.1|0.3|0.5|0.7|0.9| \n|---|---|---|---|---|---|\n|Distill-Qwen-32B|630.10 ± 13.07| 1608.34 ± 21.33 | 2313.50 ± 25.47 | 3063.10 ± 30.35 | 6815.19 ± 43.48 | \n|Qwen2.5-32B | 210.75 ± 11.69 | 469.29 ± 17.34| 629.74 ± 20.05| 755.24 ± 22.08| 1719.19 ± 32.05|\n\n---\n**Small world index**\n| Model / Layer Ratio|0.1|0.3| 0.5|0.7|0.9| \n|---|---|---|---|---|---|\n| Distill-Qwen-32B|0.96|0.88| 0.78| 0.74| 0.92| \n| Qwen2.5-32B |0.12|0.11|0.07|0.06|0.08| \n\n[1] Understanding the Reasoning Ability of Language Models From the Perspective of Reasoning Paths Aggregation, ICML'24\n\n**>W2&W3&Q2**\n> it fails to explicitly specify how to construct such data in practice. \n\n> when a large amount of supervised data has been collected, how can the authors' method be applied to select samples for constructing an optimal dataset?\n \nTo directly address this concern, and following Reviewer 8fWp’s suggestion, we extended our approach to analyze the reasoning graph during the forward pass of a frozen LLM (Qwen2.5-32B-Instruct, the same model used in S1). Specifically, **we fed various SFT datasets into the model and measured the resulting reasoning graph diameters** without requiring any training.\n\nIn this experiment, we used two versions of the S1 dataset: S1-v1.0 and S1-v1.1, where it is known that models trained on S1-v1.1 achieve better downstream reasoning performance. The table below compares the diameters of reasoning graphs induced by these datasets. \nWe find that S1-v1.1 consistently yields larger graph diameters, even before any training has occurred, suggesting that the diameter of the reasoning graph can serve as a useful signal for evaluation of SFT data quality before any model training. \n\nThis provides a concrete answer to your question: in scenarios like S1, where a large amount of supervised data is available, our method allows one to feed candidate subsets into the target model, construct the corresponding reasoning graphs, and use their properties (e.g., diameter) to guide data selection before training. This analysis does not require prohibitive computational costs due to finetuning as needed in the original paper.\n\nWe will include this result in the revised version of the paper to clarify the practical applicability of our method to SFT data selection.\n\n**Diameter** (mean±var)\n| Dataset / Layer Ratio|0.1|0.3|0.5| 0.7 | 0.9 | MATH500 | AIME2024 | \n|---|---|---|---|---|---|---|---|\n| S1-v1.0|131.24±5.18| 360.36 ± 9.40 | 691.68 ± 11.52 | 1057.64±16.36 | 2006.44  ± 20.10| 92.6% |50.0% |\n| S1-v1.1| 140.54 ± 8.85| 392.33 ± 9.32| 783.55 ± 12.24 | 1100.68±17.07| 2036.03  ± 21.97|**94.4**% |**56.7**%|\n\n**>W4**\n> the reverse does not necessarily hold: SFT data with more cycles and larger diameters do not inherently lead to better performance.\n\nWe agree with you that datasets with more cycles or larger graph diameters may not inherently guarantee better model performance. Our intention was not to suggest that **solely** optimizing for graph properties such as diameter will lead to better results.\n\nRather, we position graph metrics such as diameter as auxiliary signals for data selection, not as standalone objectives. In practice, high-quality data pools should first be constructed using strong reasoning models (e.g., Gemini or DeepSeek-R1), as in the S1 pipeline. Once such data is available, reasoning graph metrics can serve as informative heuristics to filter or prioritize samples that are more likely to support effective reasoning. These metrics are meant to complement, not replace, other quality criteria.\n\nWe explicitly do not advocate for selecting data solely to maximize graph diameter, as this could promote undesirable behaviors like overthinking. Instead, we propose reasoning graph analysis as a diagnostic tool, complementary to other strategies, to inform data selection.\n\nTo make this intended usage clearer, we will revise the relevant sections in the paper.\n\n**>W5**\n> The attribution of redundant cycles to overthinking (Lines 296-298) conflicts with the empirical analysis in Section 4.4, which conclusively traces their origin to language-mixing effects.\n\nIn Section 4.4, we presented language mixing as one concrete example that may lead to redundant cycles. Our mention of overthinking was not meant to contradict this, but rather to discuss another pattern of redundant cycles observed in prior work.\n\nDuring a rebuttal, we newly observed a failed output from Distill-Qwen-32B on GSM8K, which involved a simple arithmetic problem. Despite the simplicity, the model showed repetitive `Wait` outputs and trial-and-error loops, similar to patterns noted in [2]. This type of redundant cycle differs from the language-mixing discussed in Section 4.4 of the main paper.\n\n```\n…\nThey have 78 left, so S + 59 = 78 → S = 19.  \nSame result. So, perhaps the problem is that the initial stock is 19, but they used 38 on Day 1, ...  \nWait, maybe the problem is that the orders are received before ...  \nWait, that's what I did initially. So, perhaps the problem is ....  \nWait, maybe I made a mistake in the equations. Let me try to model it again, ...  \nWait, perhaps the problem is that the orders are received before the usage on the...  \nWait, let me try that.  \n…\n```\n\nTo avoid confusion, we will revise the relevant sentence to clarify that language mixing is one identified cause of redundant cycles, while overthinking represents another distinct pattern.\n\n[2] Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs.\n\n**>W6**\n>The paper has many minor flaws that may affect the reader's impression:\n\nThank you for pointing out the minor issues related to notation, citations, and descriptions. We appreciate your careful reading.\n\nWe will thoroughly review and revise these details in the final version to improve clarity and presentation.\n\n**>Q3**\n>The authors should explain why they chose the hidden states from the layer of 90% depth as the default rather than the last layer. \n\nAs there is no established consensus on which layer of an LLM best reflects reasoning-related representations, we conducted a comprehensive analysis using hidden states from five different depths: 10%, 30%, 50%, 70%, and 90%. The figures in the paper report results from all these layers. Among them, the 90% layer consistently showed the clearest trends, which is why we used it as the default in our main analysis.\n\nFollowing your suggestion, we also analyzed cycles using the hidden states from the final (**100%**) layer. As shown in the table below, the reasoning model (Distill-Qwen-32B) consistently exhibits more cycles than the base model (Qwen2.5-32B), and the final layer also follows the same trends. This result suggests that our analysis holds even at the final layer, supporting the robustness of our findings across different depths.\n\n**Cycles**\n| Model / Layer Ratio | 0.1  | 0.3  | 0.5  | 0.7  | 0.9  | 1.0  |\n|---|---|---|---|---|---|---|\n| Distill-Qwen-32B | 0.80 | 0.73 | 0.57 | 0.50 | 0.61 | **0.83** |\n| Qwen2.5-32B                   | 0.18 | 0.15 | 0.11 | 0.07 | 0.13 | **0.18** |\n\n**>Q4**\n>The authors could demonstrate the statistics (e.g., average number of nodes and edges) \n\nTo address your question, we report below the average number of nodes and edges in the reasoning graphs generated for each dataset: GSM8K, MATH500, and AIME2024.\nThe model used was Distill-Qwen-32B, with a target layer ratio of 0.9 and K = 200 for K-means clustering. As shown below, the number of nodes increases as task difficulty increases, reflecting the expansion of the reasoning search space.\nFor a more detailed interpretation of what each node represents in the reasoning process, please refer to our response to **Reviewer mySX, W2**.\nWe will include this table in the Appendix of the revised paper to improve the clarity.\n\n|Dataset| Avg. # Nodes | Avg. # Edges |\n|---|---|---|\n| GSM8K|18.66| 27.80|\n| MATH500 | 23.19| 60.44|\n| AIME2024 | 33.60 | 183.90 |"}]

Author rebuttal:

Meta-review: This paper offers a novel contribution on how large language models reason by leveraging reasoning graphs. Its extensive empirical evaluation and methodology are strengths that have been praised by reviewers. However, concerns about whether the graph metrics merely reflect generic properties (such as sequence length) rather than true reasoning, along with questions about the generalizability and practical applicability of the findings, have been raised in the reviews as well as some clarity and presentation issues. The authors (props to them!) managed to address the most important points imho.

Overall the aggregated reviews lean toward acceptance with some reviewers giving accept while others a borderline accept due to the  questions above. I think the paper is ready to be accepted.
Decision: Accept (poster)