Style over Substance: Distilled Language Models Reason Via Stylistic Replication
Authors: Philip Lippmann
Jie Yang
Venue: COLM 2025 (2025)
Keywords: Reasoning
language models
stylistic mimicry
pivots
synthetic data
distillation
finetuning
metacognition
TL;DR: We show that language models distilled from reasoning models primarily mimic stylistic patterns rather than internalize deeper reasoning capabilities.

Abstract: Specialized reasoning language models (RLMs) have demonstrated that scaling test-time computation through detailed reasoning traces significantly enhances performance.Although these traces effectively facilitate knowledge distillation into smaller, instruction-tuned models, the precise nature of transferred reasoning remains unclear. In this study, we investigate to what extent distilled models internalize replicated stylistic patterns during reasoning. To this end, we systematically analyze reasoning traces, identifying structural and lexical patterns that characterize successful reasoning. We then introduce two new datasets -- a dataset of emergent reasoning traces and a synthetic dataset explicitly constructed to replicate these stylistic patterns -- to precisely examine their influence on distilled models' reasoning capabilities. We find that models trained on the synthetic traces achieve comparable performance, indicating that distilled reasoning abilities rely significantly on surface-level patterns. Surprisingly, we observe an increase in performance even when the synthetic traces are altered to lead to the wrong answer. Our findings highlight how stylistic patterns can be leveraged to efficiently enhance LM reasoning across diverse model families.

Published as a conference paper at COLM 2025
Style over Substance: Distilled Language Models Reason Via Stylistic Replication
Philip Lippmann Delft University of Technology
Jie Yang
Delft University of Technology
Abstract
Specialized reasoning language models (RLMs) have demonstrated that scaling test-time computation through detailed reasoning traces signifi- cantly enhances performance. Although these traces effectively facilitate knowledge distillation into smaller, instruction-tunet models, the pre- cise nature of transferred reasoning remains unclear. In this study, investigate to what extent distilled models internalize replicated stylistic patterns during reasoning. To this end, soning traces, identifying structural and lexical patterns that characterize successful reasoning. We then introduce two new datasets a dataset of emergent reasoning traces and a synthetic dataset explicitly constructed to replicate these stylistic patterns to precisely examine their influence on distilled models' reasoning capabilities. We find that models trained on the synthetic traces achieve comparable performance, indicating that distilled reasoning abilities rely significantly surface-level patterns. Surprisingly, we observe an increase in performance even when the synthetic traces are altered to lead to the wrong answer. Our findings highlight how stylistic patterns can be leveraged enhance LM reasoning across model families.
systematically analyze rea-
1 Introduction
Reasoning is fundamental to artificial intelligence, enabling systems to solve problems, make decisions, and explain outcomes. While traditional approaches to improving lan- guage model (LM) reasoning emphasize increased train-time compute (Kaplan et al., 2020; Hoffmann et al., 2022), recent research highlights that scaling test-time compute through self-refinement is similarly effective (Snell et al., 2024). This insight has inspired specialized reasoning-focuu focused LMs (RLMs), such as ol (OpenAI, 2024) and R1 (DeepSeek-A) et al., 2025), which generate detailed reasoning traces of their thought process during inference.
Reasoning traces have proven effective for distillation (Schmidhuber, 1992; Hinton et al., 2015), efficiently transferring sophisticated cognitive skills from RLMs to smaller, instruction- tuned models (Team, 2025). Yet, the precise na nature of the reasoning knowledge transferred remains poorly understood (Allen-Zhu & Li, 2023). In particular, it is unclear whether distilled models genuinely internalize complex reasoning abilities or replicate superficial stylistic patterns from the original traces. This ambiguity echoes broader Al debates about whether language models genuinely understand content or simply engage in surface-level imitation (Bender & Koller, 2020; Bender et al., 2021; Shanahan, 2024; Mirzadeh et al., 2025). This uncertainty leads to a fundamental question: are we genuinely enhancing the reasoning capabilities of models, or merely teaching them to mimic benchmarks? Motivated by this ambiguity, aim to
-ssecffcc patterns that
happen to improve performance establish whether style is key to improvements in reasoning, where style is characterized primarily by structural attributes such as trace length, lexical coherence, and backtracking frequency, rather than comprehension itself.
To examine this, we first systematically analyze successful reasoning traces produced by state-of-the ofttt--rrttrt RLMs, identifying recurrent structural and lexical patterns. This analysis, guided by cognitive science frameworks that characterize critical stages in human problem-

---

Published as a conference paper at COLM 2025
Style drive improvements reasoning
Distiling of reasoning from teacher model via SFT on different datasets
ST. Emergent reasening traces with connect answers
ST+C: Synthelic reasoning traces set with contect erscomss
THRCRe aut with incorrect answers
ST+C-W, W: Synthetic reasering
Qusidion: many posi integers lasa 10,000 two difanent dight??
Answer: 8999
Okey,, as need figure
thes prbbees frwed
coent the nember climagam Walk, throug whse
problen
Lislased somathing sauumporre verty formul Mou how connecails
Incorrect answer prior finetuning
Now, numbarr rumber...
Absmmativeyy Thanaforr, sumbaas t1 too
wap this uainng Pusting together
Answer: 927
Answer: 927
Answer: R27
Figure 1: Reasoning trace style has noticeable influence on distilled model performance. We show how different types of reasoning traces emergent traces with correct answers, synthetic traces with correct answers, and synthetic traces with incorrect answers - affect model performance after finetuning. All three approaches give the right answer due to improved reasoning capabilities compared to base models. Pivots highlighted in bold.
solving (Newell & Simon, 1972), reveals that effective reasoning traces consistently exhibit distinct metacognitive behaviors. These behaviors are often signaled by lexical pivots markers such as "Wait" or "What if" - thaa prompt reconsideration of assumptions or integration of new insights.
Based on these findings, introduce two complementary datasets explicitly designed to clarify the role of style in improving reasoning. The first dataset, SMOLTRACES (ST), comprises verified question-answer pairs with sophisticated reasoning traces generated by a state-of-the-art RLM, displaying naturally emergent reasoning behaviors. The second dataset, SMOLTRACEE ACRD--DDDD (ST-HC), is synthetically constructed by embedding only the structural and lexical stylistic patterns identified earlier into reasoning traces generated by standard LM without specialized reasoning capabilities. By comparing models trained on these datasets through supervised fine-tuning (SFT) (Brown et al., 2020), we evaluate how stylistic consistency influences reasoning performance. Our experiments demonstrate that even stylistically consistent synthetic traces from weaker model achieve comparable downstream reasoning performance, underscoring the importance style as critical factor in model training.
Further experiments contextualize these findings through targeted ablation studies, which examine the relative importance of trace correctness versus style. Remarkably, we find that stylistically consistent reasoning traces even those explicitly designed to lead to incorrect conclusions still substantially enhance downstream reasoning performance over the base model, as shown in figure 1. These results underscore that stylistic consistency significantly influences LM reasoning capabilities, providing explanation for the effectiveness of reasoning distillation from RLMs to regular LMs.
In summary, our work makes two major contributions: (1) we demonstrate that distilled reasoning improvements rely heavily on stylistic patterns present in reasoning traces; and (2) we identify specific structural and lexical features indicative of successful reasoning. Collectively, these contributions deepen our understanding of how language models per- form reasoning tasks, and the datasets we release can serve as valuable resources for future research into synthetic data generation and fine-tuning methodologies that explicitly target the relationship between reasoning trace style and substance.
Background
Chain-of-Thought Early approaches to elicit reasoning from LMs, such as Chain-of-Thought (CoT) (Wei et al., 2023), demonstrate that intermediate reasoning steps key for LMs to
2

---

Published as a conference paper at COLM 2025
sipplly loskel
Simpllyy square roct
Glay
simplify 9471442)
prime foctortation
by recalling what means il equare
Problem Framing
icunily any parfect equares that can
Reetiing and narking
Plaat: anouus check has any parfect aquare fuctors
fectors Wiel, dixcted by
So 242.. let's see, cvon, 30 It's dissabe o by That gives mo
Exploration
242 2 121.
Wait. 121 as par fedd equare becasse 11 time 1e 121, Sa 242
Pipedruss poia pata coraldand
Now 12ih--11 1912, that' pertect sqparna So have 242 = times 1112.
con be factooed into 2 times 121. Therefone, 5on(242) con within no sopt?? 1213
Now con montll the
aprig42) eeptp 1052 1ar221 11)30 acrip)"
double-check make other 242 11, 121,
other
(F ner
Synthesls
Lot me double chock tie If equare 11 sonta should get 242 So (11 suptt2//2 1112 121 592. Yea thata connct
Inaggading nages ton Pioo ccherent colution
bager sinwedy there perfect equares consider
Actually shou will this the standard form where put comeliitent front redical So simppidteed form
Thaewhow, the samppitee and form t 11
11
Emergent
Synthetic
Comparison of emergent and synthetic reasoning traces for solving the same problem. The left side shows reasoning trace generated by an RLM, while the right side displays a synthetic trace created using our hard-coded template with predefined pivots. Both approaches follow similar cognitive stages (center): problem framing, exploration, ver- ification, and synthesis. The dashed boxes highlight examples of each stage, demonstrating that synthetic traces can effectively replicate the style of emergent reasoning.
Figure 2:
improve their problem-solving abilities (Zhang et al., 2022). While CoT improves perfor- mance on reasoning tasks, it primarily focuses on generating linear sequence of steps towards a solution. On the other hand, RLM reasoning traces (sometimes called "long CoT") whicc are the focus of this paper - differ from CoT as they do not just try to build toward the solution linearly, but instead actively backtrack, verify, and explore different lines of thinking.
Reasoning traces Reasoning traces are semi-structured textual representations that capture a model's thought process while working toward the solution of a problem during inference. Here, additional tokens are generated before the final answer is given to reason about the problem. These traces typically include explicit markers of metacognition, such as planning statements, hypothesis testing, and self-correction. For an example of a full reasoning trace, see appendix A. A key characteristic of effective reasoning traces is the presence of pivots, points where the model explicitly moves between different categories of metacognition. We delve into the specific types of pivots in section 3.1.
Finetuning on reasoning traces Recent research has demonstrated that finetuning language models on reasoning traces significantly enhances their reasoning capabilities (Huang et al., 2024a; Team, 2025; DeepSeek-Al et al., 2025). In this approach, detailed reasoning traces generated by RLMs are used as training data to transfer structured reasoning behaviors into smaller, instruction-tun tuned models via SFT (Min et al., 2024). Models finetuned these traces consistently outperform those trained only on final answers or simpler step-by-step solutions, suggesting that the explicit structure and content of reasoning traces play critical role in improving model performance (Xu et al., 2025; Labs, 2025). Despite these successes, the precise mechanisms underlying the effectiveness of reasoning trace distillation remain unclear, particularly regarding the balance between structural stylistic cues and the cognitive complexity of the reasoning itself.
3 Hard-coding reasoning traces to approximate emergent ones
In examining whether distilled models internalize genuine reasoning capabilities or pri- marily benefit from structural and lexical patterns, we investigate the relationship between style and substance in reasoning traces. Here, style encompasses the structural and lexical features while substance refers to factual correctness and semantic content. To examine stylistic influence, we develop a methodology that replicates the structural patterns of successful reasoning while varying content, allowing to assess style's ss contribution to
3

---

Published as a conference paper at COLM 2025
reasoning distillation. We hypothesize that emergent reasoning behaviors can be effec- tively approximated by encoding the metacognitive pivots characteristic of RLM traces into synthetic reasoning traces. We define emergent traces as those naturally produced by RLMs after training via reinforcement learning, while synthetic traces refer to our approxima- tions that incorporate the stylistic elements we identify key, but originate from standard LMs without specialized reasoning capabilities. This approach reduces reliance on costly RLM inference' while enabling us to control stylistic components that potentially drive performance improvements in distilled models.
3.1 Reasoning trace structure and pivot types
Trace analysis Effective reasoning traces exhibit systematic structural patterns reflective of human-like problem-solving strategies. Cognitive science literature characterizes human reasoning as structured process comprising distinct stages: problem framing, hypothesis exploration, verification, and synthesis (Newell & Simon, 1972). Guided by this framework, we systematically analyze 17K successful reasoning traces produced by a state-of-the-art RLM (see appendix B for details) and find that effective RLM-generated reasoning traces consistently align with these cognitive stages. Specifically, reasoning traces begin with explicit problem framing, restating and clarifying key aspects of the problem leading to plan, followed by an exploration stage in which hypotheses or potential solution paths are considered. This is complemented by verification stage, characterized by explicit checking of hypotheses or intermediate results, culminating in a final synthesis, integrating insights from earlier stages into coherent solution.
Pivot types Crucially, effective reasoning traces are rarely linear; instead, they frequently re- visit previous stages to correct errors, validate assumptions, or explore alternative strategies. This non-linear metacognitive behavior is operationalized through pivots - lexical markers signaling explicit shifts between reasoning stages. Our analysis identifies four primary pivot categories, each aligning closely with a corresponding reasoning stage: (1) Realization pivots, such as "Wait" or "Oh," signal recognition of errors or oversights during the exploration stage; (2) Verification pivots, initiated by phrases such as "Let me check," explicitly validate intermediate hypotheses; (3) Exploration pizoos, introduced with phrases such as "What if" or "Another approach," prompt the consideration of alternative solution paths; and (4) Integration pivofs, typically signaled by expressions such as "Now I see how," " synthesize previously explored ideas into a coherent final solution. We visualize what these stages look like in practice for emergent and synthetic traces in figure 2
Our analysis reveals that successful reasoning traces commonly employ multiple pivot types (96.1% contain at least three pivot categories, additional information per type given in appendix B), whereas unsuccessful traces frequently lack such pivots or exhibit limited diversity: This underscores the critical importance of structured, metacognitive transitions in effective reasoning. Motivated by these insights, we explicitly encode these structural and lexical patterns into a reasoning template to facilitate the generation of synthetic reasoning traces that capture RLM-like reasoning behaviors. We show the associated prompt structure in figure 3, which we subsequently use to guide reasoning trace generation (section 3.2), enabling us to systematically control and isolate stylistic reasoning elements in order to examine their impact on reasoning performance. First, the pivot categories are explicitly defined, followed by the general stages of problem solving that they correspond to. To generate the synthetic data, in addition to this prompt, the teacher LM is given the question and instructions on how to format its answer.
3.2 Reasoning trace data generation
Collect seed data Initially we curate seed data consisting of questions and their correct answers, ensuring that the accuracy of the eventual synthetic data can be verified. While previous works on RLM distillation often focus exclusively on math (Huang et al., 2024a), we
At time of writing, the difference in costs per token between flagship LMs and RLMs approach an order of magnitude from the same provider: input/ outptt pricing of $2.50/$10.00. for GPT-4o compared to $15.00 115.00/$60.00 $$00.0 for ol https://openal ctw. www./pp//ore/eecc/cccc//oo /pricing///////// [Accessed: 2025-02-21]
4

---

Published as a conference paper at COLM 2025
Prompt for Structured Reasoning with Explicit Pivots
This task requires solving problems using structured, real-time reasoning, including explicit self-monitoring and self-correction. Mimic the thought process of an agent that regularly pauses to reconsider assumptions, verify intermediate results, explore alternatives, and integrate findings into coherent solutions. Use explicit lexical pivots to signal shifts in thinking or corrections to your reasoning.
When solving the problem, follow a structured reasoning trace that clearly moves through the following stages:
Problem Framing: Restate the problem and identify key elements clearly.
2. Exploration: Consider one or more potential solution paths, openly weighing alternatives.
Verification: Explicitly test intermediate results or assumptions; if inconsistencies arise, pivot explicitly to clarify or correct.
4. Synthesis: Clearly integrate findings into coherent solution, explicitly connecting back to the original problem.
Revisit each stages as many times as necessary, backtracking in your thinking as much as possible.
When moving from one stage to the next, do so by leveraging a pivot at the start of your sentence to signal the shift from one reasoning stage to another. Here are the categories of pivots with some examples:
Realization pivots (recognizing errors or oversights): "Wait-", "Oh-", "Actu- ally-", "I missed something-".
Verification pivots (explicitly testing assumptions results): "Let me double- check-", "To verify-", "Checking again-".
(
Exploration pivots (considering alternative approaches): "What "f="",Anothee way to look at this-", "Alternatively-
Integration pivots (synthesizing different ideas or resolving contradictions): "Now I see how-", "This connects back to-", "Putting this together-".
Use direct, concise language. Short sentences should represent your evolving thoughts clearly. Use pivots naturally to signal shifts in reasoning, corrections, or deeper insights. Be explicit about confusion or uncertainty when it arises. The goal is to clearly capture the structured yet flexible process of reasoning, high- lighting non-linear thinking and self-correction, while making the logic easy follow in a Atream-of-concioussee style.
Figure 3: The prompt used to guide GPT-4o for generating the synthetic ST-HC dataset. This prompt explicitly defines the four key pivot types (Realization, Verification, Exploration, Integration) and mandates adherence to the four reasoning stages (Problem Framing, Exploration, Verification, Synthesis) derived from our analysis of emergent RLM traces (section 3.1). lts goal enforce specific stylistic patterns, including non-linear thinking and explicit self-correction, characteristic of effective reasoning.
aim to cover a wider range of additional domains that benefit from reasoning, such as coding, science, and logic. Specifically, for questions from several scientific domains, we select OlympicArena (Huang et al., 2024b). For logic and coding, we select AGIEval (Zhong et al., 2023) and LiveCodeBench v4 (Jain et al., 2024), respectively. We select NuminaMATH (LI et al., 2024), where we randomly select subset of 20,000 samples, and OmniMath (Gao et al., 2024) as sources of quantitative reasoning problems for our seed data, resulting a total of 31 31,586 question-answer pairs.
Generate synthetic traces We use the seed data to generate high-quality synthetic reasoning traces via state-of-the-art RLMs and LMs. For ST, choose R1 as, at the time of writing, it the best performing RLM that provides its full reasoning traces as part of its response. For ST-HC, choose GPT-4o as our teacher model using the prompt structure specified in
5

---

Published as a conference paper at COLM 2025
figure 3. We perform up to five rollouts per seed sample, discarding incorrect responses, stopping the model provides the correct answer. The rollouts are done in a zero-shot manner, Le. we provide only the question to the model and do not keep previous attempts as context.
Filtering synthetic samples First, filter out samples that are of short length (less than 50 tokens for the entire trace) to prioritize sample quality, as shorter samples typically contain few pivots and are therefore not as impactful in training. In effort to provide fairer comparison, we align the datasets to the same number of samples by downsizing the larger dataset. As the final ST-HC dataset contained fewer samples (N=18,242), we downsample the larger ST dataset to match this size, randomly removing questions not present in ST-HC and ensuring both datasets used for finetuning contain an equal number samples. This results in a final 18K samples for both ST and ST-HC, each in the form of a triple (question, reasoning trace, answer). We provide a more detailed overview the resulting datasets in appendix C.
3.3 Experimental details
Model finetuning We finetune a range of already instruction-tuned base LMs using our contributed reasoning datasets. For this, we select recent models of different families and sizes, namely: Llama 3.2 3B (Meta, 2024), Ministral 8B (liang et al., 2023), and Qwen2.5 32B (Qwen et al., 2025). We choose these models as they are all high performing for their parameter count, come with open-source weights, and have permissive licenses. The learning rate used during SFT varies for each LM in line with the model's parameter count. For 3B models we use a peak learning rate of 6 x 10 for 8B models we use 4 x 10 and for 32B models we use 1 x 10 5, For all models we use an effective batch size of 16. All models are trained for five epochs using a linear warmup for the first 10% of steps followed by cosine annealing. We use the AdamW optimizer (Loshchilov & Hutter, 2019) with P 0.9 and B2 0.95 and weight decay of 1 x 10-4, Training is performed on system comprising 8 Nvidia H100 GPUs using bfloat16 16 precision.
Baselines To contextualize our findings, evaluate (1) the base instruction-tuned model, (2) the model after SFT on ST, and (4) the model after SFT on ST-HC. Additionally, to gauge the impact of the particular style replication we propose versus distilling from regular CoT, we add another baseline, where we instruct the generating LLM to think step-by-step (SBS), following Kojima al. (2023). We then use the resulting CoT for distillation similarly to how we use the emergent (ST) and synthetic (ST-HC) reasoning traces. This comprehensive evaluation allows us to isolate the impact of our hard-coded reasoning approach across different model sizes and compare it against the reasoning capabilities of the model itself. For completeness, we evaluate the teacher models used to generate our datasets as well.
Evaluation To evaluate the reasoning capabilities of all models, we select challenging, widely used benchmarks that test reasoning capabilities. These include MATH500 (Lightman et al., 2023), AIME2024, and GPQA (Rein et al., 2024); covering math and range of scientific domains. Specifically, MATH500 and AIME2024 feature 500 and 30 competition math problems, respectively. GPQA consists of
198 questions from range of scientific fields such as Biology and Chemistry. We focus only on the hardest ("Diamond") subset of this particular dataset.
4 Results and discussion
4.1 Hard-coded reasoning results
The performance across all combinations of models and datasets evaluated in our study is presented in table 1. Our results clearly indicate that models finetuned with structured reasoning traces, whether emergent (ST) or synthetic (ST-HC), consistently and significantly outperform their baseline instruction-tuned counterparts across all benchmarks. Notably, even the smaller 3B and 8B parameter models exhibit substantial performance improve- ments. For instance, the Llama 3.2 3B model gains over 31 absolute percentage points

---

Published as a conference paper at COLM 2025
Model
Variant
Params
MATH500
AIME2024
GPQA (D)
Base
3B
36.4
6.7
26.3
Llama 3.2
SBS
45.8
10.0
28.3
ST
68.4
23.3
31.3
ST-HC
3B
64.2
16.7
29.3
Base
52.8
10.0
28.8
Ministral
SBS
8B
60.6
16.7
31.3
ST
8B
78.2
33.3
38,9
ST-HC
8B
77.0
33.3
34.8
Base
32B
76.8
16.7
49.0
Qwen2.5
SBS
32B
78.2
20.0
49.5
32B
89.0
53.3
56.1
ST-HC
32B
83.4
46.7
53.0
R1
671B
96.8
76.7
71.7
Teacher Models
GPT-4o
75.4
13.3
53,0
GPT-4o HC
81.2
16.7
55.1
Table 1. Performance comparison of language models finetuned reasoning traces. We compare base models against versions finetuned on SBS (step-by-step thinking GPT-4o), ST (emergent traces from R1), and ST-HC (synthetic traces from GPT-4o with the prompt in figure 3). This evaluates the impact of reasoning trace style on downstream performance across model scales and benchmarks. GPT-4o HC refers to the base LM prompted with our structure from figure 3. All results pass@1 accuracy percentage.
MATH500 when finetuned on ST, and the Ministral 8B model sees its AIME2024 score more than triple with either ST or ST-HC finetuning. These findings demonstrate that smaller models can markedly benefit from structured reasoning finetuning, challenging previous suggestions of minimal improvements for models of this scale (Team, 2025). Comparing the two fine-tuning approaches, we observe that models trained synthetic traces achieve performance that approaches, and some cases matches those trained on emergent RLM traces. This demonstrates that replicating the style of reasoning, even using less capable teacher model guided by our prompt, is highly effective for distilling reasoning capabilities. Finally, the results show that while generating synthetic data with SBS prompt is beneficial compared to the base model, there is significant performance gap between SBS and our ST and ST-HC methods. This demonstrates that the performance improvements are indeed substantially driven by the specific stylistic patterns we identified, rather than solely by distilling the generating LLM's general reasoning abilities.
When analyzing the thinking process during evaluation, we observe a clear correlation between successful reasoning and longer reasoning traces, as illustrated in figure Models finetuned either emergent or synthetic reasoning traces consistently produce substantially longer outputs compared their respective base models across evaluation benchmarks. This suggests that adopting and replicating structured, elaborate reasoning style whether learned from emergent traces or via synthetic ones - is a key mechanism driving the enhanced downstream reasoning capabilities we observed. Notably, figure 4 shows that finetuning on ST yields slightly longer reasoning traces than ST-HC, which correlates with their relative performance to another corroborating that larger number of tokens spent thinking typically correlates with improved reasoning (Muennighoff et al., 2025).
4.2 Ablative study
To further isolate the impact of reasoning style versus factual correctness, we conduct an ablation study using two modified datasets. First, we create ST-HC-W by adapting the synthetic ST-HC traces to retain their stylistic structure while leading to incorrect final answers generated by GPT-4o-mini (details given in appendix D). Second, we create ST- NT, which contains only the question-answer pairs from the original ST data, removing the reasoning traces entirely. The results, presented table 2, reveal several key insights. Models finetuned on ST-HC-W consistently outperform the base instruction-tune models
7

---

Published as a conference paper at COLM 2025
(3B)
Ministral (8B)
Owen2.5 (328)
MARIda
SPOA
Average Token Count
1899
stee
areed
Model Type
Figure Average token count by model family and training dataset across evaluations.
across all evaluation benchmarks. This demonstrates that learning the stylistic patterns of reasoning enhances problem-solving capabilities even when the training data's final conclusion is incorrect. However, ST-HC-W models perform noticeably worse than those trained on the stylistically similar but factually correct ST-HC dataset, confirming the value of accurate data. Unsurprisingly, models trained on ST-NT show only modest gains over the base models, as they do not learn generate additional thinking tokens the style of an RLM during inference.
These ablations confirm that while factual correctness is important for optimal performance, the stylistic patterns inherent in reasoning traces play critical role in enhancing the rea- soning abilities of distilled models. Crucially, we do not claim that style alone improves reasoning as the body of the traces still contains correct reasoning up until the answer, but rather em emphasize that style essential for improving reasoning capabilities. Thus, unlike traditional distillation methods relying predominantly on sample correctness, our results suggest that explicitly transferring a specific output structure is important.
Related work
Language model reasoning Language model reasoning has received increasing interest in recent years (Cobbe et al., 2021). Initially, LMs conditioned on reasoning examples dur- ing pretraining, post-training or in-context to improve their reasoning capabilities (Zhang et al., 2022; Wang et al., 2023; Wei et al., 2023; Li et al., 2024), but test-time scaling (Snell et al., 2024; Muennighoff et al., 2025) has introduced a new paradigm for improving LM reasoning. Further, DeepSeek-AI et al. (2025) explore training RLMs with reinforcement learning, finding that they mimic human reasoning processes like self-reflection and verifica- tion. The resulting RLMs provide reasoning traces as training data enhance LM reasoning ability. Our work demonstrates that the structural and lexical patterns in these traces play significant role improving reasoning performance, offering insights into what actually transferred during reasoning distillation.
Generalization language models Generalization in language models implies the ability to tackle unseen problems rather than simply reciting training data (Kang et al., 2024). LMs struggle with problems that differ from their training distribution, often closely following observed patterns down to individual terms (Razeghi al., 2022). Recent research shows LMs rely heavily on memorized patterns rather than developing generalizable reasoning capabilities (Schwarzschild et al., 2024). This pattern-matching behavior is especially evident

---

Published as a conference paper at COLM 2025
Model
Variant
Parameters
MATH500
AIME2024
GPQA Diamond
ST-HC-W
3B
48.2
10.0
28.4
Llama 3.2
ST-NT
3B
40.6
6.7
26.9
ST-HC-W
8B
62.8
20.0
29.9
Ministral
ST-NT
8B
56.2
13.3
30.0
ST-HC-W
32B
80.2
26.7
51.3
Qwen2.5
ST-NT
32B
78.8
20.0
49.8
Table 2. Ablation study results evaluating the distinct contributions of reasoning trace style and answer correctness. We compare model performance after finetuning on: (1) ST-HC-W, featuring stylistically consistent synthetic traces from ST-HC but deliberately leading incorrect answers, and (2) ST-NT, containing only the question-answer pairs from ST without reasoning traces. All results pass@1 accuracy as percentage.
in mathematical reasoning, where LMs show variance across different instantiations of the same question and declining performance when only numerical values change (Mirzadeh et al., 2025). Data contamination has also been identified as a source of apparent but false generalization (Jiang et al., 2024). Our work extends this discussion by investigating how stylistic patterns in reasoning traces influence model performance, revealing that structural elements of reasoning may be as important as factual content for enhancing problem-solving capabilities.
Reasoning distillation Distillation has long been used to improve various aspects of ma- chine learning models (Schmidhuber, 1992; Hinton et al., 2015; Sanh et al., 2020). For reasoning tasks with verifiable solutions, researchers have implemented rejection sampling methodologies that extract and validate advanced models' reasoning processes (Zelikman et al., 2022). More recently, significant performance improvements have been achieved through SFT on synthetic datasets generated by superior LMs (Gunasekar et al., 2023). With the emergence of RLMs, reasoning performance of instruction-turet substantially enhanced by finetuning on high-quality reasoning traces (DeepSeek-Al 2025). Multiple efforts have demonstrated success in distilling RLMs via SFT on reasoning traces that contain step-by-step thinking (Min et al., 2024; Huang et al., 2024a; Team, 2025; Labs, 2025; Xu et al., 2025). Our work attempts to decompose what is actually being trans- ferred during this distillation process, revealing that stylistic elements of reasoning traces contribute significantly to performance gains independent of their factual correctness.
LMs can be
al.,
Conclusion
Our work challenges existing assumptions by demonstrating that improvements in rea- soning capabilities in distilled language models can be driven by stylistic replication. By analyzing reasoning traces through the lens of established principles from cognitive sci- ence on how humans solve problems, markers (pivots) characteristic of effective LM reasoning. We validated these findings by synthetically replicating the stylistic patterns found in emergent reasoning traces using weaker model, achieving comparable downstream reasoning performance after distillation. Ablation studies confirmed that even reasoning traces with incorrect solutions enhance downstream performance, highlighting the effectiveness of stylistic mimicry in distilled models. Collectively, our results contribute to deeper understanding of reasoning in language models and offer practical insights into efficient finetuning strategies and data generation practices.
identified structural patterns and key lexical
References
Zevuan Allen-Zhu and Yuanzhi Li. Towards understanding ensemble, knowledge distil- lation and self-distillation in deep learning. In The Eleventh International Conference on Learning Representations, 2023. htttps////oppmreeeeeeeee/frroore

---

Published as a conference paper at COLM 2025
Emily M. Bender and Alexander Koller. Climbing towards NLU: On meaning, form, and understandingg in the age of data. In Proceedings of the 58th Anmual Meeting of the Association org/2020.
for Computational Linguistics, pp. 5185-5198, 2020. URL
Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the
2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT '21, pp. 610-623, New York, NY, USA, 2021. Association for Computing Machinery. ISBN 9781450383097. doi: 10.1145/34421888 34.444492.. URL httpps///ol org/10. 0011/114444411 88. 3445922.
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners, 2020. URL https:/ http://aaxvvvvrrro //s///b//220 2005. 14165.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems, 2021. URL 14168
DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guannbo Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jiawei Wang, Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, J. L. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang, Lecong Zhang, Liang Zhao, Litong Wang, Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Meng Li, Miaojun Wang, Mingming Li, Ning Tian, Panpan Huang, Peng Zhang, Qiancheng Wang, Qinyu Chen, Qiushi Du, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang, R. J. Chen, R. L. Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang, Shuiping Yu, Shunfeng Zhou, Shuting Pan, S. S. Li, Shuang Zhou, Shaoqing Wu, Shengfeng Ye, Tao Yun, Tian Pei, Tianyu Sun, T. Wang, Wangding Zeng, Wanjia Zhao, Wen Liu, Wenfeng Liang, Wenjun Gao, Wenqin Yu, Wentao Zhang, W. L. Xiao, Wei An, Xiaodong Liu, Xiaohan Wang, Xiaokang Chen, Xiaotao Nie, Xin Cheng, Xin Liu, Xin Xie, Xingchao Liu, Xinyu Yang. Xinyuan Li, Xuecheng Su, Xuheng Lin, X. Q. Li, Xiangyue Jin, Xiaojin Shen, Xiaosha Chen, Xiaowen Sun, Xiaoxiang Wang, Xinnan Song, Xinyi Zhou, Xianzu Wang, Xinxia Shan, Y. K. Li, Y. Q. Wang, Y. Wei, Yang Zhang, Yanhong Xu, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Wang, Yi Yu, Yichao Zhang, Yifan Shi, Yiliang Xiong, Ying He, Yishi Piao, Yisong Wang, Yixuan Tan, Yiyang Ma, Yiyuan Liu, Yongqiang Guo, Yuan Ou, Yuduan Wang, Yue Gong, Yuheng Zou, Yujia He, Yunfan Xiong, Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Y. X. Zhu, Yanhong Xu, Yanping Huang, Yaohui Li, Yi Zheng, Yuchen Zhu, Yunxian Ma, Ying Tang, Yukun Zha, Yuting Yan, Z. Z. Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhean Xu, Zhenda Xie, Zhengyan Zhang, Zhewen Hao, Zhicheng Ma, Zhigang Yan, Zhiyu Wu, Zihui Gu, Zijia Zhu, Zijun Liu, Zilin Li, Ziwei Xie, Ziyang Song, Zizheng Pan, Zhen Huang, Zhipeng Xu, Zhongyu Zhang, and Zhen Zhang. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning, 2025. 12948.
URL.
Bofei Gao, Feifan Song, Zhe Yang, Zefan Cai, Yibo Miao, Qingxiu Dong, Lei Li, Chenghao Ma, Liang Chen, Runxin Xu, Zhengyang Tang, Benyou Wang, Daoguang Zan, Shanghaoran Quan, Ge Zhang, Lei Sha, Yichang Zhang, Xuancheng Ren, Tianyu Liu, and Baobao Chang. Omni-math: A universal olympiad level mathematic benchmark for large language models, 2024. URL https: //arxi/.aiii.a///a wtw./bs.//////oooooooooo 0. 07985.
10

---

Published as a conference paper at COLM 2025
Suriya Gunasekar, Yi Zhang, Jyoti Aneja, Caio César Teodoro Mendes, Allie Del Giorno, Sivakanth Gopi, Mojan Javaheripi, Piero Kauffmann, Gustavo de Rosa, Olli Saarikivi, Adil Salim, Shital Shah, Harkirat Singh Behl, Xin Wang, Sébastien Bubeck, Ronen Eldan, Adam Tauman Kalai, Yin Tat Lee, and Yuanzhi Li. Textbooks are all you need, 2023. URL https://arxiv.orr/ab///22
100..116444 11644.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network, 2015. URL
///.///533.021111 02531.
Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hennigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, Erich Elsen, Jack W. Rae, Oriol Vinyals, and and Laurent Sifre. Training compute-optimal large language models, 2022. URL https://arxiv. httpss//arriv..og/abb//2... 15556.
Xin Huang, Rui Xu, Jiawen Gu, Yuxi Jiang, Lingfeng Zhao, Zhenguo Jiang, Wanjun Yin, Daxin Jiang, Xipeng Huang, and Xipeng Qiu. Ol: The journey of replication. arXiv 04692.
preprint (tto:206.044999 2024a. URL https:
Zhen Huang, Zengzhi Wang, Shijie Xia, Xuefeng Li, Haoyang Zou, Ruijie Xu, Run-Ze Fan, Lyumanshan Ye, Ethan Chern, Yixin Ye, Yikai Zhang, Yuging Yang, Ting Wu, Binjie Wang, Shichao Sun, Yang Xiao, Yiyuan Li, Fan Zhou, Steffi Chern, Yiwei Qin, Yan Ma, Jiadi Su, Yixiu Liu, Yuxiang Zheng, Shaoting Zhang, Dahua Lin, Yu Qiao, and Pengfei Liu. Olympicarena: Benchmarking multi-discipline cognitive reasoning for superintelligent ai, 2024b. URL https: /arxxvv. org/abs/2406 12753.
Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Ar- mando Solar-Lezama, Koushik Sen, and Ion Stoica. Livecodebench: Holistic and contami- nation free evaluation of large language models for code. arXiv preprint arXiv:2403.07974, 2024.
Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Ana Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mistral 7b, 2023, URL httpp://arxxv.org//bbs//
06825.
Minhao Jiang, Ken Ziyu Liu, Ming Zhong, Rylan Schaeffer, Siru Ouyang, Jiawei Han, and Sanmi Koyejo. Investigating data contamination for pre-training language models, 2024.
URL htt.htsss//aaxiioorg//a/////////
06059.
Katie Kang, Amrith Setlur, Dibya Ghosh, Jacob Steinhardt, Claire Tomlin, Sergey Levine, and Aviral Kumar. What do learning dynamics reveal about generalization in Ilm reasoning?, 2024. URL https: //axxiv org/abs/244 076611.
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models, 2020. URL https: //axxiv////ivv.a//aa http://aaxxv..oo/abb.a/s
org/abs/////eoooooooog.oooeo .08661.
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners, 2023. URL http:s:://axii..rr//aaaaaaa 11916.
Bespoke Labs. Bespoke-stratos:
The unreasonable effectiveness of rea-
soning distillation, 2025. duspoke-strator haapooee-sstttstttt---eeeaaaaaa htt.-uurresoobbee-ffefiiiise--ss -eaeeeess------rrrssoon--n---eie Accessed: 2025-01-22.
URL https://wwe bespokelasssslass..e..o.. aasspokelabs.ai/blog ai/blog/
Chen Li, Weiqi Wang, Jingcheng Hu, Yixuan Wei, Nanning Zheng, Han Hu, Zheng Zhang, and Houwen Peng. Common 7b language models already possess strong math capabili- ties, 2024. URL httpp::/aarxvv.rr//bbb///// 24700 04706.
11

---

Published as a conference paper at COLM 2025
Jia LI, Edward Beeching, Lewis Tunstall, Ben Lipkin, Roman Soletskyi, Shengyi Costa Huang, Kashif Rasul, Longhui Yu, Albert Jiang, Ziju Shen, Zihan Qin, Bin Dong, Li Zhou, Yann Fleureau, Guillaume Lample, and Stanislas Polu. Numinamath, 2024. URL. https://githut report/numina_dataset.. pddf.
Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ily Sutskever, and Karl Cobbe. Let's verify step by step, 2023. 20050.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization, 2019. URL
Meta. Llama 3.2 lightweight modelss model-cards-andc patprrarrd---a---rro-t---rrrreee fromrmotttrrrrttss/lmmmmsmaa Accessed: 2025-01-22.
models (1b/3b), 2024. URL https://ww/ 11a.m.aaamammmmmmmmmm..mm htw.aa.co//.......moo com/docs/
Yingqian Min, Zhipeng Chen, Jinhao Jiang, Jie Chen, Jia Deng, Yiwen Hu, Yiru Tang, Jiapeng Wang, Xiaoxue Cheng, Huatong Song, Wayne Xin Zhao, Zheng Liu, Zhongyuan Wang, and Ji-Rong Wen. Imitate, explore, and self-improve A reproduction report on slow-thinking reasoning systems, 2024. URL httpps///arr.....rga/////22..2
Seyed Iman Mirzadeh, Keivan Alizadeh, Hooman Shahrokhi, Oncel Tuzel, Samy Bengio, and Mehrdad Farajtabar. GSM-symbolic: Understanding the limitations of mathematical reasoning in large language models. In The T'hirtennth International Conference on Learning Representations, 2025. URL https: pttts://oppe http://///////eew...//// wwt/teeetttideaaaaeeeee wttpr//wwww.....ooooo//////////a
Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. sl: Simple test-time scaling, 2025. URL https://arrktt www. hergg/abs/250 19393.
Allen Newell and Herbert Alexander Simon. Human problem solving. Prentice-Hall, 1972.
OpenAI. Learning to reason with Ilms, 2024. Baaming-to-reeaso-vii
URL https://openai.comm cow/ /ot/in/e//c//////x///////// index/
Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, P'ei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report, 2025. URL /ttps://arxian http:/aaxxvv....gaab/aa at///4///1111 15115.
Yasaman Razeghi, Robert L. Logan IV, Matt Gardner, and Sameer Singh. Impact of pretrain- ing term frequencies on few-shot reasoning, 2022. URL https: /ttpaaawwwi..rg/////////./// 07206.
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. GPQA: A graduate-level google- proof q&a benchmark. In First Conference on Language Modeling, 2024. URL https: wtt.:iwwww.ee.ooooiiiitii------.
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter, 2020. URL https://arxiv.. orgg / 1910. 01108. 1008.
Jürgen Schmidhuber. Learning complex, extended sequences using the principle of history compression. Neural Computation, 4(2)) 242-24,,422222 03 1992. ISSN 0899-7667. doi: 10.1162/ neco..199..222233.. URL httht: http:////oooorg/////./ org/10. 1162/neco. 1992.4. 2.
Avi Schwarzschild, Zhili Feng, Pratyush Maini, Zack Lipton, and Zico Kolter. Rethinking Ilm memorization through the lens of adversarial compression. ar preprint, 2024.
12

---

Published as a conference paper at COLM 2025
Murray Shanahan. Talking about large language models. Commun. ACM, 67(2):68-79, January 2024. ISSN 0001-0782. doi: (0.1145/362472) URL https:://oi..orr hrg.d/oo...///oooooooo https://ddoi.or//11.11144 1145/ 3624724.
Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling Ilm test-time compute optimally can be more effective than scaling model parameters, 2024. URL https://
arxiv.org/abss/248.. 03314.
NovaSky Team. Sky-t1: Train your own ol preview model within $450. https://novasky: biggihub..o/ppstt//kktll 2025. Accessed: 2025-01-09.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models, 2023. URL http:::/arxxv..org//bb// 11111
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large lan- guage models, 2023. URL https://ariiv.orr/bbb/22.. 11903.
Haotian Xu, Xing Wu, Weinong Wang, Zhongzhi Li, Da Zheng, Boyuan Chen, Yi Hu, Shijia Kang, Jiamiig Ji, Yingying Zhang, Zhijiang Guo, Yaodong Yang, Muhan Zhang, and Debing Zhang. Redstar: Does scaling long-cot data unlock better slow-reasoning systems?, 11284.
2025. URL https: //axxvv http///xxiv..oggbbbbbe/a
Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Goodman. STar: Bootstrapping reasoning with reasoning. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho Advances in Neural Information Processing Systems, 2022. URL https://openrevies
Det/fforum2id-.BERRdg222
Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. Automatic chain of thought prompting in large language models, 2022. URL http:::/aaxxi...rrabb//////// 23400009331.....3...
Wanjun Zhong, Ruixiang Cui, Yiduo Guo, Yaobo Liang, Shuai Lu, Yanlin Wang, Amin Saied, Weizhu Chen, and Nan Duan. Agieval: A human-centric benchmark for evaluating foundation models, 2023. URL https: //aps::///axeee wwt.rrww........///../. wtt..ra///34.00335553 06364.
13

---

Published as a conference paper at COLM 2025
A Emergent Reasoning Trace Example
We give an example of a reasoning trace in figure Here, a RLM answers a mathematical puzzle question, initially producing extensive reasoning trace with frequent backtracking, followed by giving the final correct answer.
Reasoning Trace Example
Question: A juggler can juggle 16 balls. Half of the balls are golf balls, and half of the golf balls are blue. How many blue golf balls are there?
Okay, let's see. The problem says juggler can juggle 16 balls. Half of them are golf balls, and half of the golf balls are blue. I need to find out how many blue golf balls there are. Alright, let me break this down step by step.
Reasoning:
First, the total number of balls is 16. Half of those are golf balls. So, half of 16 would be... 16 divided by 2, which is 8. So, there are 8 golf balls. Got that down. Now, out of these 8 golf balls, half blue. Hmm, so again, half of 8 would be 8 divided by 2. Let me calculate that. 8 divided by 2 is 4. So, does that mean there are 4 blue golf balls? That seems right, but let me double-check to make sure I didn't skip any steps or misunderstand the question.
Wait, the problem says half of the balls are golf balls. So total balls: 16. Half are golf balls: 8. Then half of the golf balls are blue. So half of 8 is indeed 4. So, 4 blue golf balls. Let me just visualize this. If there are balls in total, and half (so 8) are golf balls, and of those 8 golf balls, half are blue and half are another color. So splitting 8 into two equal parts gives 4 and 4. So yes, blue golf balls. That makes sense.
Is there another way to approach this problem? Maybe by using fractions. Let's see. Total balls = 16. Half are golf balls: so 1/2 of 16. Then half of those are blue: so 1/2 of 1/2, which is 1/4. So 1/4 of the total balls are blue golf balls. 1/4 of 16 is 4. Yep, same answer. So whether do it step by step or use fractions, I still get 4. That seems consistent.
Alternatively, if were to write equations, let's denote total balls as T= 16. Number of golf balls = 2. Then number of blue golf balls B G / 2. Substituting, (16 / 2) / 2 8 2 = 4. Same result. So all methods lead to 4. I think that's solid. Let me just check if interpreted "half of the golf balls are blue" correctly. The problem didn't mention anything about the other half, but since half of the golf balls are blue, the other half could be any other color, but that's irrelevant to the question. The question only asks for the number of blue golf balls, so 4 is the answer.
Wait, just to be thorough, the problem doesn't 't have any trick wording, right? Like, when it says half of the balls are golf balls, that's straightforward. Then half of those golf balls blue. So no, no tricky parts. it had said half of the blue balls are golf balls, that would be different, but it's the other way around. Half of the golf balls are blue. So straightforward.
Another check: total golf balls are 8. If 4 are blue, then 4 are not blue. That adds up to 8. Then the other 8 balls (non-golf balls) are something else entirely. But since the question is only about blue golf balls, we don't need to consider those.
think that's all. All angles checked. So, 4 blue golf balls. Confident that's correct.
Final Answer: The number of blue golf balls is
Figure 5: A detailed example of mathematical pooie from R1 with full reasoning trace.
B Emergent Trace Analysis
This appendix provides quantitative details from the analysis performed on 16,710 successful emergent reasoning traces generated by R1 model, taken from Labs (2025). This analysis
14

---

Published as a conference paper at COLM 2025
informs the discussion of pivot types and reasoning stages in Section 3.1. On average, these traces are substantial in length, comprising 2,713 tokens. Structurally, they contain an average of 141.7 paragraphs. Metacognítive activity is frequent, with an average of 93.5 identified pivots occurring within each reasoning trace.
B.1 Details of Categorizing the Reasoning Trace Contents
To analyze reasoning traces, developed systematic approach for identifying pivots and reasoning stages using regular expression pattern matching. For each of the four pivot categories (Realization, Verification, Exploration, and Integration), we created comprehen- sive regex patterns capturing lexical markers that signal metacognitive transitions for instance, phrases such as "Actually" for Realization pivots or "Therefore" for Integration pivots. Similarly, we defined patterns for the four reasoning stages (Problem Framing, Exploration, Verification, and Synthesis) based on characteristic expressions and structural elements. We experimented with using an LM in the form of GPT-4o mini for this task but found it to be no more performant. This framework enabled automated extraction and quantification of reasoning elements across all 16,710 traces. While matching to regular expressions not a perfect way to extract this information, we found to perform well considering the very standardized nature of reasoning traces. We iteratively refined our patterns over multiple rounds after manual inspection of reasoning traces by the authors, ensuring balanced detection across all categories. Each trace was analyzed for both the frequency of pivot occurrences and the presence of reasoning stages, allowing us to quantify both the melacognitive transitions and the structural patterns that characterize effective reasoning.
B.2
Pivot Analysis
Pivot Diversity A key indicator of complex reasoning is the variety of metacognitive shifts employed. The analyzed traces show an average diversity of 3.51 distinct pivot types per trace. Furthermore, a very high majority, 96.1% of the traces, contains at least three different pivot categories. This high percentage strongly confirms the observation that successful traces typically involve multiple forms of reflection and correction during the reasoning process.
Pivot Type Frequencies The frequency and prevalence each specific pivot type across the dataset are summarized in Table 3. Notably, Integration and Realization pivots remain extremely common, appearing in nearly all analyzed traces. Exploration pivots are the least prevalent, present in 87.0% of traces, indicating consideration of alternative paths or hypotheses does not occur as frequently.
Pivot Type Realization Exploration Verification Integration
Avg. Occurrences per Trace % Traces Present
18.96
98.6%
16.11
87.0%
1.37
89.6%
67.64
100.0%
Table Frequency and prevalence of identified pivot types within the analyzed emergent reasoning traces (N=16,710).
B.3 Reasoning Stage Analysis
The analysis also quantifies the presence of segments corresponding to the four cognitive reasoning stages discussed in section 3.1. The difference compared to the pivot types analy- sis is that single reasoning stage can contain multiple pivots, even separate types. In this case, ascribe the trace to the reasoning stage found at the beginning of the trace. The average occurrences and prevalence of these stages are detailed in table 4. Synthesis stages, often comprising multiple steps integration points, universally present. Problem
15

---

Published as a conference paper at COLM 2025
Framing, Verification, and Exploration stages are all highly prevalent, appearing in the vast majority of traces, consistent with the high frequency of their corresponding pivot types.
Reasoning Stage
Avg. Occurrences per Trace
% Traces Present
Problem Framing
3.13
79.2%
Exploration Verification Synthesis
6.81
87.0%
3.34
89.6%
84.08
100.0%
Table 4. Frequency and prevalence of identified reasoning stages within the analyzed emergent reasoning traces (N=16,710).
B.4 Key Insights from Analysis
The quantitative analysis reinforces the qualitative observations presented in the main paper. Successful emergent reasoning traces consistently utilize a diverse range of pivot types, averaging 3.51 distinct types per trace, with overwhelming 96.1% using three or more, indicating frequent and varied metacognitive adjustments. Furthermore, the presence and frequency of identified reasoning stage segments, detailed in Table generally align with the cognitive science framework encompassing problem framing, exploration, verification, and synthesis. The high prevalence of all four stage types underscores the iterative nature of the observed reasoning process.
Data Source
Number of Samples
OlympicArena (Huang et al., 2024b)
4,250
AGIEval (Zhong et al., 2023)
2,385
LiveCodeBench v4 (Jain et al., 2024)
713
NuminaMATH (LI et al., 2024)
20,000
OmniMath (Gao et al., 2024)
4,238
Total
31,586
Table 5. Composition of the seed data pool used for generating reasoning traces. The number of samples from NuminaMATH reflects randomly selected subset.
C Dataset Statistics
This section provides detailed statistics about the datasets created and utilized in our study, namely the seed data pool and the derived reasoning trace datasets SMOLTRACES (ST) and SMOLTRACES, AMoL.TRACRS-AARCODDD (ST-HC).
C.1
Seed Data Compilation
The foundation for generating our reasoning trace datasets is curated collection of question- answer pairs sourced from diverse benchmarks spanning mathematics, science, logic, and coding. As detailed in Section 3.2, we selected problems from OlympicArena (Huang al., 2024b), AGIEval (Zhong et al., 2023), LiveCodeBench v4 (Jain et al., 2024), NuminaMATH (LI et al., 2024), and OmniMath (Gao et al., 2024). This process resulted in a final seed dataset comprising 31,586 unique question-answer pairs. Table 5 presents breakdown of the sources contributing to this seed data pool. To ensure the integrity evaluations, performed decontamination this initial pool by removing any ques- tions overlapping with our chosen evaluation benchmarks (MATH500, AIME2024, GPQA Diamond).
our downstream
16

---

Published as a conference paper at COLM 2025
Statistic
Dataset Values
Trace Origin
ST: Emergent (R1)
ST-HC: Synthetic (GPT-40 Hard-coded Prompt, Fig. 3)
Seed Questions Source
ST: Shared Pool (Table N=31,586)
ST-HC: Shared Pool (Table 5, N-31,586)
Final Number of Samples
ST: 25,802 (adjusted to N=18,242 after)
ST-HC: 18,242
Filtering Applied
ST: Correct Answer (up to attempts), Min. Length (50 tokens)
ST-HC: Correct Answer (up to 5 attempts), Min. Length (50 tokens)
Avg. Trace Length (Tokens)
ST:2,521
ST-HC: 2,101
Avg. Pivots per Trace
ST: 93.4
ST-HC: 89.1
Table 6. Summary statistics for the final generated reasoning trace datasets used in fine-tuning. Values reflect the approximately 18,250 filtered and balanced samples each dataset (ST and ST-HC). Average length and pivot counts are illustrative; actual values depend on the final composition.
C.2 Generated Reasoning Trace Datasets
Using the curated seed data, we generated two parallel datasets featuring detailed reasoning traces, described in Section 3.2. Table provides summary comparing key statistics of the final, balanced ST and ST-HC datasets. Average trace length and pivot counts reflect measurements across these final samples.
SMOLTRACES (ST): This dataset contains emergent reasoning traces generated by the state-of-the-art RLM R1 (DeepSeek-Al et al., 2025). For each seed question, we prompted R1 in a zero-shot manner up to five times, retaining the first trace that yielded the correct final answer.
SMOLTRAACES-AARDCODDD (ST-HC): This dataset comprises synthetic reasoning traces generated using GPT-4o, guided by the structured prompt detailed in figure (Figure 3), This prompt enforces the inclusion of specific structural elements and lexical pivots identi- fied in our analysis (Section 3.1). Similar ST, generation involved up five zero-shot attempts per seed question, keeping the first correct trace.
Balancing: To ensure fair comparison in finetuning experiments, we balanced the datasets by size. As the ST-HC dataset contained fewer samples after filtering, we downsample the larger ST dataset by randomly removing samples that are not present in the smaller dataset until both datasets have an equal number of samples. Therefore, the final versions of both ST and ST-HC used for finetuning contain an equal number samples (N=18,242).
D Synthetic Traces with Wrong Answer Details
This section details the construction process for the ST-HC-W dataset, used in our ablation study (Section 4.2) investigate the impact of reasoning trace style independent of final answer correctness. The goal is to create dataset that retains the stylistic and structural characteristics of the synthetic ST-HC traces but deliberately leads to an incorrect final answer. First, we use the ground truth answer associated with the seed question for each sample in ST-HC. Then, we prompt GPT-4o-mini, instructing it to provide different, incorrect answer that is similar in format (e.g. "9.11" instead of "9.9"). We compare the answer generated by GPT-4o-mini against the known correct answer and if the generated answer matches the correct one, we repeat the procedure.
17

---

Published as a conference paper at COLM 2025
The resulting ST-HC-W dataset mirrors the size of ST-HC, containing 18,242 samples. Each sample inchudes the original question, a reasoning trace stylistically similar to ST-HC, but which concludes with incorrect final answer. This construction allows for better isolation of the effect of learning stylistic reasoning patterns during finetuning.
18


Reviews:
[{"review_id": "OpPakvuWmo", "reviewer": "Reviewer_nWWk", "summary": "This paper argues that distilling reasoning abilities in LLMs relies more on mimicking \"stylistic patterns\" (e.g., structured steps, backtracking, phrases like \"Let me check\") than learning deep reasoning. Using two datasets—SMOLTRACES (real RLM traces) and SMOLTRACES-HARDCODED (synthetic traces mimicking style)—the authors show that models trained on synthetic style-focused traces (even with wrong answers) perform nearly as well as those trained on real RLM traces. Key experiments suggest \"style replication\" drives reasoning gains, not just correct answers.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This paper challenges the assumption that distillation requires deep reasoning. Style imitation alone boosts performance.  \n2. The methodology of analyzing 17K RLM traces to identify cognitive stages (problem framing, exploration, verification, synthesis) and lexical pivots is well-grounded in cognitive science (Newell & Simon, 1972) and provides a structured way to define \"style.\" The quantitative analysis in Appendix B (pivot diversity, frequencies) adds rigor.  \n3. The experiments are smart.  1) Synthetic traces (ST-HC) with explicit style rules match real RLM performance. 2) ST-HC-W (stylistic traces with wrong answers) still improve models over baselines, proving style matters.  \n4. The SMOLTRACES (ST) and SMOLTRACES-HARDCODED (ST-HC, ST-HC-W) datasets, along with the analysis of RLM trace styles, are valuable resources for the community.", "reasons_to_reject": "1. Some \"style\" elements (e.g., verification steps) are part of reasoning itself.  The paper defines style by \"structural attributes such as trace length, lexical coherence, and backtracking frequency, rather than comprehension itself\" (lines 38-40). However, many of these \"stylistic\" elements (e.g., explicit verification steps like \"Let me double-check,\" exploration of alternatives like \"What if\") are arguably integral parts of a substantive reasoning process. Can you truly separate the style of verification from the act of verification?\n\n2. If models improve significantly even with ST-HC-W, what exactly are they learning? Are they learning to generate longer, more structured outputs that, by chance or by some implicit bias in the evaluation benchmarks, lead to better scores? Or are they learning a meta-cognitive scaffold that, even if filled with a final error in training, helps them structure their \"thoughts\" better at test time on new problems? The paper leans towards the latter but could explore this more.  \n\n3. While GPT-4o is a \"standard LM\" compared to a specialized RLM like R1, it's still an extremely powerful model. The prompt in Figure 3 is quite detailed in instructing how to reason. How much of the \"style\" benefit comes from the explicit prompting versus the inherent (and significant) reasoning capabilities of GPT-4o itself, even if not \"specialized\" in the RLM sense? Could a much weaker model generate equally effective \"stylistic\" traces with the same prompt?  \n\n4. Real traces (ST) come from R1, a model already optimized for reasoning. Its \"style\" is refined, not raw.  \n\n5. The paper shows improved performance on standard benchmarks. How well would these \"stylistically trained\" models perform on problems requiring genuinely novel insights or reasoning patterns not captured by the identified styles? Is there a risk of overfitting to a particular way of expressing reasoning, which might be brittle?", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yhQy6YkO1S", "reviewer": "Reviewer_x9kt", "summary": "This paper researches the reason of reasoning ability distillation. Results show that the distillation mainly benefit from the stylistic pattern instead of the true reasoning abilities. To do that, the authors first analyze the cot with correct answer and find out some lexical pivots. And then they use the generated synthetic data to finetune models and achieve roughly the same performance compared to the real cot. This implies that the styl is key to improvements in reasoning.", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The idea is quite novel. The research question is important. And the insights that the authors provide are quite inspiring.\n\n2. The experiment results are strong. It's convincing to support the proposed claim with the performance on the synthetic training setting. The design of synthetic data is quite reasonable with the support of lexical pivots. \n\n3. The overall writing is clear.", "reasons_to_reject": "I didn't foresee any severe issues of this work. But if possible, more qualitative study (e.g., case study) could be helpful to further support the findings of this work. Afterall, the claim in this work is quite new and it might be controversial.\n\nBesides, it would be great if the authors could provide more insights about the lexical pivots. This claim might also be controversial. Providing more insights or descriptions would be helpful for readers to accept/understand this point.", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "V2cjheL3EJ", "reviewer": "Reviewer_CCPR", "summary": "In this study authors investigate distillation through finetuning on data generated by strong LLMs. Authors start with reasoning traces generated by RLMs, and find that successful reasoning traces consistently align with cognitive stages of human reasoning, while employing multiple pivot types (realization, verification, and exploration). Authors call these \"stylistic patterns\". Based on this analysis, authors craft a prompt to guide strong non-reasoning LM (GPT-4o) to generate solutions while adhering to RLLMs' reasoning traces. Authors find that small LMs finetuned on these synthetic data significantly improve performance across multiple tasks, even if traces in train data lead to incorrect answers.", "questions": "", "limitations": "", "rating": 8, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- paper is clear and well-written\n- authors perform comprehensive experiments, comparing models of different sizes and families\n- authors test performance on multiple datasets requiring different types of reasoning\n- authors first to clearly define stylistic patterns of the reasoning traces, proving the importance of form over substance\n- this paper deepens the understanding of reasoning abilities of LLMs and offers a way to transfer them to smaller models", "reasons_to_reject": "Paper could benefit from additional ablation studies (see below)\n\nMy main concerns were addressed in rebuttal.", "questions_to_authors": "In table 1, you compare models finetuned on traces generated by RLLM and on synthetic traces generated by GPT. In Table 2 you additionally ablate on wrong answers, and data without traces. However, it was previously shown that finetuning on CoT data generated by strong models (such as GPT), can significantly boost small LMs. What I'm missing is comparing with distillation on GPT's CoT data, generated by \"Let's think step-by-step\" prompt. How close will it be to ST-HC results? Is it possible to replicate process described in 3.2, but with \"Let's think step-by-step\" prompt, to answer the following: are the performance improvement brought by stylistic learnings, or through distilling GPT's reasoning abilities? in other words, how much improvement prompt in Fig 3 brings over \"Let's think step-by-step\" prompt for synthetic data generation?", "extra_scores": {}}, {"review_id": "lqXycwwsLT", "reviewer": "Reviewer_F6zJ", "summary": "This paper studies the role of reasoning traces in training language models for reasoning tasks. Building on prior work that highlights the effectiveness of detailed reasoning steps, the authors introduce two datasets: emergent reasoning traces and synthetic reasoning traces, both curated to produce correct final answers. Experiments show that language models trained on these datasets perform similarly. Moreover, models trained with synthetic reasoning traces that include incorrect final answers still show improved performance, suggesting that the structure of reasoning, rather than correctness alone, may be beneficial.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper tackles an important and timely question in LLM reasoning—what elements of reasoning traces contribute most to model performance. The proposed dataset construction methodology is clear, and the finding that structured reasoning (even with incorrect answers) can improve model performance is intriguing and potentially useful for future prompting and training strategies.", "reasons_to_reject": "The central claims are not sufficiently substantiated. The similarity in performance between the two datasets does not conclusively demonstrate that the synthetic structure is the key contributing factor. Moreover, emergent traces still slightly outperform synthetic ones, which may point to the importance of diversity and naturalness in reasoning rather than structure alone. The paper would benefit from deeper analysis and stronger theoretical or empirical justification for its conclusions.", "questions_to_authors": "How do you justify that style replication is the key contributing factor, given that the synthetic traces (ST-HC) still underperform compared to the emergent traces (ST)?\n\nAdditionally, have you experimented with using other teacher models besides GPT-4o to generate synthetic reasoning traces?\n\n=================\nThe authors have addressed my concerns by providing additional experiments.", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper argues that when large reasoning models are distilled into smaller models, the smaller models mainly are learning stylistic patterns rather than deeper reasoning.

Pros:
1. The paper provides important insights about the nature of reasoning distillation.
2. The paper provides some careful analyses.
3. The paper involves the release of datasets that will be useful artifacts.
4. The paper provides interesting grounding in cognitive science.

Cons:
1. Reviewer nWWk raised some big-picture questions about the framing and implications (e.g., can style and reasoning truly be separated? How would these results extend to benchmarks that require more novel reasoning?) The authors gave thoughtful responses to these questions; to the extent that space allows, incorporating these thoughts into the paper would be helpful.
2. Reviewer CCPR suggested some additional ablations, which the authors have run; I encourage the authors to add these to the paper.

[Automatically added comment] At least one review was discounted during the decision process due to quality]
Decision: Accept