Revisiting Chain-of-Thought in Code Generation: Do Language Models Need to Learn Reasoning before Coding?
Authors: Ren-Biao Liu
Anqi Li
Chaoding Yang
Hui Sun
Ming Li
Venue: ICML 2025 poster (2025)
Keywords: Code Generation
Supervised Fine-Tuning
Chain of thought
CodeLLM
Primary area: deep_learning->large_language_models
TL;DR: We propose that generating code first and then providing CoT as explanations can improve CodeLLM performance with Supervised Fine-Tuning.

Abstract: Large Language Models (LLMs) have demonstrated exceptional performance in code generation, becoming increasingly vital for software engineering and development. Recently, Chain-of-Thought (CoT) has proven effective for complex tasks by prompting LLMs to reason step-by-step and provide a final answer.
However, research on *how LLMs learn to reason with CoT data for code generation* remains limited.
In this work, we revisit classic CoT training, which typically learns reasoning steps before the final answer.
We synthesize a dataset to separate the CoT process from code solutions and then conduct extensive experiments to study how CoT works in code generation empirically.
We observe counterintuitive phenomena, suggesting that the traditional training paradigm may not yield benefits for code generation. Instead, training LLMs to generate code first and then output the CoT to explain reasoning steps for code generation is more effective.
Specifically, our results indicate that a 9.86% relative performance improvement can be achieved simply by changing the order between CoT and code. Our findings provide valuable insights into leveraging CoT to enhance the reasoning capabilities of CodeLLMs and improve code generation.

Revisiting Chain-of-Thoughe in Code Generation: Do Language Models Need to Learn Reasoning before Coding?
Ren-Biao Liu 12 Anqi Li 12
Chaoding Yang 12 Hui Sun
Ming Li 12
Abstract
Training Loss with Smooth
Average Performance (Pass@)
2.6
Large Language Models (LLMs) have demon- strated exceptional performance in code genera- tion, becoming increasingly vital for software en- gineering and development. Recently, Chain-of- Thought (CoT) has proven effective for complex tasks by prompting LLMs reason step-by-step and provide final answer. However, research how I.LMs learn reason with CoT data for code generation remains limited. In this work, we revisit classic CoT training, which typically learns reasoning steps before the final answer. We synthesize a dataset to separate the CoT process from code solutions and then conduct extensive experiments study how CoT works in code generation empirically. We observe counterintu- itive phenomena, suggesting that the traditional training paradigm may not yield benefits for code generation. Instead, training LLMs to generate code first and then output the CoT to explain rea- soning steps for code generation more effective. Specifically, our results indicate that a 9.86% rel- ative performance improvement can be achieved simply by changing the order between CoT and code. Our findings provide valuable insights into leveraging CoT to enhance the reasoning capabil- ities of CodeLLMs and improve code generation.
3.5
0.3
0.2
0
200
400
500
300
1200
1200
200
4:00
600
500
1000
1200
Figure 1: The smoothed training loss curve and the average performance across benchmarks of different SFT strategies.
strated significant potential in understanding and generating complex programs (Brown al., 2020; Askell et al., 2021; Dubey et al., 2024; Guo et al., 2025), thereby attracting considerable attention. However, training large-scale LLMs with many learnable parameters is often highly time- and desource-intensive (Han et al., 2024). As result, LLMs need to improve computational efficiency and task perfor- mance when adapting to specific downstream domains (Zhu et al., 2024), including code generation (Austin al., 2021).
Supervised Fine-Tuning (SFT) (Bai et al., 2022; Stiennon et al., 2020) offers a promising solution to enhance the spe- cific capabilities of pre-trained LLMs for downstream tasks, Prior studies (Zhou et al., 2023; Gunasekar et al., 2023) suggest that high-quality training datasets for SFT can lead to significant performance gains with fewer time and re- source requirements. Specifically, Zelikman al. (2022) and Hsieh al. (2023) emphasize that detailed Chain-of- Thought (CoT) (Wei al., 2022) can improve data qual- ity, leading to enhanced performance in complex reasoning tasks, including math (Guan et al., 2025; Ho et al.. 2023), logic (Xu et al., 2024; Huang & Chang, 2023), and com- monsense (Li et al., 2024a; 2023b; 2022). CoT is an ef- fective technique proposed to encourage LLMs to generate intermediate reasoning steps before outputting the final an- swer. SFT with CoT data enables the model to provide more reliable answers through thoughtful consideration and explanation, as empirically (Ye et al., 2024; Yu, 2024) and theoretically (Feng et al., 2023; Liu et al., 2024b) confirmed.
1. Introduction
Code generation (Jiang al., 2024; Roziere al., 2023; Hui et al., 2024) aims automatically generate high-quality, executable programs that satisfy specific problem specifica- tions. Recently, Large Language Models (LLMs), trained on vast amounts of natural language and code, have demon-
NNational Key Laboratory for Novel Software Technology. Nan- jing University, Nanjing, China "School of Artificial Intelligence, Nanjing University, Nanjing, China. Correspondence to: Ming Li <lim@ lamda samda.jju.cd..n22.
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
Code generation task also relies on the model's 's rigorous and complex reasoning ability (Petty et al., 2024; Shen & Zhang,

---

Revisiting Chain-of-Thought in Code Generation: Do Language Models Need to Learn Reasoning before Coding?
2024). However, the research about how LLMs learn to reason with CoT data for code generation in SFT remains limited. Therefore, this work revisits classic CoT training, which typically organizes reasoning steps before the final answer. We aim to investigate how CoT reasoning affects the code generation performance of models in the SFT stage and identify the underlying reasons for these effects.
have emerged as powerful tools in Natural Language Pro- cessing (NLP), advancing language understanding and gen- eration. LLMs (Radford et al., 2019; Askell et al., 2021; Bai et al., 2022) exhibit robust generalization capabilities due to large-scale pre-training. However, due to the distribution gap between code data and the general text corpora (Huang et al., 2024), their code generation performance remains limited, leading to difficulties in logical reasoning and syn- tactic precision (Jimenez et al., 2024; Dou et al., 2024). Consequently, CodeLLMs (Roziere et al., 2023; Bai et al., 2023) are trained for programming tasks through specific optimizations such as syntax-aware learning (Gong et al., 2024), specialized tokenization (Bavarian et al., 2022), and program translation (Du et al., 2025; Li et al., 2025).
Due to the absence of datasets in a unified format, we first construct a diverse, high-quality, and compact dataset that includes natural language descriptions of program ques- tions, reasoning processes, and code solutions. This dataset enables an analysis of the behavior and capabilities of lan- guage models during the SFT phase, providing a robust foundation for our study. We conduct SFT experiments using different base models with the enriched dataset of 50K data pairs. This approach enables independent and ob- jective conclusions unaffected by any specific pre-training dataset. Additionally, we examine key parameters during the SFT process to ensure consistency in our findings across different configurations. Our result in Figure shows that we can significantly improve performance by changing the order between CoT and code. The findings suggest that high-quality code is already suitable act as COT. At the same time, traditional CoT should be regarded as an expla- nation of the code rather than a description of the reasoning process. We provide comprehensive analysis to investigate the behavioral patterns of LLM in code generation and fur- ther examine the proposed findings, In summary, the main contributions of this work are as follows:
Code Generation with LLMs CodeI.LMs, such as Star- Coder (Li et al., 2023c), StarCoder2 (Lozhkov et al., 2024), DeepSeek- (Guo et al., 2024), and Qwen2.5- Coder (Hui et al., 2024) exhibit strong foundational pro- gramming capabilities and excel in code generation. To fur- ther enhance these models, Supervised Fine-Tuniee Jine-Tuning(SFT) has been employed improve task-specific performance and model alignment (Luo et al., 2024; Wei et al., 2023; Zheng et al., 2024). For example, Code Alpaca (Chaud- hary, 2023) adopts the Self-Instruct method (Wang et al., 2023), a technique that utilizes ChatGPT to generate high- quality instruction datasets. In contrast, Vicuna (Peng et al., 2023) is fine-tuned daily conversational datasets collected from a data-sharing platform. Similarly, WizardLM (Luo et al., 2024) utilizes the Evol-Instruct method to enhance instruction datasets iteratively, leading to more complex and diverse tasks. Meanwhile, SelfCodeAlign (Wei et al., 2024) employs a base model to infer aligned data. SFT for code generation is gaining significant attention (Zhu et al., 2024) as enables models to adapt to specialized tasks and generate more precise and context-aware outputs.
Synthetic Dataset We build a high-quality and well- organized dataset of 50K pairs optimized for code gen- eration through seed collecting and data synthesis.
Strategies Study We demonstrate that high-quality code can serve as practical CoT, while traditional CoT should be considered an explanation.
Key Insights We uncover how CoT position and dataset pattern influence performance, providing es- sential insights and confirming our findings.
Chain of thought The performance of LLMs heavily de- pends on the quality of input prompts (Gao et al., 2023b). Inspired by the success of Chain-of-Thougeee (CoT) tech- niques (Wei et al., 2022) in logical reasoning, researchers are exploring their applications in code generation. For example, (Li et al., 2023a) proposes structured CoT ap- proach to help models understand complex intentions and solve challenging problems. Jiang et al. (2023) introduces a self- eelf-planning method consisting of distinct planning and implementation phases. Additionally, Yang et al. (2024a) develops a technique that automatically uses lightweight lan- guage models to generate CoTs for code generation. Chen et al. (2023), Gao et al. (2023a), and Hu et al. (2023) use LLMs to output text and programming language statements, and finally, an answer. Current research primarily focuses on exploring various CoT prompting strategies and their underlying mechanisms (Madaan et al., 2023; Yang et al.,
The remainder of this paper is organized as follows: Sec- tion 2 reviews recent studies relevant to our work. Sec- tion 3 presents the preliminaries required to understand the background. Section 4 provides detailed information about implementing our empirical study, including data collec- tion and training strategies. Sections 5 and 6 present the main results and further discussion, respectively. Section 7 concludes the paper and outlines future work.
2. Related Work
Large language models Large Language Models (LLMs) (Brown et al., 2020; Achiam et al., 2023; Zhao et al., 2023)
2

---

Revisiting Chain-of-Thoughh in Code Generation: Do Language Models Need to Learn Reasoning before Coding?
2024d: Chen al., 2024a; Kudo et al., 2024; Feng et al., 2023). Some recent studies have explored knowledge dis- tillation of CoT from LLMs smaller language models (Magister et al., 2023; Hsich et al., 2023). However, these studies primarily focus on traditional logical tasks, paying limited attention to code generation.
where (%) denotes the binomial coefficient, which repre- sents the number of ways to choose & items from n, while ("x") represents the number of ways to select & incorrect samples from the n - incorrect samples. Under the setup of greedy decoding, Pass@) 1 corresponds to the proportion of problems in the benchmark successfully solved when the model generates a single deterministic output per problem with a temperature of 0.0.
3. Preliminaries
Decoder-only LLM In this work, we focus on a decoder- only large language model (LLM) as the base model for downstream tasks. Given an input sequence x = (21, .2) of length I, the model processes x through multiple transformer-based decode layers to produce hidden states h. The final hidden state h is projected by a language modeling head Linear into logits 2, which are normalized via Softmax function to yield the probability distribution p over the vocabulary for the next token:
4. Method
In existing open-source SFT datasets, CoT and Code com- ponents are commonly integrated into responses to code generation tasks. Inconsistent formats hinder the evalua- tion of different components' contributions to model perfor- mance. This section outlines our synthetic dataset to address these limitations. Our objective is to develop well-organized datasets designed to enable controlled experiments and ana- lyze the impact of SFT on CodeLLM'ss performance.
exp(z;)
p= = mxx) and Pj
IV 2==1 exp(zx)
4.1. Data collection
where | is the vocabulary size, and 2j denotes the j-th logit. This mechanism enables LLM to generate the next token based on the input context, supporting various tasks.
Previous studies often use code snippets from GitHub (Wei et al., 2023) or Common Crawl (Yang et al., 2024c) as seed data during the SFT stage of large code models, guiding the models in generating ever, taking raw code snippets as seed data is often un- likely create problems that are sufficiently challenging, diverse, and representative of instruction distributions (Song et al., 2024). Consequently, the solutions usually lack comprehensive reasoning steps and high-quality code solu- tions. To address this issue, we collect and filter program tasks from open-source datasets to construct a seed set, in- cluding Magicoder- InS-Instr (ISE-UIUC, 2024), the Python Code subset of ShareGPT (Innovations, 2023), Evol- CodeAlpaca (Tam, 2024), Evol-Instruct-Code (Roshdieh, 2023),
intructiio-answeer pairs. How-
Supervised Fine- TunnnT Supervised Fine- TineiT (SFT) essential for adapting I.IM to specific applications. The SFT process can be described as follows: Given a labeled dataset D = ((xx(), (((()))))))) | where (x(() is the input se- quence and y is the corresponding output with length 4, the goal to minimize the loss function L over the dataset:
D Rd
0- arg min
ΣΣ
c ( 1=
(2) (vit1, Softmax(Liieear(M (9) (h))))))
where 0 represents the model parameters, 1,(5) is the final hidden state for the context sequence (a(() + v/ and Ut+1 is the encoded token at timestamp f. The loss function [(v.p) - و Mj log(p,) is typically cross-entropy loss, which measures the difference between the predicted probability distribution p and the true one-hot label y.
TerExxercise-Pthhor (CodeFuse-Al, 2023a), and Codefuse-Evor InstruIntt (CodeFuse-Al, 2023b), as recom- mended by prior research. We put the detailed statistical information of these used datasets in Appendix C.
4.2. High-Quality CoT and Code
We start with an initial seed dataset consisting of program questions x and their corresponding responses y. repre- sented as S = { 11 As outlined in (Bai et al., 2022; Snell et al., 2022), we adopt the method of context distillation, which takes a general model M, as teacher to synthetic data, using it to SFT base student model M,. The teacher model can follow instructions to generate contextu- ally relevant outputs and align them with specific behavioral objectives. It reduces the time and cost of labeling super- vised data, requiring minimal human intervention.
Pass@k For evaluating CodeLLM performance, we mainly measure the likelihood that a model generates at least one correct code sample out of k attempts for given problem with the metric Pass@k= = 1 - (1 p)², where p represents the probability of generating correct sample, from (Chen et al., 2021; Kulal et al., 2019). To mitigate the computational challenges of directly calculating Pass@k, an unbiased estimation method is used:
Pass @k
Our technique starts with a small set P of examples where
3

---

Revisiting Chain-of-Thought in Code Generation: Do Language Models Need to Learn Reasoning before Coding?
[P] = < (S]. By drawing inspiration from (Zelikman et al., 2022), we provide the original (Izz 1/) the hint to prompt teacher model Mr, sample multiple times and then select the best CoT and Code to include in the few shots set P. Building on the In-Context Learning method proposed in (Huang et al., 2022), we use a few-shot prompt from P and each seed from S, calling M, to generate detailed reasoning steps ry and a high-quality code solution e; for (z,, y1). The prompt template is in Appendix B. We assume incorrect reasoning solutions may introduce noise into synthetic datasets, potentially hindering model learning. Therefore, we use the Self-Consistency (Wang et al., 2023) to drop part of the generated data, retaining only those that produce the correct answer. Specifically, we prompt the model M, to generate multiple test cases, selecting only the o that suc- cessfully pass the test code. We use the most powerful open- source model currently available, DeepSeek- Vo.rk--22.--12-- 1.10 ' as the teacher to construct our synthetic dataset. This pro- cess ensures that code snippets are syntactically valid and consistent, with reasoning standardized and aligned to the code content. The details can be found in Appendix A.
5. Experiments
We conduct experiments on several code generation bench- marks evaluate the performance under different SFT strategies. The benchmarks used in our experiments are exhaustively described in Appendix D. We use OpenCodeE- val (Liu, 2024) our evaluation framework. Most experi- CrepSeek-Coder mental results are derived from training Base-6.7B 7B (Guo et al., 2024) for 3 epochs. The learning rate dynamically adjusted using a cosine decay scheduler with a warm-up ratio of 0. and a peak value of le -. 5. The maximum sequence length is fixed at 4096 tokens during training. More information can be found in Appendix E.
5.1. Evaluation Results
We use the EvalPlus benchmark (Liu et al., 2023) to evaluate the code generation capabilities of fine-tuned models under different strategies. As shown in Table 1, Gresede has 9.86% relative performance improvement compared with Cfallow-
Table I:
The performance of different SFT models on code generation by HumanFval, MBPP, and extended versions.
4.3. Different Strategies for SFT
We consider a base model M, with a filtered dataset D (dr,, yo, ri,, [) and design four distinct data strategies and utilize them. During SFT, the model takes the problem description as input, with output varying by dataset format.
Method
HumanEval(+)
MBPP(+)
Average
Seed
68.29(62.20)
77.51(65.61)
68.40
Owio
70.73(64.63)
80.42(67.72)
70.88
Cilllow
67.07(59.75)
(4.33660.58)
65.43
Opeccede
71.95(67.68)
80.69(67.20)
71.88
Seed Dataset The initial dataset consists of questions and their corresponding response y. represented as {( |i) This dataset forms the foundation for training and can serve as the baseline, denoted as Seed.
We further benchmark our SFT models using Live- CodeBench (Jain et al., 2024) to ensure a fair comparison. Notably, this benchmark enables evaluation of the models' out-of-distribution (OOD) generalization, including evolv- ing test subsets reflecting temporal shifts in data. Table presents Pass @ results for subsets generated after different start dates to analyze temporal performance trends.
|D]
Code without CoT In the dataset ((ri, 3 where CoT is excluded, the data format is more standardized as generated by the same model M,. The model is specifically trained to output the Code, denoted as Owis-
Table 2: The performance of different SFT models on Live- CodeBench. Newer Start Dates Reflect Lower Contamina- tion for Evaluating OOD Generalization.
Code follow CoT In the dataset {Z Ti + = the model is fine-tuned first to generate the CoT based the problem, followed by the corresponding code solution. This approach adheres to the CoT prompting methodology com- monly used in LLMS and is denoted as Croloow-
Contest Start date
Method
2023-09-01
2023-07-01
2023-05-01
Seed
20.12
21.77
23.89
Cwto
20.47
21.67
23.51
Croloow
20.69
22.31
23.07
Cprecedo
21.08
22.56
24.16
Code precede CoT In contrast to the previous approach, the dataset {(,,, Q + [D] is used fine-tune the model first to generate the code solution, followed by the corre- sponding COT. This less conventional method takes CoT as an explanation and is denoted as Cprecod.
To deeply understand the advanced capabilities of our fine-tuned model, we extend our evaluation by using Big- CodeBench (Zhuo et al., 2024). This allows us to specif- ically examine its performance in two crucial areas: fol- lowing complex natural language instructions and calling
'https://platiee http://aa...rr..deeseeeeeekk deepseek.com
4

---

Revisiting Chain-of-Thought in Code Generation: Do Language Models Need to Learn Reasoning before Coding?
DeepSeek-Coder-6.78
Llama-3.1-2
Qwen2.5-Chate
Swen2.5-Coder-78
72
60
Average Pass@1(9%
58
56
66
72
64
70
62
60
120
1220
0012
1002
1200
100
1200
1200
1000
1200
Seed Dataset
Code without CoT
Code follow CoT
Code precede CoT
Figure 2: The average performance various base models fine-tuned under different strategies across steps (x-axis).
external function tools. As shown in Table 3, the primary sults are explicitly derived from the benchmark's instruction split, isolating the model's ability to interpret and execute given instructions accurately.
Model performance is evaluated at checkpoints every 100 training steps. The results EvalPlus, as shown in Fig- ure 2, reveal that the base model has notable effect on SFT outcomes; this observation aligns with findings widely reported the literature. Although a distributional gap may inevitably exist between our high-quality SFT data and the pre-training data, different models still benefit from explana- tion after coding and avoiding overthinking. We also discuss other key hyperparameters of SFT in Appendix H.
Table The performance of different SFT models on Big- CodeBench. Different task results obtained for the Full set and more challenging Hard subset.
Task Set
Method
Full
Hard
5.3. Study of Teacher Model
Seed
37.71
13.51
To examine whether our conclusions remain valid when synthetic data is generated using different teacher models, we employ GPT-4o-0806 as alternative teacher model to regenerate the dataset to validate our approach further. We then fine-tuned the same student model using the methodolo- gies outlined above. The models are subsequently fine-tuned and evaluated under the same settings described above. The results, detailed in Table 6, demonstrate consistent trend.
Gwlo
38.25
15.54
Clollow
36.05
15.54
Cpeceede
38.33
16.22
We place the other results in Appendix F and significance analysis Appendix Our experimental results show that teaching the model to write CoT before code does not benefit explanation
downstream tasks, but if we treat CoT of the code, the model performs better. The classic CoT training method, which typically organizes reasoning steps before generating a solution, does not work well on code generation tasks. The code is reasoning process, while latter CoT helps to understand. These findings highlight the importance of the CoT position and verify that the code itself is reasoning process.
Table 4: The performance of different SFT models on the EvalPlus benchmark. These models are fine-tuned on an- other dataset synthesized by GPT-4o-0806.
Method HumanEval(+)
MBPP(+)
Average
Owio
71.95
77.77
70.76
Cialton
65.24(59.
76.45(62.96)
65,95
Opeevede
72.56(66.46)
78.57(67.72)
71.33
5.2. Study of SFT Configurations
5.4. Study of Synthesis Order
This study examines whether our conclusions hold across different pre-training datasets and model architectures by varying the base models. The choice of the base model is critical, it determines the initial capabilities and founda- tional architecture of language models, which in turn affects downstream task performance. The experiment involves tne-tuning various base models of different sizes and archi- tectures while keeping all other hyperparameters constant.
To mitigate the bias introduced by providing reference code during data synthesis, modify data generation ap- proach to investigate the synthetic order's impact and ensure that the teacher model must generate CoT reasoning without access to code-related information. Instead of providing a reference solution, we directly utilize educational instruc- tions from the SelfCodeAlign (Wei et al., 2024) dataset to
5

---

Revisiting Chain-of-Thought in Code Generation: Do Language Models Need to Learn Reasoning before Coding?
synthesize data. The results are listed in Table 5.
balance the learning in the two parts. These differences arise because source code has stricter syntax and semanties than COT. Therefore, some tokens are easily inferred based on grammatical rules when generating Code. In contrast, CoTI' permits greater flexibility and may cause overthinking behavior (Chen et al., 2024b) when generated first.
Table 5: The performance of different SFT models on the EvalPlus benchmark. These models are fine-tuned on an- other dataset synthesized by different order.
Method
HumanEval(+)
MBPP(+)
Average
Cole
67.07(60.97)
76.71(60.31)
66.27
Code Tataal
Code
Ciallw
62.19(54.87)
71.95(57.93)
61.74
Acasy Aacroou
Total
Cpeecede
68.29(60.36)
78.57(62.16)
67.35
Acass 0,31
0.55
0.00
5.5. Study of Data Source
To investigate whether our conclusions can be applied to the pre-training stage, we utilize Stack (Kocetkov et al., 2022) to synthesize data as an additional data source, which is a 3.1 TB dataset of permissively licensed source code in 30 programming languages, designed to foster open and responsible research on CodeLLMs. Subsequently, the mod- els are trained and tested following the previously outlined setup. Table 6's results reveal clear and consistent pattern.
Figure 3: Conditional PPL Distribution Histogram with Gaussian Fit: Cpesode (left) and Ciollow (right), representing learning difficulty gap on different parts in the dataset.
Memorization and Generalization To further investi- gate the impact of different orders of context on learning efficiency. we analyze the model's memorization and gener- alization before and after training. Figure 4 compares two SFT strategies by evaluating them on the validation set. The left plot depicts the KI. divergence between the fine-tuned models (after training) and the base model (before training) under two distinct data strategies. The minor difference in KI. divergence suggests that both SFT strategies modify the base model similarly. Conversely, the right plot shows the validation loss, reflecting the accuracy of the next-token pre- dictions. The notable difference in this plot shows that the two SFT strategies leverage the data in distinct ways. The results suggest that while both strategies fit the training set similarly, the Gpeeede strategy exhibits significantly better generalization, as reflected in code generation ability.
Table 6: The performance of different SFT models on the EvalPlus benchmark. These models are fine-tuned on other dataset synthesized from the Stack's 's codes.
Method
HumanEval(+)
MBPP(+)
Average
Cuu
68.29(61.58)
76.98(62.43)
67.32
Ctatow
61.58(55.48)
75.13(60.31)
63.13
Greeede
69.51(64.63)
77.24(63.22)
68.65
6. Discussions
This section analyzes several key aspects of model behavior under different strategies: 1. The factors influencing perfor- mance variations when changing the position of CoT in the training. 2. The specific data components that enhance the model's capability in code generation tasks during the SFT process. 3. The generalizability of our proposed conclusions across different scenarios. 4. The distinctive characteristics of outputs generated by various SFT models.
Gprcass
Gpecses
Opton
Cytor
0.13
A
Loss
0.07
33
35
2.
620
6.13
015
0.5
1.50
2.55
Figure 4: The Distribution Histogram Plot with Gaussian Fit lines: the KL Divergence between Base and SFT Mod- els (left) and the Validation Loss (right).
6.1. Model Behavior Analysis
Conditional Perplexity Gap In our experiments, we an- alyze how the order of information affects the difficulty of learning data (Li et al., 2024c;b) by examining the distribu- tions of perplexity across two strategies: Cpecede and Crollow- We visualize the perplexity of the preceding part in both setups and analyze the conditional perplexity of the latter part, considering the former as its context. As shown in Figure 3, while the overall perplexity distributions are simi- lar, placing the Code first reduces the gap between the two distributions. The result suggests that the model can better
Context Attention Weight In our experiments, we in- vestigate the effect of information ordering on the model's attention weights, specifically examining the influence of natural language instructions, CoT, and Code. We apply bilinear interpolation to downsample the average attention weight matrix from the model's final layer to a fixed spatial resolution, ensuring consistent comparison across inputs with different orderings. The visualization in Figure indi-
6

---

Revisiting Chain-of-Thought in Code Generation: Do Language Models Need to Learn Reasoning before Coding?
cates that Code exhibits no distinct attention bias towards COT, with attention values evenly distributed across the re- gions corresponding to Code segments. However, when Code precedes CoT, we observe that the model allocates in- creased attention to Code, potentially indicating an attempt to better comprehend its relationship with COT. These re- sults suggest that employing the Crollow strategy during SFT alters the model's expectations regarding the underlying data distribution. Consequently, enforcing fixed attention order between CoT and Code might hinder the model's s abil- ity to adapt to these constraints. This misalignment could subsequently impair the model's generalization capabilities on downstream tasks.
Qami
Qpm
0.6
Kpri
Kpo)
Gradient Narm Value
Viri
Vire
Ozni
Open
0.4
0.0
Figure 6: The l2 norm of gradients across different layers (x- axis) under two parts: CoT (left) and Code (right).
when compared to models trained on self-consistent data. These results suggest that consistency is not the sole deter- minant of code quality, particularly given that LLMs often exhibit limited proficiency in generating accurate test cases, Even Code containing errors can offer valuable insights and direction for the model's learning process. These findings underscore that high-quality Code, even imperfect, accom- panied by detailed generation process, serves as valuable training signal for CodeLLMs.
Split between NL? and Ords Spti beteeon and
Solt between and GCT Soll between and
Figure 5: The Visualization Results of Compressed At- tention Weight Matrices in the last layer under Different Strategies: Cpeecede (left) and Clollo (right).
Seed Dataset
Layer Gradient Norm In this subsection, aim to ana- lyze the gradients in the dynamic training process of LLMs, particularly their scaling and distribution across layers. We investigate the gradient across different L.MM layers under various data parts while maintaining the same initial model state. We primarily analyze the gradients attention-relatee layers, including QPooj. KPooj. VProj. and OProj. Given the im- mense number of layers and parameters in current LLMs, it is computationally infeasible to analyze these large gradient matrices directly. Therefore, we use the & norm to capture the gradient characteristics of each layer, particularly its strength. Figure 6 shows that different data have different impacts on the parameters, and as we mentioned earlier, the overall impact of the code part relatively weak. These findings highlight the significant effect of training order on gradient contributions.
Code without Col
Ramove
Code follow
Ramore Covmart
Code precede Coll
Rertue Sipestur
300
dl
do
tsks
12bs
230
436"
1605
1200
Figure 7: The average performance on the EvalPlus bench- mark of two studies: inconsistent data study (left) and sig. nature removal study (right).
Signature Removal Analysis Our experiments this sub- section investigate whether annotation components within the Code significantly affect the model's ''s understanding and learning capabilities. Specifically,
examine forms of annotations: function or class signatures and inline code comments. We analyze how removing these annotation ele- ments from the dataset impacts model performance across various benchmarks. The right plot of Figure 7 illustrates that removing signatures leads to the most significant perfor- mance decline throughout training, highlighting the critical role of signatures in facilitating the model's learning process. These findings suggest that function class signatures crucial in code generation, as they bridge the gap between natural language and programming languages (Yang et al., 2024b). Conversely, removing inline comments does not appear to have a significant impact, likely due to different code styles of L.MMs.
6.2. Data Pattern Discussion
Inconsistent Data Analysis In our experiments, we ana- lyze the impact of inconsistent Code, defined Code that fails to pass its self-generated test cases, on model perfor- mance under various strategies. As depicted in the left plot of Figure 7, the results indicate that conclusions from our previous study on various data strategies remain valid; the model exhibits only a marginal decrease in performance

---

Revisiting Chain-of-Thought in Code Generation: Do Language Models Need to Learn Reasoning before Coding?
Mixed Pattern Analysis This study examines whether LLMs benefit from mixed-data strategy, specifically investigating whether diverse data distributions enhance model generalization. To this end, we combine the Crollow and presede strategies and fine-tune the base model for one epoch. For a fair comparison, we apply these strategies separately for two epochs, ensuring an equivalent number of training steps. The results presented in the left plot of Figure 8 reveal that the mixed-data strategy yields lower performance than single strategy. The experiment suggests that the mixed-data strategy may be more challenging for the model to learn, implying an increase in the overall difficulty of the dataset from the model's 's perspective. Presenting an LLM with two different answers for the same problem establishes two distinct optimization objectives.
initial analysis remain consistent across these larger archi- tectures. These findings highlight the scalability and general applicability of the insights derived from our experiments, thereby reinforeing the importance of thoughtful dataset con- struction for maximizing performance in large-scale models.
Batsot
Seed barosr
precade
procede
Figure 9: The average performance on the EvalPlus bench- mark of large size:
70
70
Coder-33B (left) and Qwen2.5-Coder- Cooder-3B--Baaa (right).
44-
69
Generalization Across Difficulties To explore whether the Cpreced: approach generalizes effectively to more com- plex problems, we design an experiment to assess the robust- ness of this data strategy in challenging scenarios. Table 7 reveals that incorporating CoT explanation consistently im- proves model performance, particularly for tasks requiring higher logical complexity. Furthermore, the relative posi- tioning of the CoT part demonstrates measurable impact on outcomes, underscoring the importance of data strategy. These findings suggest that considering the CoT position into SFT effective and exhibits strong potential for seal- ability to more complex problems. Our method offers a robust framework for enhancing reasoning in code genera- tion tasks.
04
Mix Code and CoT
without GoT
Code follow CoT
Attacked Code Solution
43
Code pracede CoT
Original Code Solution
160 20o 300
140
800 "02"" 1003"
20
do
ods
uee
Figure& 8: The average performance the EvalPlus bench- mark of two studies: mixed pattern dataset (left) and at- tacked code dataset (right).
Adversarial Code Robustness Previous studies suggest that LLMs learn programming languages by processing Code as sequences of tokens and are easy to attack due to their strong dependence on lexical features. Therefore, we design a lexical-level attack on code snippets that preserves syntax and semantics investigate the impact of training model manipulated data. Specifically, replace all vari- able names in the Code with random, meaningless strings, thereby limiting the model's access to lexical information. The right plot of Figure 8 illustrates an assessment of model robustness under adversarial conditions by comparing per- formance on original solutions with that on the attacked Code. Although an upward trend is observed with increas- ing training steps, performance on the attacked solutions remains consistently lower, suggesting that the model has difficulty mitigating the effects of adversarial noise, a find- ing consistent with (Yan & Li, 2021).
Table 7: The performance of different strategies on the LeetCode Contest Benchmark with three difficulty levels.
Method
Easy(45)
Medium(91) Hard(44)
Overall(180)
Seed
42.2%(19)
15.4%(14)
9.1%(4)
20.6%:(37
Culo
42.2%.(19
16.5%(15)
2.3%(1)
19 19.4%)(35)
Chatoon
31.1%((4)
14.3%((33
4.5%((2
16.1% (29)))
Cpncode
44.4%
24.2%((2)
6.8%(3)
25.0%
Discuss Pass@k Impact Different values reflect differ- ent aspects of model performance, which reveals different evaluations of model behavior and potential. Figure 10 left illustrates the effect of varying k in Eq. on the average per- formance across different benchmarks. As k increases, all strategies demonstrate improvement in accuracy, indicat- ing models exploring more possibilities and increasing the chances of producing correct outputs. The results show that Cpecode almost outperforms all other strategics, highlighting the importance of positioning the CoT explanations.
6.3. Generalization Discussion
Generalization Across Sizes To assess the robustness of our findings, we extend the experimental setup to include two larger language models: DeepSeek-Codee Core--Coorr-BB.. -Bss and Owen2.5-Coder-322. Base. The results presented in Figure demonstrate that the conclusions drawn from our
8

---

Revisiting Chain-of- Thoought in Code Generation: Do Language Models Need to Learn Reasoning before Coding?
in Table 9. Similarly, we report the CodeBLEU score for code outputs assess their syntactie and semantic similarity to the expected Code. The results indicate minimal differ- ences between the two models in terms of BLEU. However, presede can achieve a marginally higher CodeBLEUU score for the code outputs with fewer steps and tokens, which means it gets better results with less inference cost.
82
Seed Dataset
Seed Dataset
Code without Col
Code without Col
47
Code follow CoT
Code follow COT
CaT
Table 9:
Comparison of CoT and Code parts' length and quality under different SFT strategies.
Figure 10: The average performance on the EvalPlus bench- mark of different in Pass (left) and temperature (right).
CoT Tokens
Code Tokens
Method
Steps
BLEU
Steps
Co
Discuss Temperature Impact When LLMs generate the answers, the temperature affects the model's performance, with low temperatures tending to make the model's out- put more fixed and high temperatures encouraging more exploration in action. In the right plot of Figure 10, we aim to show the effect of varying the temperature parame- ter. As the temperature increases, the overall performance tends to decline across all strategies, with the steepest drop observed in presede- The result highlights the trade-off be- tween diversity and accuracy, as higher temperatures lead to more diverse but less precise outputs. Cprecede maintains relatively stable performance, reinforcing its robustness.
Croioow Oprecede
5.05
197.99
60.34
13.02
258.94
68.19
5.02
195.46
60.22
12.02
256.97
68.96
Response Match Instruction We analyze the similarity between the instruction and two other components: the Code and the COT. Specifically, we use OpenAl's embed- ding model eoxt-embedding-3-larrge to obtain the represen- tation of the text and then compute the cosine similarity between them. This evaluation assesses the alignment and coherence between the instructions and the corresponding outputs, including the executable Code and the reasoning processes in the COT. By comparing these similarities in Figure 11, we aim to gain insight into the degree of semantic match correspondence and the extent to which the generated outputs adhere to the given instructions.
6.4. Model Outputs Discussion
LLM-as-a-Judge To address the high cost of collecting human preferences (Chiang et al., 2024), we adopt a pair- wise comparison approach, where an LLM evaluates ques- tion alongside two answers and determines which response is of higher quality. We prompt GPT-4o-0806 to assess dif- ferent parts of the responses based on specific criteria. Each row represents a different evaluation part, and each column represents the winner, the response the evaluator thinks is better. We change the response order and conduct evalua- tions twice enhance efffectiveness. The results in Table 8 indicate that the precede approach aligns more closely with real-world preference on all three parts.
Cpraess Gyeww
Gyneads
Gralo
A
0.01
646
0.2
0.26
1.00
0.55
436
660
053
470
Figure 11: Cosine Similarity Distribution with Gaussian Fit: Instruction and Code (left), Instruction and CoT (right).
Table 8: Result of LLM AaMMaas---in Judgee of different strategies.
7. Conclusion
In this study, we investigate the impact of incorporating CoT reasoning into the SFT process to improve code gen- eration in LLMs. By constructing a high-quality dataset, we facilitate comprehensive experimentation across a range of base models and configurations. Our results show that high-quality code can already serve as reasoning process, while traditional CoT should be regarded as an explana- tion of the code. Additionally, we gain key insights into how CoT strategies and data patterns influence fine-tuning outcomes, providing practical guidelines for future studies. These findings highlight the effectiveness and scalability of CoT strategies SFT for the code generation model.
Winner
Collow
Tie
Cpeecdee
CoT Code
22.1%(242)
52.8%(577)
25.1%((74)
20.1%(228)
51.0%(5557
28.2%
Total
23.2%
51.2%)(560
25.5%(279)
Length and Quality The study presents comparative analysis of the outputs generated by Cfollww and Cpreode concerning the CoT and the Code in outputs. Besides the number of tokens and steps, we report the BLEU score for CoT, which measures alignment with the teacher output

---

Revisiting Chain-of-Thought in Code Generation: Do Language Models Need to Learn Reasoning before Coding?
Acknowledgements
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan,
J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G.,
This paper supported by NSFC (62076121) and the Major Program (JD) of Hubei Province (2023BAA024). The au- thors thank Dr. Wang-Zhor Dai and Hao-Yur Yuan He for their insightful discussions regarding this work.
Askell, A., Agarwal, S., Herbert- A., Krueger, G., Henighan. T., Child, R., Ramesh, A., Ziegler, D. M., Wu,
J., Winter, C., Hesse, C., Chen, M., Sigler, E.,, Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., and Amodei, D. Language models are few-shot learners. In Proc. of NeurIPS, 2020.
Impact Statement
Our work provides critical insights into how LLMs learn to reason through CoT code generation, challenging conven- tional training paradigms. Beyond immediate performance gains, offer novel insights into how LLMs learn to reason with CoT data for code generation, revealing that restructur- ing the order significantly improves performance. Further- more, our exploration of the properties of CoT training and data strategies contributes to advancing reasoning method- ologies in LLMs, potentially shaping the development of effective and scalable AI-driven software engineering.
Cassano, F., Gouwar, J., Nguyen, D., Nguyen, S., Phipps- Costin, L.,, Pinckney, D., Yee, M.-H., Zi, Y., Anderson, C. J., Feldman, M. Q. et al. Multipl-e: A scalable and extensible approach to benchmarking neural code genera- tion. ArXiv Preprint, abs/2208.0822 2022.
Chaudhary, S. Code alpaca: An instruction-followiinn llama model for code generation. Gitllub repository, 2023.
Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. D. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al. Evaluating large language models trained on code. ArXiv Preprint, abs/2107.033 .33344, 2021.
References
Abuduweili, A. and Liu, C. Revisiting the initial steps in adaptive gradient descent optimization. ArXiv Preprint, abs/2412.02153,, 2024.
Chen, Q. Qin, L., Wang. J., Zhou, J., and Che, W. Unlock- ing the capabilities of thought: A reasoning boundary framework to quantify and optimize chain-of-thought. In Proc. of NeurlPS, 2024a.
Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. Gpt-4 technical report. ArXiv Preprint, abs/2303.08774, 2023.
Chen, W., Ma, X., Wang, X., and Cohen, W. W. Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks. Transactions on Machine Learning Research, 2023.
Askell, A., Bai, Y., Chen, A., Drain, D., Ganguli, D., Henighan, T., Jones, A., Joseph, N., Mann, B., DasSarma, N., et al. A general language assistant as a laboratory for alignment. ArXiv Preprint, abs/2112.00866. 2021.
Chen, X., Xu, J., Liang, T., He, Z., Pang, J., Yu, D., Song, L., Liu, Q., Zhou, M., Zhang, Z., et al. Do not think that much for 2+ 3=? on the overthinking of ol-like 1lms. ArXiv Preprint, abs/2412.211 187, 2024b.
Austin, J., Odena, A., Nye, M., Bosma, M., Michalewski, H., Dohan, D., Jiang, E., Cai, C., Terry, M., Le. Q. al. Program synthesis with large language models. ArXiv Preprint, 00772108.07732, 2021.
Chiang, W., Zheng, L., Sheng, Y., Angelopoulos, A. N., Li, T., Li, D., Zhu, B., Zhang, H., Jordan, M. I., Gonzalez, J. E., and Stoica, I. Chathot arena: An open platform for evaluating Ilms by human preference. In Proc. of ICML. PMLR, 2024.
Bai, J., Bai, S., Chu, Y., Cui, Z., Dang, K., Deng, X., Fan, Y., Ge, W., Han, Y., Huang, F. et al. Qwen technical report. ArXiv Preprint, abs/2309.16609, 2023.
CodeFuse-AL. URL. https: Codeexercise-pyrhon227k 2023a. code modefuse-ai//e aateffueeeea//cceeeeeeeeeeeeeeee 27k. Accessed: 2023-12-20.
co/datasets
Bai, Y., Jones, A., Ndousse, K., Askell, A., Chen, A., Das- Sarma, N., Drain, D., Fort, S., Ganguli, D., Henighan, T., et al. Training helpful and harmless assistant with rein- forcement learning from human feedback. ArXiy Preprint, abs/2204.05862, 2022.
CodeFuse-Al AL. Evol-insttuction-666. 2023b. URL
https / /
co/datasets/ http:/oooooissstttioooooooeeoeoe 66k. Ac- cessed: 2023-11-30.
Bavarian, M., Jun, H., Tezak, N., Schulman, J., McLeavey, C., Tworek, J., and Chen, M. Efficient training of lan- guage models to fill in the middle. ArXiv Preprint, abs/2207.14255. 2022.
Dao, T., Fu, D. Y., Ermon, S., Rudra, A., and Ré, C. Flashat- tention: Fast and memory- exact attention with io-awareness. In Proc. of NeurlPS, 2022.
10

---

Revisiting Chain-of-Thought in Code Generation: Do Language Models Need to Learn Reasoning before Coding?
Defazio, A., Cutkosky, A., Mehta, H., and Mishchenko, K. When, why and how much? adaptive learn- ing rate scheduling by refinement. ArXiv Preprint, abs/2310.07831, 2023.
Han, Z., Gao, C., Liu, J., Zhang, J., and Zhang, S. Q- Parameter-effiiien fine-tuning for large models: A com- prehensive survey, ArXiv Preprint, 1a4/2243.114608 2024.
Ho, N., Schmid, L.,, and Yun, S.-Y. Large language models are reasoning teachers. In Proc. ACL, pp. 14852-14882 Association for Computational Linguistics, 2023.
Dou, S., Jia, H., Wu, S., Zheng, H., Zhou, W. Wu, M., Chai, M., Fan, J., Huang, C., Tao, Y., et al. What's wrong with your code generated by large language models? an extensive study, ArXiv Preprint, abs/2407.06153, 2024.
Hoffer, E., Hubara, I., and Soudry, D. Train longer, general- ize better: closing the generalization gap in large hatch training of neural networks. In Proc. of NeurIPS, pp. 1731-1741, 2017.
Du, Y., Sun, H., and Li. M. Post-incorporating code struc- tural knowledge into llms via in-context learning for code translation. ArXiv Preprint, abs/2503.22776 2025.
Hsieh, C.-Y., Li, C.-L.,, Yeh, C.-k., Nakhost, H., Fujii, Y., Ratner, A., Krishna, R., Lee, C.-Y., and Pfister, Distill- ing step-by-step! outperforming larger language models with less training data and smaller model sizes. In Proc. of ACL Findings, pp. putational Linguistics. 2023.
Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A. et al. The Ilama 3 herd of models. ArXiv Preprint, abs/2407.21783, 2024.
Association for Com-
Feng, G., Zhang, B., Gu, Y., Ye, H., He, D., and Wang, L. Towards revealing the mystery behind chain of thought: A theoretical perspective. In Proc. of NeurlPS, 2023.
Hu, Y., Yang, H., Lin, Z.,, and Zhang, M. Code prompting: neural symbolic method for complex reasoning in large language models. ArXiv Preprint, abs/2305. 2023.
Gao, L., Madaan, A., Zhou, S., Alon, U., Liu, P., Yang. Y., Callan, J., and Neubig. Pal: Program-aide aided language models. In Proc. of ICML. PMLR, 2023a.
Huang. J. and Chang, K. C.-C. Towards reasoning in large language models: survey. Proc. of ACL Findings, pp. 1049-1065, Association for Computational Linguistics, 2023.
Gao, S., Wen, X.-C., Gao, C., Wang, W., Zhang, H., and Lyu, M. R. What makes good in-context demonstra- tions for code intelligence tasks with Ilms? In 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE), pp. 761-773. IEEE, 2023b.
Huang, S., Cheng, T., Liu, J. K., Hao, J., Song, L., Xu, Y., Yang. J., Liu, J., Zhang, C., Chai, L., et al. Opencoder: The open cookbook for top-tier code large language mod- els. ArXiv Preprint, abs/241 1.04909 2024.
Gong. L., Elhoushi, M., and Cheung, A. AST-T5: structure- aware pretraining for code generation and understanding. In Proc. of ICML. PMLR, 2024.
Huang, Y., Chen, Y., Yu, 7., and McKeown, K. In-context learning distillation: Transferring few-shot learning abil- ity of pre-trained language models. ArXiv Preprint, 0os/2212.106111 2022.
Guan, X.. Zhang, L. L., Liu, Y., Shang, N., Sun, Y., Zhu, Y., Yang, F,, and Yang. M. rstar-math: Small llms can master math reasoning with self-evolved deep thinking. ArXiv Preprint, abs/2501.0451/ 2025.
Hui, B., Yang, J., Cui, Z., Yang, J., Liu, D., Zhang, L., Liu, T., Zhang, J., Yu, B., Lu, K., et al. Qwen2.5-coder technical report. ArXiv Preprint, abs/2449.121888 2024.
Gunasekar, S., Zhang, Y., Aneja, J., Mendes, C. C. T., Del Giorno, A., Gopi, S., Javaheripi. M., Kauffmann, P., de Rosa, G., Saarikivi, O., et al. Textbooks are all you need. ArXiv Preprini, abs/2306.1 1644, 2023.
Innovations, F. URL Llibawa-20233 10023//yyhon-Cooe-220 S
Python-codee22kksharegg 2023.
Accessed: 2023-11-11.
Guo, D.. Zhu, Q. Yang, D., Xie, Z., Dong, K., Zhang, W., Chen, G., Bi, X., Wu, Y., Li, Y., et al. Deepseek-codd When the large language model meets programming-tit rise of code intelligence. ArXiv Preprint, abs/2401.14199 2024.
ISE-UIUC. Migicoddrross--nssmmtt777.. 2024. URL https://pugginggfacccc//aaaaaa Accessed: 2024-05-25.
Jain, N., Han, K., Gu, A., Li, W.-D., Yan, F., Zhang, T., Wang, S., Solar-Lezama, A., Sen, K., and Stoica, I. Livecodebench: Holistic and contamination free eval- uation of large language models for code. ArXiv Preprint, abs/2403.07974 2024.
Guo, D., Yang, D., Zhang, H., Song, J., Zhang, R., Xu, R., Zhu, Q., Ma, S., Wang, P., Bi, X., et al. Deepseek-rl: In- centivizing reasoning capability in Ilms via reinforcement learning. ArXiv Preprint, abs/2501. 22501.12949 12948, 2025.

---

Revisiting Chain-of-Thought in Code Generation: Do Language Models Need to Learn Reasoning before Coding?
Jiang, J., Wang, F., Shen, J., Kim, S., and Kim, S. A survey large language models for code generation. ArXiv Preprint, abs/2406.00015,, 2024.
Starcoder: may the source be with you! ArXiv Preprint, abs/2305.06161, 2023c.
Li, S., Chen, J., Shen, Y., Chen, Z., Zhang, X., Li, Z., Wang. H., Qian, J., Peng, B., Mao, Y., et al. Explanations from large language models make small reasoners better. ArXiv Preprint, abs/2210.06722 2022.
Jiang. X., Dong, Y., Wang, I.., Zheng. F., Shang, Li G., Jin, Z., and Jiao, W. Self-planning code generation with large language models. ACM Transactions on Software Engineering and Methodology, 2023.
Li, X.-Y. Du, Y.-L.,, and Li, M. Enhancing Ilms in long code translation through instrumentation and program state alignment. ArXiv Preprint, abs/2504.02017, 2025.
Jimenez, C. E.,, Yang. J., Wettig, A., Yao, S., Pei, K., Press, O., and Narasimhan, K. Swe-bench: Can language mod- els resolve real-world github issues? In Proc. of ICLR. OpenReview.net, 2024.
Liu, J., Xia, C. S., Wang, Y., and Zhang, L. Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation. In Proc. of NeurIPS, 2023.
Kocetkov, D., Li, R., Allal, L.. B., Li, J., Mou, C., Ferrandis, C. M., Jernite, Y., Mitchell, M., Hughes, S., Wolf, T., et al. The stack: tb of permissively licensed source code. ArXiv Preprint, abs/2211.1553, 2022.
Liu, J., Xie, S., Wang, J., Wei, Y., Ding, Y., and Zhang, L. Evaluating language models for efficient code generation. ArXiv Preprint, abs/2408.06453 06450, 2024a.
Kudo, K., Aoki, Y., Kuribayashi, T., Sone, S., Taniguchi, M., Brassard, A., Sakaguchi, K., and Inui, K. Think-to-talk or talk-to-think? when Ilms come up with an answer in multi-step reasoning. ArXiv Preprint, abs/2412.01113,, 2024.
Liu, R.-B. Opencodeeval: An extensible, efficient, and easy- to-use evaluation framework for code generation tasks on large language models, 2024. URL https: htap::::///tttttt fgithub. com/ www.aaaaroooiioooeeoeeooceeeeeee
Kulal, S., Pasupat, P., Chandra, K., Lee, M., Padon, O., Aiken, A., and Liang, P. Spoc: Search-based pscu- docode to code. In Proc. of NeurlPS, volume 32, 2019.
Liu, Z., Liu, H., Zhou, D., and Ma, T. Chain of thought em- powers transformers to solve inherently serial problems. In Proc. of ICLR. OpenReview.net, 2024b.
Li, J., Li, G., Li, Y., and Jin, Z. Structured chain-of- thought prompting for code generation. ArXiv Preprint, Abs/2305.06599, 2023a.
Lozhkov, A., Li, R., Allal, L., B., Cassano, F., Lamy-Poirier, J., Tazi, N., Tang, A., Pykhtar, D., Liu, J., Wei, Y., et al. Starcoder 2 and the stack v2: The next generation. ArXiv Preprint, abs/2402.19173. 2024.
Li, J., Cao, P., Wang, C., Jin, Z., Chen, Y.. Zeng. D., Liu, K., and Zhao, J. Focus on your question! interpreting and mitigating toxic cot problems in commonsense reasoning. 2024a.
ArXiv Preprint, abs/2402.1834
Luo, Z., Xu, C., Zhao, P.. Sun, Q., Geng, X., Hu, W., Tao, C., Ma, J., Lin, Q., and Jiang, D. Wizardcoder: Empowering code large language models with evol-instruct. In Proc. of ICLR. OpenReview.net, 2024.
Li, L. H., Hessel, J., Yu, Y., Ren, X., Chang, K.WW., and Choi, Y. Symbolic chain-of-thought distillation: Small models can also "think" sepp-by-ttep. In Proc. ACL, pp. 2665-2679. Association for Computational Linguistics, 2023b.
Madaan, A., Hermann, K., and Yazdanbakhsh, A. What makes chain-of-thouglee prompting effective? a counter- factual study. In Proc. of EMNLP Findings, pp. 1448- 1535. Association for Computational Linguistics, 2023.
Li, M., Zhang, Y., He, S., Li, Z., Zhao, H., Wang, J., Cheng, N., and Zhou, T. Superfiltering: tcak-to-strong data filtering for fast citrucctionttunng ArXiv Preprint, abs/2402.005 005300 2024b.
Magister, L. C., Mallinson, J., Adamek, J., Malmi, E., and Severyn, A. Teaching small language models to reason. In Proc. ACL, pp. 1773-1781, Association for Compu- tational Linguistics, 2023.
Li, M., Zhang, Y., Li, Z., Chen, J., Chen, L., Cheng, N., Wang, J., Zhou, T., and Xiao, J. From quantity to qual- ity: Boosting I.LM performance with self-guided data selection for instruction tuning. In Proc. of NAACL, pp. 7602-7635. Association for Computational Linguistics, 2024c.
Peng, B., Li, C., He, P., Galley, M., and Gao, J. Instruction tuning with gpt-4. ArXiv Preprint, abs/2304.03277,, 2023.
Petty, J., van Steenkiste, S., and Linzen, T. How does code pretraining affect language model task performance? ArXiv Preprint, abs/2409.00 045566, 2024.
Li, R., Allal, L. B.. Zi, Y., Muennighoff N., Kocetkov, D., Mou, C., Marone, M., Akiki, C., Li, J., Chim, J., et al.
12

---

Revisiting Chain-of-Thought in Code Generation: Do Language Models Need to Learn Reasoning before Coding?
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al. Language models are unsupervised multitask learners. OpenAl blog, 1:9, 2019.
Wei, Y., Wang, Z.,, Liu, J., Ding, Y., and Zhang, L. Magi- coder: Source code is all you need. ArXiv Preprint, abs/2312.021200, 2023.
Rajbhandari, S., Rasley, J., Ruwase, O., and He, Zero: Memory optimizations toward training trillion parameter models. In SC20: International Conference for High Per- formance Computing, Networking, Storage and Analysis, pp. 1-16. IEEE, 2020.
Wei, Y., Cassano, F., Liu, J., Ding, Y., Jain, N., Mueller, Z., de Vries, H., von Werra, L.,, Guha, A., and Zhang, L. Selfcodealign: self-alignment for code generation. In Proc. of NeurIPS, 2024.
Xu, J., Fei, H., Pan, L., Liu, Q., Lee, M.-L., and Hsu, W. Faithful logical reasoning via symbolic chain-of-thought. ArXiv Preprint, abs/2405.18357, 2024.
Rasley, J., Rajbhandari, S., Ruwase, O., and He, Y. Deep- speed: System optimizations enable training deep learn- ing models with over 100 billion parameters. In Proc. of KDDD, pp. 3505-3506 ACM, 2020.
Yan, F. and Li, M. Towards generating summaries for lexi- cally confusing code through code erosion. In Proc. of IJCAI, pp- 3721-3727. ijcai.org, 2021.
Roshdieh, N. Exol-instruuc-code--B000 2023. URL
https: hetpss//hoogiinffeee.co/ottttts miclrrosh/Boll--nstticc---- Accessed: 2023-07-11.
Yang, G., Zhou, Y., Chen, X., Zhang, X., Zhuo, T. Y., and Chen, T. Chain-of-thouu in neural code generation: From and for lightweight language models. IEEE Trans- actions Software Engineering, 2024a.
Roziere, B., Gehring, J., Gloeckle, F., Sootla, S., Gat, I.,
Tan, X. E., Adi, Y., Liu, J., Sauvestre, R., Remez, T., al. Code llama: Open foundation models for code. ArXiv Preprint, abs/2308. 12950, 2023.
Yang. G., Zhou, Y., Yang, W., Yue, T., Chen, X., and Chen, T. How important are good method names in neural code generation? a model robustness perspective. ACM Transactions on Software Engineering and Methodology. 33:1-35, 2024b.
Shen, W. and Zhang, C. Policy filtration for rlhf to mitigate noise in reward models. ArXiv Preprint, abs/2409.06957, 2024.
Snell, C., Klein, D., and Zhong, R. Learning by distilling context. ArXiv Preprint, abs/2209.15189, 2022.
Yang, J., Yang. J., Jin, K., Miao, Y., Zhang. L., Yang, L., Cui, Z.,, Zhang, Y., Hui, B., and Lin, J. Evaluating and aligning codellms human preference. ArXiv Preprint, abs/2412.052100, 2024c
Song, Z., Wang, Y., Zhang, W., Liu, K., Lyu, C., Song, D., Guo, Q. Yan, H., Lin, D., Chen, K., and Zhao. C. Al- chemistcoder: Harmonizing and eliciting code capability by hindsight tuning on multi-source data. In Proc. of NeurlPS, 2024.
Yang, S., Gribovskaya, E., Kassner, N., Geva, M., and Riedel, S. Do large language models latently perform multi-hop reasoning? ArXiv Preprint, abs/2402.16837,, 2024d.
Stiennon, N., Ouyang, L., Wu, J., Ziegler, D. M., Lowe, R., Voss, C., Radford, A., Amodei, D., and Christiano, P. F. Learning to summarize with human feedback. In Proc. of NeurlPS, 2020.
Ye, T., Xu, Z., Li, Y., and Allen-Zhu, Z. Physics of language models: Part 2.1, grade-school math and the hidden rea- soning process. ArXiv Preprint, abs/2407.205 203111 2024.
Tam, Z. R. Evol-codealpaca- 110k, 2024.
URL
Yu, Y. Lims do not think step-by-step in implicit reasoning. ArXiv Preprint, abs/2411.15888 2024.
https: h 2haelaackkat1122//vaa-dddddaaadd Accessed: 2024-03-11.
Zelikman, E., Wu, Y., and Goodman, N. D. Star: Self-taught reasoner. ArXiv Preprint, abs/2203. 14465, 2022.
Wang, Y., Kordi, Y., Mishra, S., Liu, A., Smith, N. A., Khashabi, D., and Hajishirzi, H. Self-instruct: Aligning language models with self- eeff-generated instructions. In Proc. ACI, pp. 13508. Association for Computational Linguisties, 2023.
Zhao, W. X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y..
Min, Y., Zhang, B., Zhang, J., Dong, Z., et al. A survey of large language models. ArXiv Preprint, abs/2303.18223, 2023.
Wei, J., Wang. X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E. H., Le, Q. V., and Zhou, D. Chain-of- thought prompting clicits reasoning in large language models. In Proc. of NeurlPS, 2022.
Zheng, T., Zhang, G., Shen, T., Liu, X., Lin, B. Y., Fu, J., Chen, W., and Yue, X. Opencodeinterpreter: Integrating code generation with execution and refinement. ArXiv Preprint, abs/2402. 14658, 2024.
13

---

Revisiting Chain-of-Thought in Code Generation: Do Language Models Need to Learn Reasoning before Coding?
Zhou, C., Liu, P., Xu, P., Iyer, S., Sun, J., Mao, Y., Ma, X., Efrat, A., Yu, P., Yu, I.., Zhang, S., Ghosh, G., Lewis, M., Zettlemoyer, L., and Levy, O. LIMA: less is more for alignment. In Proc. of NeurlPS, 2023.
Zhu, X., Li, J., Liu, Y., Ma, C., and Wang, W. survey on model compression for large language models. Transac- tions of the Association for Computational Linguistics, 12:1556 15115661577, 2024.
Zhuo, T. Y., Vu, M. C., Chim, J., Hu, H., Yu, W., Widyasari, R., Yusuf, N. B., Zhan, H., He, Paul, I., al. Big- codebench: Benchmarking code generation with diverse function calls and complex instructions. ArXiv Preprint, 155/2406.18877, 2024.
14

---

Revisiting Chain-of-Thought in Code Generation: Do Language Models Need to Learn Reasoning before Coding?
Appendix
A. Synthetic Dataset Process
To select suitable seed data, we established specific rules choose questions that have educational value and are worth reasoning. After that, errors that would prompt the model to reject them. The overall process is illustrated in Figure 12.
AST parse the Python code in the answer, ensuring that do provide answers with
prompt
Chain of Thought
Rule filtered
Reference Solution
PY
Teacher Generator
Raw Dataset
Unit Tests
CoT Dataset <NL, CoT, Code>
<NL, Code>
Seed Dataset <NL, Code>
AST Parser
No
Consistent?
Figure 12: Synthetic Dataset Process
B. Prompts Template
Here is the prompt for synthetic self-consister code generation:
Prompt for Instruction Synthesis
You teaching assistant helping to create Python programming task from given code snippet. You will respond best to the Python programming task, including the reasoning process, reference solutions, and test code.
[Code Snippet] {Code}
The response must have these parts: [Analysis]
{Analyze the task and reason about the given task step by step}
[Solution]
[Write a high-quality reference solution in a self-contained script that solves the task)
[Test]
[Provide ten assert statements check the correctness of your solution}
C. Dataset statics
In this section, we present the details of the datasets in Table 10. Finally, we obtain a comprehensive dataset comprising 52,293 samples. We randomly select 51,200 the training set and the rest the validation set.
D. Benchmark
HumanEval The HumanEval dataset(Chen et al., 2021), proposed by OpenAl, comprises 164 handwritten programming problems that cover areas such as algorithms, mathematical computations, and string manipulation. Each programming problem includes natural language description of the requirements, function signature, and test cases.
15

---

Revisiting Chain-of-Thought in Code Generation: Do Language Models Need to Learn Reasoning before Coding?
Table 10: Dataset statics
Dataset
Total
Filtered
Self-Consistent
Ratio
CodeExercissePyyiee
27,224
147,36
6,963
27.56%
Codefuse-EvollInsttee
66.862
16,176
8,000
11.96%
Evol-Instruct-Codd
78,264
23,175
12,003
15.34%
Evol-CodeAlpacee
111,272
27,566
13,529
12.16%
Python ShareGPT
22,608
14,061
6,131
27.11%
Magicoder-OSS-Instuuu
75,197
14,499
5,667
7.54%
MBPP MBPP (Mostly Basic Programming Problems) (Austin et al., 2021) a benchmark dataset proposed by Google. consists of 974 programming tasks written by humans and reviewed by experts, involving simple numeric manipulations or basic usage of standard libraries.
HumanEval+ and MBPP+ HumanEval+ and MBPP+ (Liu al., 2023) are advanced benchmarks designed provide more challenging evaluations for CodeLLMs. They combine LLM-based and mutation-based input generation provide diverse test inputs for accurately evaluating the correctness of LLM-generated code. These two benchmarks provide more comprehensive assessment of a model's 's coding capabilities, enabling a more thorough examination of CodeLLMs.
LiveCodeBench LiveCodeBench (Jain et al., 2024) is contamination- frreeettattto-eeeeeeeee benchmark for evaluating code generation models. These tasks are curated from online judge websites, such as Codeforces and LeetCode, and feature an average of over 20 test cases per task. Although LiveCodeBench is a comprehensive benchmark addressing four problem types, the present study focuses on the code generation task to assess the function generation capabilities of LLMs.
BigChheBench BigCodeBench (Zhuo et al., 2024) is a benchmark that challenges LLMs to invoke multiple functions as tools across 1,140 fine-grate fine-grained tasks spanning 139 libraries and seven domains. In addition, the benchmark introduces a variant, BigCodeBeench-Inttre which automatically converts original doestrings into concise instructions. BigCodeBench also features subset named BigCodeBench-Haee Hard, which comprises 148 challenging programming tasks.
MultiPI-EE MultiPL-EE (Cassano al., 2022) serves benchmark evaluate the effectiveness of CodeL.L.Ms in programming languages other than Python. MultiL--E extends unit test-based evaluations to numerous languages by translating two established Python benchmarks into 18 different programming languages. This diverse set of languages enables a detailed analysis of how language frequency and structural features impact model performance.
EvalPerf EvalPerf (Liu et al., 2024a) demonstrates the effectiveness of the Differential Performance Evaluation (DPE) framework and provides valuable insights into program efficiency. The DPE framework addresses these limitations by focusing on computationally intensive tasks and implementing more stringent performance evaluation criteria. provides a structured approach for evaluating the code generation efficiency of LLM.
E. Experiment setup
The experiments are conducted on eight NVIDIA A100-SXM4-80GB GPUs, utilizing to enhance computational efficiency and reduce memory usage. We use ZeRO-3 (Rajbhandari et al., 2020) from Deep- Speed (Rasley et al., 2020) to minimize memory consumption during training. In our experiments, the micro- batch size fixed at per GPU to ensure consistent data processing across devices. At the same time, the gradient accumulation steps are varied based on the training configurations. To accelcrate the training process, we employ FlashAttention-2 (Dao et al., 2022), memory- effieieetf attention mechanism, be the implementation for LLMss. The optimizer used was Adam W.
training (BF16)
We monitored performance trends on specified benchmarks during training to systematically evaluate fine-tuned CodeLLMs under different SFT strategies. The models are initialized using a variety of pre-trained base models. To ensure the robustness and consistency our findings, we systematically studied key hyperparameters, including batch size, learning rate, and total epochs, under controlled experimental conditions. In various experiments, we adjusted the save step during training to ensure comparable number of checkpoints were available monitor model performance trends.
16

---

Revisiting Chain-of-Thought in Code Generation: Do Language Models Need to Learn Reasoning before Coding?
F. Benchmark results
use EvalPerf to evaluate the efficiency code generated by LLMs. Since efficiency is only essential when generated code is correct, we focus our analysis on models with pass rate of over 50%. Specifically, we utilize default settings, where each model generates 100 samples for each task temperature 1.0. For tasks that produce at least correct outputs, analyze up to 20 correct samples from each model. Model rankings based win rates, with pairwise comparisons conducted using the DPS method on standardized set of valid tasks. The result is shown in Table 11.
Table 11: EvalPerf results.
Method
DPS
Pass @
Task Win Rate
Model Win Rate
Seed
81.8%
69.4%
45.1%
40.5%
Clo
82.5%
67.8%
46.2%
45.9%
follow
80.8%
70.9%
44.0%
16.2%
Greecdee
84.0%
69.8%
50.2%
70.3%
To explore whether the knowledge acquired from Python can be transferred to other programming languages, compare the p accuracy of different fine-tuned models on MultiPL-E. Results in Table 12 show that the model series have different multilingual coding proficiency. Moreover, we find that the reasoning abilities the model acquired from training on synthetic Python datasets can be transferred to other programming languages, and our conclusion remains valid across different programming languages.
Table 12: MultiPL-E. results.
Method
C++
Java
РHP
Bash
Average
Seed
54.04%
51.90%
50.93%
26.58%
45.86%
Owo
54.04%
52.53%
52.17%
32.28%
47.76%
Collow
53.42%
52.53%
51.55%
31.65%
47.29%
presesee
54.66%
54.43%
56.52%
34.81%
50.11
G. Significance Analysis
We evaluate the model's performance on the EvalPlus benchmark and use a (-test to show the statistical significance. We re-run five experiments on the synthetic dataset with different random seeds and fine-tune the DeepSeek-Coder- BeepSeek-Coder--.77... BB-Base model. The results Table 13 show that Cprezete has a significant performance improvement over the Ctollow strategy.
Table 13: Significance Analysis Results.
Random Seed
1
2
4
Average
Owlo
69.19
70.06
70.34
70.48
70.83
70.17:0.62
Ciollow
64.88
66.54
66.03
66.38
65.84
65.93-10.65
Opeecede
71.49
71.18
71.82
71.14
70.96
71.31110.4
H. SFT Configurations Study
Study of Batch Size This study investigated whether varying the batch size during fine-tuning significantly affects the change in model performance, which in turn influences computational efficiency and memory usage. This experimental setup aims to enhance the adaptability of the conclusion to varying batch sizes, thereby supporting training scenarios with limited GPUs. Model performance was assessed at checkpoints occurring every 10% of the training steps to enable fair comparisons across batch sizes. The results, shown in Figure 13, indicate that varying batch sizes do not affect our findings.
17

---

Revisiting Chain-of-Thought in Code Generation: Do Language Models Need to Learn Reasoning before Coding?
Batch Size=32
Batch Size=64
Batch Size=128
Batch Size=256
72
70
Average Pass@1(99
68
66
64
1200
1002
Figure 13: Average Performance of base models fine-tuned under different batch sizes across four benchmarks.
Study of Train Epoch This study investigated whether varying the number of training epochs affects the conclusion in SFT. Previous work (Hoffer et al., 2017) has shown that the epoch count a crucial parameter in closing the generalization gap during large-batch training neural networks, including LLMs. determines the number of dataset passes and significantly influences memorization and generalization. The performance of the SFT checkpoints is evaluated every 10% of the training steps to ensure comparability across configurations. The results in Figure 14 can also verify our findings.
Total Train Epochs=1
Total Train Epochs=2
Total Train Epochs=3
Total Train Epochs=4
72
Average Pass
70
68
66
64
62
60
100
200
200
400
650
100
1500
1200
400
800
12:0
0.00
Figure 14: Average Performance of base models fine-tuned under different total epochs across four benchmarks.
Study of Learning Rate This study investigated whether varying the learning rate during fine-tuning has a significant impact on model performance change. The learning rate is crucial for the speed and stability of convergence, both of which are critical for optimizing LLMs (Defazio et al., 2023; Abuduweili & Liu, 2024). Model performance is assessed checkpoints spaced every 10% of the total training steps ensure fair comparisons across learning rates. The results, shown in Figure 15, indicate that variations in learning rate do not change the validity of our conclusions.
Learning Rate-5a 5e-6
Learning Rate =1e-
Learning Rate=2e-
Learning
Average Pass (%)
62
Figure 15: Average Performance of base models fine-tuned under different learning rates across four benchmarks.
18


Reviews:
[{"review_id": "OBejwhAldG", "reviewer": "Reviewer_gzpe", "summary": "The paper argues that in the context of fine tuning a code generation LLM, appending the CoT after the code solution works better compared to the typical setting of prepending it. In order to show this they generate 50k code generation problems with code solutions and CoTs. Experimentally, they demonstrate SFT on training data with CoT preceding code degrades performance, while CoTs following code results in improvements.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the helpful comments! We will summarize your concerns and provide our responses below. We explored many other training data distributions:\n\n1. **different teacher model**: We take **GPT-4o-2024-08-06** as another teacher model to synthesize the CoT&Code dataset again. We perform experiments on DeepSeek-Coder-6.7B-Base using the methods mentioned in the paper and evaluate the model on EvalPlus with the following results:\n\n   |               | HumanEval Base | HumanEval Plus | MBPP base | MBPP Plus | Avg.  |\n   | ------------- | :------------- | -------------- | --------- | --------- | ----- |\n   | $C_{w/o}$     | 71.95          | 65.85          | 77.77     | 67.46     | 70.76 |\n   | $C_{follow}$  | 65.24          | 59.14          | 76.45     | 62.96     | 65.95 |\n   | $C_{precede}$ | 72.56          | 66.46          | 78.57     | 67.72     | 71.33 |\n\n2. **different training stage**: We take **the-stack** dataset from pre-training stage to synthesize the dataset for the pre-training stage. We fine-tune the models and evaluate them with the same setting in **(1)**:\n\n   |               | HumanEval Base | HumanEval Plus | MBPP base | MBPP Plus | Avg.  |\n   | ------------- | :------------- | -------------- | --------- | --------- | ----- |\n   | $C_{w/o}$     | 68.29          | 61.58          | 76.98     | 62.43     | 67.32 |\n   | $C_{follow}$  | 61.58          | 55.48          | 75.13     | 60.31     | 63.13 |\n   | $C_{precede}$ | 69.51          | 64.63          | 77.24     | 63.22     | 68.65 |\n\n3. **different synthetic template**: We direct use educational instructions from OpenCoder to synthesize the dataset without giving the reference code. We fine-tune the models and evaluate them with the same setting in **(1)**:\n\n   |               | HumanEval Base | HumanEval Plus | MBPP base | MBPP Plus | Avg.  |\n   | ------------- | :------------- | -------------- | --------- | --------- | ----- |\n   | $C_{w/o}$     | 67.07          | 60.97          | 76.71     | 60.31     | 66.27 |\n   | $C_{follow}$  | 62.19          | 54.87          | 71.95     | 57.93     | 61.74 |\n   | $C_{precede}$ | 68.29          | 60.36          | 78.57     | 62.16     | 67.35 |\n\n4. **different dataset source**: We take the code generation dataset synthesized with Deepseek-R1 from the open source website, and directly treat the think and output of R1 as CoT and result, respectively. We fine-tune the models and evaluate them with the same setting in **(1)**:\n\n   |               | HumanEval Base | HumanEval Plus | MBPP Base | MBPP Plus | Avg.  |\n   | ------------- | :------------- | -------------- | --------- | --------- | ----- |\n   | $C_{w/o}$     | 64.63          | 56.70          | 71.16     | 62.43     | 63.73 |\n   | $C_{follow}$  | 56.09          | 51.82          | 64.55     | 55.02     | 56.87 |\n   | $C_{precede}$ | 64.02          | 57.31          | 71.69     | 62.43     | 63.86 |\n\n5. **CoT precedes code degrading performance**: In this case, the model has to learn the teacher's reasoning without prior knowledge. This step tends to be more difficult for the base model because natural language distribution is more diverse. In particular, the base model often struggles to learn directly the thought generated by another strong model.\n\n6. **discussion of attention weight**: The length of each sequence (instruction, code, and CoT) is different, we first extract the attention matrix of the last layer and then partition it into 3×3 sub-blocks using a bilinear 2D interpolation method. We normalize the attention matrix across the validation set to ensure that the attention weights are comparable across rows. Therefore, the x-axis of the resulting plot represents the sequential input of tokens from the concatenated input, with its division into three equal segments reflecting the equal-sized attention weight matrices between the instruction, code, and chain-of-thought parts.\n\n7. **discussion of layer gradient norm**: The gradient norms of different layers in the right plot are generally smaller and show a smoother, more gradual transition between layers. Previous research suggests that smooth gradient norm distributions during training can reduce the risk of unstable optimization, resulting in more stable and consistent parameter updates.\n\n8. **discussion of inconsistent data**: This discussion aims to investigate the conditions under which high-quality code data can serve as a CoT. Our results in the inconsistent data experiment suggest that self-consistency alone is not a reliable indicator of code quality since the teacher model may generate the wrong test cases, and the generated code may not be incorrect.\n\n9. **discussion of adversarial code**: The experiments investigate whether lexical information affects the quality of the code. Our results show that model performance suffers significantly under such perturbations, suggesting that current models often rely heavily on lexical representations for semantic understanding.", "claims_and_evidence": "The primary claim in the paper is that the best order for a CoT coding SFT dataset is: prompt -> code -> CoT. The paper provides ample evidence that this applies to the specific dataset the authors created as part of this submission. This is a caveat that should be better highlighted and discussed in the paper. In particular, the question remains whether the findings generalize to SFT data with CoTs that are generated through a different template or generally with a different process.", "methods_and_evaluation_criteria": "Appending CoTs to the code instead of prepending it is sensible when the CoT is viewed as an explanation. The dataset used for the main results (LiveCodeBench) is the current gold standard.", "theoretical_claims": "no theory", "experimental_designs_or_analyses": "Some ablations in the discussion section only seem tangentially related and could benefit from some further explanation:\n- context attention weight plots: what's the x axis? why's the split between code and CoT the same region between left and right? How come both split between NLP and Code/CoT are equally large?\n- layer gradient norm: what's the significance of this result? Why does it matter that the gradients of the two datasets have different magnitudes? how does that compare to other datasets where the order of the CoT isn't relevant or works better in the standard order?\n- inconsistent data analysis, adversarial code robustness: how are these relevant to the main claim of the paper?", "relation_to_broader_scientific_literature": "Currently, inducing CoTs before the solution is the predominant method to make use of extra tokens and also the way in which CoT SFT data is constructed. This paper challenges this approach in the context of code generation.", "essential_references_not_discussed": "not that I'm aware of", "other_strengths_and_weaknesses": "strengths:\n- the paper points out an interesting result in the context of SFT for code generation\n- the experimental results and analysis are quite extensive\n\nweaknesses:\n- more care and discussion should have been devoted to the relation between the specific training dataset and the experimental findings. In particular, appending CoT to code appears to only slightly improve over code without CoT (if at all), while CoT before code appears to significantly degrade the performance. the latter result is surprising and can benefit from more analysis and discussion", "other_comments_or_suggestions": "the Figure 1 legend is hard to read. maybe make the lines thicker and add the symbols to the legend?", "questions_for_authors": "My primary concern is that it's unclear whether these findings will generalize to other training datasets. As mentioned in the previous comments, I'd like to understand:\n1. will other training data distributions yield the same results or is there a specific characteristic for the CoTs that make them more useful when appended?\n2. why does appending the CoT degrade the performance so significantly?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9VKzJdINov", "reviewer": "Reviewer_PRDj", "summary": "This paper primarily investigates how chain-of-thought reasoning affects code generation performance. The paper first constructs a dataset of 50k pairs for code generation. A series of experiments is then conducted to investigate how the presence and position of a chain-of-thought affect resulting code generation performance. \n\nOn HumanEval, BigCodeBench, and LiveCodeBench; the authors find that the standard procedure of having code follow the CoT is in many cases substantially worse than having the CoT follow the code. A series of experiments with different model families (DeepSeek, Llama, Qwen-Chat/Coder) show similar results. \n\nThere are a large number of additional experiments ablating and studying this effect in various settings.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for helpful comments! We appreciate the reviewers for pointing out some other reference in related work, and we promise that these relevant references will definitely be included in the future revision.", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "Not applicable.", "experimental_designs_or_analyses": "I checked both Section 5 and Section 6.", "relation_to_broader_scientific_literature": "I do not think the specific research question addressed by the authors has been explored by prior work. I think the closest papers to this paper are papers like Program-of-Thoughts, which demonstrate that reasoning in code can be superior to reasoning in natural language. A common assumption in most reasoning papers is that CoT should come before the final answer, but I don't think any paper has shown the counterintuitive empirical result that the CoT is better off coming after the code.", "essential_references_not_discussed": "None.", "other_strengths_and_weaknesses": "I think the main finding of the paper is very interesting and throws into question the useful of chain-of-thought for code generation. In addition, the experiments are well done, and there is a comprehensive discussion section that includes a large number of ancillary results.", "other_comments_or_suggestions": "None.", "questions_for_authors": "None.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xk2GgPJcxR", "reviewer": "Reviewer_ueRf", "summary": "This paper researches on the order of CoT and code in code generation tasks. The authors collect pairs of program questions and response codes in six code data, and then prompt an LM to generate CoT and finally create a dataset with question, code and natural language CoT triplets. Their experiments show that the (question, code, CoT) (that is code precede) surpasses (question, CoT, code) (that is code follow), and they also have many observation about the difference between these two patterns. shed light into how to train a code generation model.\n\n## Update after rebuttal\n\nExperiments with larger models, longer training stage, other CoT source and additional study about the difficulty have partly solved my concerns but I am still not sure whether the conclusions are still correct for most powerful models (like models with >100B parameters and billions of training tokens).", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We address the concerns as below:\n\nQ1: How well do the paper's conclusions hold with large-scale training and a powerful model?\n\nR1:**Table A. Extent to a larger scale training dataset:**\n\n We construct a large dataset of 800K samples. We train Llama-3.1-8B and compare it with DeepSeek-R1-Distill-Llama-8B on EvalPlus:\n\n|                        | HumanEval Base | HumanEval Plus | MBPP base | MBPP Plus | Avg.          |\n| ---------------------- | -------------- | -------------- | --------- | --------- | ------------- |\n| DS-R1-Distill-Llama-8B | 76.82          | 70.73          | 71.42     | 59.52     | 69.62         |\n| Code follow CoT        | 74.39          | **70.12**      | 67.98     | 55.55     | 67.01         |\n| Code precede CoT       | **76.82**      | 69.51          | **74.86** | **62.69** | **70.****97** |\n\n **Table B. Extent to pre-training stage:**\n\n We take \"the-stack\" as the source seed to synthesize the dataset of pretraining. We perform experiments on DeepSeek-Coder-6.7B-Base:\n\n|                  | HumanEval Base | HumanEval Plus | MBPP base | MBPP Plus | Avg.      |\n| ---------------- | -------------- | -------------- | --------- | --------- | --------- |\n| Code follow CoT  | 61.58          | 55.48          | 75.13     | 60.31     | 63.12     |\n| Code precede CoT | **69.51**      | **64.63**      | **77.24** | **63.22** | **68.65** |\n\n **Table C. Extent to more powerful models**\n\n We conduct experiments with a larger LLM, Qwen2.5-Coder-Base-32B. The results are reported as follows\n\n|                  | HumanEval Base | HumanEval Plus | MBPP base | MBPP Plus | Avg.  |\n| ---------------- | -------------- | -------------- | --------- | --------- | ----- |\n| Code follow CoT  | 85.98          | 79.27          | 87.83     | 71.69     | 81.19 |\n| Code precede CoT | 86.59          | 79.27          | 89.15     | 76.19     | 82.80 |\n\n The results in Tables A, B, and C show that our conclusion, \"CoT following the code can work better,\" is held when scaling up the training dataset and model size.\n\nQ2: How can the conclusion be applied in practice, and extensive discussions?\n\nR2: A key contribution is that using the \"code precedes CoT\" pattern outperforms the \"code follows CoT\" approach: (1) it establishes a more effective paradigm for integrating CoT with code during training. (2) During inference, it can facilitate early stopping by generating only the code, thereby reducing computational cost and latency. All discussions and experiments are designed to support our main conclusion. We will reorganize them.\n\nQ3: Additional questions raised in the comments:\n\n Q3.1: Performance across various task difficulties (such as various code length):\n\n R3.1: We took your suggestion and reclassified the questions' difficulty according to the length of the output answers: \n\n|                  | Length < 500 | 500 < Length < 1000 | 1000 < Length < 1500 | Length > 1500 |\n| ---------------- | ------------ | ------------------- | -------------------- | ------------- |\n| Code follow CoT  | 0.0          | 20.17               | 10.64                | 0.0           |\n| Code precede CoT | 25.0         | 29.20               | 19.23                | 9.09          |\n\nAnd DeepSeek-Coder-Instruc-6.7B got only 9.1 on Leetcode Hard with the same base.\n\n Q3.2: Nature of the phenomenon & Q3.3: Training data and process: \n\n R3.2 & R3.3: These questions have been addressed in R1, which shows that our conclusion still holds with a more powerful model and a larger training set.\n\n Q3.4: Bias from synthetic CoT.\n\n R3.4: We provide additional experiments where the CoT is generated first. We directly use educational instructions from OpenCoder to synthesize the CoT without the code and hold the same conclusion.\n\n|                  | HumanEval Base | HumanEval Plus | MBPP base | MBPP Plus | Avg.  |\n| ---------------- | -------------- | -------------- | --------- | --------- | ----- |\n| Code follow CoT  | 62.19          | 54.87          | 71.95     | 57.93     | 61.74 |\n| Code precede CoT | 68.29          | 60.36          | 78.57     | 62.16     | 67.34 |\n\nQ3.5: Variance Analysis.\n\nR3.5: We adhere to standard settings using sufficiently large model and dataset sizes, which ensures stable experimental results even without including variance. Furthermore, we have presented extensive experimental results across various configurations in Appendix G. We appreciate your suggestion and plan to report the variance in a future version.\n\nQ3.6: Explain the o1-like model is better than general ones and the relation with your conclusion.\n\nR3.6: A proper CoT process is crucial for deriving accurate answers, particularly in mathematical and logic domains. However, limited research link CoT and Code for code generation. We argue that using code preceding CoT is an effective data paradigm. Well-designed, high-quality code should be recognized as a valid form of CoT. We checked open-source datasets generated by R1 and confirmed the existence of this phenomenon.", "claims_and_evidence": "The main claim of the paper is that the code-precede CoT outperforms code-follow CoT in code generation tasks. This claim is supported by experiments on various datasets (e.g., HumanEval, MBPP, LiveCodeBench) and models (e.g., DeepSeek-Coder-6.7B, Llama-3.1-8B, Qwen2.5-chat-7B, Qwen2.5-coder-7B). While the results are convincing, with a consistent and large gap between the two approaches, I believe the absence of variance analysis in the figures and tables is a minor issue. Including such analysis would make the results more robust.\n\nHowever, I find the authors' explanation \"code itself is a suitable CoT, while traditional CoT should be regarded as an explanation of the code rather than reasoning\" to be insufficiently supported. I have several concerns:\n\n1. **Task Difficulty**: While the pretrained code corpus is diverse and the LeetCode experiments discuss about different difficulty levels, it's unclear whether the difficulty of the code generation tasks is sufficiently varied. Task complexity ranges from simple translation of instructions to code, to solving complex algorithmic problems or ambiguous GitHub issues. I believe task difficulty significantly impacts the role of code and CoT. For simpler tasks, CoT may act mainly as a post-explanation, but for more complex tasks, defining the problem clearly, choosing the right algorithm, and debugging are crucial parts of the CoT, and these cannot be fully replaced by the code itself. Although the authors provide experiments on Leetcode with varying difficulty, they focus only on code problem complexity. Additional factors like the clarity or completeness of the question, or code length, should also be considered. At the same time, all the four method shows <10% performance on the Leetcode Hard subset, this results may not be very convincing.\n\n2. **Nature of the Phenomenon**: The authors argue that \"code itself is a suitable CoT\", but it is unclear whether this phenomenon is inherent to code generation tasks or a result of differing difficulty between generating code and CoT. Since code is a formal language, it might be easier for the model to learn than natural language CoT. Further supporting my concerns, Figure 3 shows that the perplexity (ppl) on CoT tokens is significantly higher than that on code tokens, both in the code-precede and code-follow models. This suggests the model is less familiar with natural language CoT. In such cases, CoT might not be helpful for the model. If so, it is doubtful whether a sufficiently powerful model would have yielded different conclusions. After all, being harder does not necessarily mean ultimately worse performance.\n\n3. **Training Dataset and Process**: The training dataset is relatively small (51,200 samples), and the training process only involves SFT. This is much smaller than real-world datasets. For example, DeepSeek-Coder-Instruct uses 2 billion tokens to fine-tune natural language abilities. The paper does not address whether the conclusions would still hold with a larger training set, or if the code-follow approach might perform better with more training data.\n\n4. **Bias from Synthetic CoT**: The synthetic CoT in the experiments is generated by LLM, conditioned on the question and code, which makes it more of an explanation of the code rather than a reasoning process. The authors do not explore whether the results would change if the CoT were generated based on the question and before the code generation, or if it were generated as a human-like reasoning process. In real-world scenarios, CoT typically consists of two parts: one before the code, offering general instructions on how to solve the problem, and one after, explaining the code in detail. The former is more like a reasoning process, and the latter is a code explanation. I believe this distinction should serve as a baseline for comparison.", "methods_and_evaluation_criteria": "They create datasets with a classic method by using an LLM to generate CoT. There is a concern that the synthetic CoT could introduce bias to the conclusion, as I claimed in the above point 5.\n\nThey do not provide a new evaluation metrics but use the commonly used pass@k.", "theoretical_claims": "There is no theoretical claims.", "experimental_designs_or_analyses": "Except for the main results and the concerns I mentioned above. There are many experiments used to explain the different roles of CoT and code in section 6 Discussion. I feel some of them are not necessary, or the authors should explain their motivation more clearly. For example, why the \"layer gradient norm\", \"adversarial code robustness\", and \"signature removal analysis\" are necessary to explain the conclusion. The subsection \"Generalization across sizes\" can be just moved to the figure 2. The discussion about \"temperature\" and \"pass@k\" is too detailed for me and can be moved to the appendix.\n\nAnd some other experiments need to be explained more clear: \n\n1. In figure 4, the reader cannot understand whether the KL-divergence 0.13 is smaller enough or not.\n2. There is a lacking of results analysis of Figure 11.", "relation_to_broader_scientific_literature": "The observation that code-precede CoT surpasses code-follow CoT is useful to train better code generators, which is an important task in the field of AI. However, I worry about whether the conclusion can be generalized to pretraining stage or on a much larger dataset. If the conclusion is due to the current models are more familiar with code than natural language CoT, or the natural language CoT is naturally harder to learn than code, the current conclusion may not hold when the model is finally trained well on CoT.\nAt the same time, how the conclusion can be used in practice is not clear. Does the conclusion means that the model should be trained to first generate code and then generate CoT? (Or even without CoT? because I find that even the w/o model show not bad performance.) If so, how can we explain the phenomenon that o1-like long reasoning model always show better performance than general ones like gpt-4o?", "essential_references_not_discussed": "The related work should be updated with some more recent works. For example, though the authors use qwen2.5-code in their experiments, they do not mention the qwen2.5-code in the related work. Some recent works like repo-level code generation (like [SWE-bench](https://www.swebench.com/)) or oi-competition (like [deepseek-r1](https://deepseek.ai/)) should be mentioned. There are some work talking about the CoT and the code like [PAL](https://arxiv.org/abs/2211.10435) and [code-prompt](https://arxiv.org/abs/2305.18507) using code as CoT to solve the math problem. These work are very related to the paper and should be mentioned in the related work.", "other_strengths_and_weaknesses": "I believe one of the strengths of the article is its extensive experimental content. However, this also raises some concerns for me, as it may be a sign of insufficient core contributions in the paper. This is especially true when considering that many of the experiments in the main text seem somewhat redundant or overly detailed. I think the author should reorganize the three-page discussion more carefully, clearly explaining the motivation, results, and relationships of each experiment. Particularly, if certain experiments are not tightly linked to the conclusions, they should either be removed or placed in the appendix. Based on the current conclusions of the paper, I feel that the contribution falls slightly short of what would be expected from a complete conference paper. The overly detailed experiments, in my opinion, do not add significant value to the overall contribution.", "other_comments_or_suggestions": "See aboves.", "questions_for_authors": "There are two main concerns. First, to what extent can the conclusions of the paper hold when the training scale is sufficiently large and the model's capabilities are strong enough? This is crucial as it will determine how the conclusions of the paper impact the training and application of related models. Second, regarding the substantial amount of discussion included in the paper, I would like the author to justify its necessity, explaining how these discussions contribute to the core contributions of the paper, rather than merely summarizing the experimental results.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qdaLE0pe39", "reviewer": "Reviewer_793T", "summary": "The paper explores the impact of the data formatting when finetuning LLMs with synthetically generated data. Specifically, the data--composed of code and corresponding reasoning steps for a NL-to-code problem--is generated by a stronger \"teacher\" model (the authors state that it is a DeepSeek model in Appendix B, but do not specify which one). The data is then formatted in different ways by either putting the reasoning before or after the code snippet, or leaving it out altogether. The authors find that, somewhat surprisingly, including the reasoning before the code snippet does not seem to benefit downstream performance. They then go on to argue that including it *after* the code snippet may in fact benefit the SFT'd model the most.\n\n## update after rebuttal\nI thank the authors for a productive and engaging rebuttal discussion, which has certainly strengthened the findings of the paper. In particular, the addition of proper statistical testing, which should have been included in the original paper given the many possible confounding factors present, was much appreciated.\n\nIn the end, I have decided to nonetheless keep my original score of 2 (Weak Reject). If this is surprising to the authors then suffice it to say that I sympathize with them regarding the difficulty of presenting convincing evidence for counterintuitive findings. In its current form, the paper has me convinced that there is something amiss in the way we think about combining reasoning and code, but what exactly that is I am not convinced of. I note that the authors surely have tried to answer this question from a bunch of different angles, but none of them seem to have shed much light on the issue; in particular, none of the experiments in Section 6.1 seem quite insightful enough to me.\n\nI will note to the AC that my vote for rejection is indeed to be taken to be quite weak; if the other reviewers are happy for the paper to be accepted in its current form then I will not vehemently oppose it. I simply think that the authors seem to have struck upon something deeply surprising and interesting, and with a few more months would likely be able to uncover what the cause is in much greater and more illuminating detail, ultimately leading to a much stronger contribution to the literature. As is, the paper will simply raise a few eyebrows, but not offer any lasting insights (in my opinion).", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for helpful comments! We would like to summarize your concerns and provide our responses below:\n\n1. **which is the teacher model**: We select **DeepSeek-V2.5-1210** as our teacher model because of its strong capabilities and acceptable cost. It is the most suitable version available during our primary experiment phase.\n\n2. **different teacher model**: We take **GPT-4o-2024-08-06** as another teacher model to synthesize the CoT&Code dataset again. We perform experiments on DeepSeek-Coder-6.7B-Base using the methods mentioned in the paper and evaluate the model on EvalPlus with the following results:\n\n   |               | HumanEval Base | HumanEval Plus | MBPP base | MBPP Plus | Avg.  |\n   | ------------- | :------------- | -------------- | --------- | --------- | ----- |\n   | $C_{w/o}$     | 71.95          | 65.85          | 77.77     | 67.46     | 70.76 |\n   | $C_{follow}$  | 65.24          | 59.14          | 76.45     | 62.96     | 65.95 |\n   | $C_{precede}$ | 72.56          | 66.46          | 78.57     | 67.72     | 71.33 |\n\n3. **whether the conclusion contradicts the results of DeepSeek-R1**: Following the release of DeepSeek-R1, we have reviewed our experimental results and believe that our conclusions are indeed consistent with, rather than contradictory, their work. We agree that a proper CoT process is essential for deriving accurate answers, especially in mathematical and logical domains. Instead, we are against the suggestion that CoT processes must be expressed through natural language. Well-designed and high-quality code should also be recognized as a valid form of CoT reasoning, and models do not need to learn to reason through natural language but can learn to code directly and then explain. Thus, our study complements rather than contradicts the results of DeepSeek-R1. To further support this, we take the open source code generation dataset synthesized with **DeepSeek-R1** and directly treat the think and output of R1 as CoT and result, respectively. We fine-tune the models and evaluate them with the same setting in **2**:\n\n   |               | HumanEval Base | HumanEval Plus | MBPP Base | MBPP Plus | Avg.  |\n   | ------------- | :------------- | -------------- | --------- | --------- | ----- |\n   | $C_{w/o}$     | 64.63          | 56.70          | 71.16     | 62.43     | 63.73 |\n   | $C_{follow}$  | 56.09          | 51.82          | 64.55     | 55.02     | 56.87 |\n   | $C_{precede}$ | 64.02          | 57.31          | 71.69     | 62.43     | 63.86 |\n\n   We note that, as mentioned in other recent studies of R1, there is a certain probability that the model will fail to stop thinking when using $C_{follow}$ format.\n\n4. **whether the evidence sufficiently supports the claim**: We thank the reviewer for raising this point. Indeed, this is consistent with our first argument in the paper, that well-organized and high-quality code can effectively serve as a CoT. Thus, it is reasonable that in some cases the model may have learned enough from the code alone. However, this phenomenon is not universally observed in our experiments. In particular, the explanations following the code added value in most scenarios. Therefore, we hope that the reviewer will notice the second finding, namely that placing the CoT after the code generally provides an additional benefit in most experimental settings, which builds on our initial proposal and is mainly observed. We adhere to standard settings using sufficiently large model and dataset sizes, which ensures stable experimental results even without including variance. Furthermore, we have presented extensive experimental results across various configurations in Appendix G. We appreciate your suggestion and plan to report the statistical result in a future version.\n\n5. **discussion of pass@k and tempature**: Different k values reflect different aspects of model performance. If model A gives [10, 0, 0] and model B gives [4, 3, 3] over three problems with 10 attempts, both have the same pass@1, but B has a higher pass@10. Small k captures the model's confidence (A solves a problem very well), while larger k reflects the coverage of the model's ability (B can solve more problems). Therefore, varying k reveals different evaluations of model behavior and potential. In addition, temperature also affects the model's performance, with low temperatures tending to make the model's output more fixed and high temperatures encouraging more explore space. We want to show that the conclusions are consistent.\n\n6. **brief description in discussion**: We apologize for any confusion caused by our brief description of the experiments. We confirm that each experiment was carefully designed with reliable conclusions, and we will make the meaning of the discussions more explicit in future versions.\n\n7. **missing references**: We appreciate the reviewers for pointing out the omissions in related work, and we promise that these relevant references will definitely be included.", "claims_and_evidence": "I do not think the claim/observation that formatting the data such that the reasoning follows the code benefits performance is supported by the evidence. This findings appears a bit hand-picked; there are many experiments in which, for example, Code w/o CoT does better than $C_\\text{precede}$, but this is not even acknowledged (Fig 10, right, high temps; Fig 9, left; Fig 7, right; Fig 2, llama).\n\nThe large number of completely unrelated experiments, many of which are then not even discussed much in the text, does not inspire a great deal of confidence in the results being statistically significant, either. For example, the discussion of the last experiment is simply \"we aim to gain insight into the degree of semantic match correspondence and the extent to which the generated outputs adhere to the given instructions.\" That just says what you *wanted* the experiment to tell you; it says absolutely nothing about what you then actually found, and indeed the figure does not seem particularly telling either.", "methods_and_evaluation_criteria": "The dataset, model choice and metrics all make sense to me. In particular, it's good that the authors included leakage-resistant benchmarks like LiveCodeBench. Their findings would have been strengthened somewhat by trying a different teacher model, too, although it is understandable that this may not have been in scope in terms of compute costs.", "theoretical_claims": "N/A; no theoretical claims or proofs.", "experimental_designs_or_analyses": "Evaluating different data formats for SFT by simply doing SFT with a few different base models makes sense. This is the core experiment in the paper, which I am on board with.\n\nHowever, there are many, many \"side experiments\" that I do not really understand the relevance of. Take for example the data mixing experiment: training on data is inconsistently formatted hurts performance. Is that really a finding? What does that tell you about how to format your data? To my eyes, this is a complete non-result.\n\nThe worst case of this at the end of P.7, when the authors \"evaluate how varying the number of samples `k` in `pass@k` affects performance for the different methods) (Figure 10, left), and conclude that: \"As k increases, all configurations demonstrate an improvement in accuracy, indicating models exploring more possibilities and increasing the chances of producing correct outputs\". This doesn't even make any sense as an experiment; pass@k as a metric is monotonically non-decreasing by construction, since if you draw more samples i.i.d. the probability that at least one of them is correct cannot go down. This is a very concerning methodological misunderstanding and raises serious suspicion as to the validity of the other experiments, too.", "relation_to_broader_scientific_literature": "The finding that including reasoning steps when distilling from a bigger model to a smaller code-specific model does not seem to benefit downstream performance is somewhat counter-intuitive, and I think an interesting contribution. This goes against the grain of recent findings that suggest that using SFT to distill CoT reasoning discovered with RL into a smaller model can help in math and reasoning domains (c.f. the DeepSeek-R1 paper).", "essential_references_not_discussed": "I was surprised that Austin et al's \"Program Synthesis with Large Language Models\" (2021) was not cited as an example of early use of LLMs for code, since it is typically looked at as the seminal paper in the field.\n\nMore importantly, there does not appear to be a citation for pass@k, even though the unbiased estimate employed by the authors was first introduced in the Codex paper (\"Evaluating Large Language Models Trained on Code\", Chen et al., 2021). I believe the original idea of a pass@k metric is due to Kulal et al.'s \"SPoC: Search-based Pseudocode to Code\" (2019).", "other_strengths_and_weaknesses": "I appreciate that this paper is, at its core, contributing a non-obvious and somewhat counterintuitive finding: that including the reasoning steps when distilling the ability to generate code from a larger model to a smaller model does not appear to make a measurable difference. That is, to me, a valuable contribution; however, it is hard to tell how much that depends on the teacher model (which they keep fixed).", "other_comments_or_suggestions": "None.", "questions_for_authors": "1. Do you think the core finding--that including reasoning is not necessary when distilling from one code llm to another--would translate to different base/teacher models, too?\n2. Why do you think the effects of varying the number of samples or the sampling temperature are indicative of the different SFT'd models' underlying capacity?\n3. Do you think your supposed finding that putting the reasoning after the code is better than before (or not including it at all) is statistically significant?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This work looks into question of tuning a smaller model for NL code generation using CoT traces. Surprisingly, they find that tuning a model to first produce the code and then the CoT yields improved models compared to the “traditional” option of first using the CoT and then the code. An extensive evaluation (some in the rebuttal text) evaluates this effect and ablates various parameters.

Overall, this seems like an interesting and unexpected result that the ICML community should be aware of and hence I recommend that this work is accepted. I would kindly ask the authors to update their work to include the extensive results shown in rebuttal.
Decision: Accept (poster)