Title: Scaling Code-Assisted Chain-of-Thoughts and Instructions for Model Reasoning
Venue: NeurIPS 2025
Accepted: yes
Keywords: Data Synthesis

Reasoning
Abstract: Reasoning capability is pivotal for Large Language Models (LLMs) to solve complex tasks, yet achieving reliable and scalable reasoning remains challenging. While Chain-of-Thought (CoT) prompting has become a mainstream approach, existing methods often suffer from uncontrolled generation, insufficient quality, and limited diversity in reasoning paths. 
Recent efforts leverage code to enhance CoT by grounding reasoning in executable steps, but such methods are typically constrained to predefined mathematical problems, hindering scalability and generalizability. 
In this work, we propose \texttt{Caco} (Code-Assisted Chain-of-ThOught), a novel framework that automates the synthesis of high-quality, verifiable, and diverse instruction-CoT reasoning data through code-driven augmentation. Unlike prior work, \texttt{Caco} first fine-tunes a code-based CoT generator on existing math and programming solutions in a unified code format, then scales the data generation to a large amount of diverse reasoning traces. Crucially, we introduce automated validation via code execution and rule-based filtering to ensure logical correctness and structural diversity, followed by reverse-engineering filtered outputs into natural language instructions and language CoTs to enrich task adaptability. This closed-loop process enables fully automated, scalable synthesis of reasoning data with guaranteed executability. 
Experiments on our created \texttt{Caco}-1.3M dataset demonstrate that \texttt{Caco}-trained models achieve strong competitive performance on mathematical reasoning benchmarks, outperforming existing strong baselines. Further analysis reveals that \texttt{Caco}’s code-anchored verification and instruction diversity contribute to superior generalization across unseen tasks. Our work establishes a paradigm for building self-sustaining, trustworthy reasoning systems without human intervention.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces Caco (Code-Assisted Chain of Thought), a framework for generating large-scale, high-quality, and executable reasoning data (in the form of Python code) to train large language models (LLMs). It addresses key limitations of traditional Chain of Thought (CoT) prompting, which is typically done in natural language and suffers from issues such as unverifiability, reliance on manual annotation, and logical inconsistencies. The authors demonstrate that fine-tuning both specialized and general-purpose open-source models on the proposed dataset significantly improves their performance on mathematical reasoning benchmarks.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n- A dataset where reasoning traces are executable (i.e., Python programs) can significantly improve the verifiability and reliability of reasoning chains. Unlike natural language-based reasoning, which is difficult to parse and not directly executable.\n- Fine-tuning results show consistent improvements across benchmarks.\n\n**Weaknesses**\n\n- Although the dataset is useful, the core research contributions are somewhat unclear. Representing reasoning in executable code and using fine-tuned models like CodeGen for data generation have both been explored in prior work. For example, \\[1] uses (a similar idea) Formalizer models to generate formal proof data from informal sources. It would be great if the authors could clarify their technical contributions and how they compare with existing, well-known ideas.\n- While the results reported in Table 1 are promising, existing methods such as rStar-Math appear to achieve substantially better performance with similarly sized models (see Table 5 in \\[2]). The authors should clarify why this baseline was not included and, if possible, provide a fair comparison with such existing methods.\n\n\\[1] *Goedel-Prover: A Frontier Model for Open-Source Automated Theorem Proving*\\\n\\[2] *rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinking*\n\nQUESTIONS:\nSee the weakness section\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a technique called “Caco” for synthetic data generation which works by translating math reasoning problems to and from code. Specifically, Caco has the following stagers: (1) prompt an LLM (Qwen-2.5-72B-Instruct) to convert existing math datasets to a unified code format (which the paper calls a “Code CoT”) which are filtered to ensure they produce the same answer as in the original dataset; (2) finetune a new LLM (unclear which one) on the Code CoTs to produce a model called CodeGen; (3) sample from the CodeGen model to create novel Code CoTs; (4) prompt an LLM (unclear which one) with the Code CoTs to produce corresponding natural language prompts, and filter the resulting prompts (using Qwen3-32B) to ensure that they are solvable. This procedure results in a dataset of 1.3M math problems which, when used to finetune various models (DeepSeekMath-7B, Qwen-2.5-Math-7B, LLaMA3-8B), substantially improves their reasoning performance across six benchmarks, outperforming the performance of those same models when trained on a variety of other math-specific datasets.\n\nSTRENGTHS AND WEAKNESSES:\nPros:\n* Originality: It is an interesting idea to leverage a formal language to constrain the synthetic data generation process, which enables more reliable verification. This is not a completely original idea on its own (e.g. methods like AlphaGeometry also leverage this idea) but to my knowledge the particular instantiation of this idea in this paper is new.\n* Significance: Synthetic data generation is an important area of LLM research especially in reasoning domains, as high-quality, expert-level human-generated data is hard to come by.\n* Clarity: I appreciate that all the prompts used in the pipeline are included in the appendix, thank you for including this level of detail.\n\nCons:\n* Clarity: I found the paper difficult to read at times. (1) The method is relatively straightforward to explain yet I found the explanation—especially in the abstract and introduction—to be unclear and it was not until I had fully finished reading the methods that I was able to understand it. Similarly, I found Figure 2 to not be particularly easy to understand—I needed to read the text first to understand it, whereas it should really be the other way around. (2) Many details are also scattered throughout the paper & appendix making it difficult to completely understand the full picture.\n* Clarity: I find the term “Code CoT” to be misleading. It’s not a chain-of-thought at all because it’s not step-by-step reasoning—it’s a programmatic solution to the given problem.\n* Quality: Important details are missing from the paper, and there are some inconsistencies. I detail these further below in the “Questions” section.\n* Quality: Given that the paper uses a larger model (Qwen-2.5-72B-Instruct) to generate the seed dataset of Code CoTs, I have a concern about whether the gains come from the knowledge from the more advanced model, or from the Caco procedure. I discuss this concern in more detail below in the “Questions” section.\n* Quality: The proposed procedure shares some similarities with STaR/ReST, which are much simpler methods. Ideally one iteration of STaR would be included as a baseline to confirm that the gains obtained from Caco are indeed due to leveraging the code representation. I discuss this concern in more detail below on the “Questions” section.\n\nQUESTIONS:\n### Missing, Unclear, and Inconsistent Details\n\nThere are some missing details and inconsistencies. I would need to see all of these addressed to raise my score:\n\n* Which model is finetuned to produce the CodeGen model?  \n* Which model is used to produce the natural language instructions and solutions (detailed in Section 3.3)?  \n* More generally (e.g. lines 114, 147, 176, 187, 192): rather than saying “a LLM” the paper should always be explicit about which particular model is used.  \n* Caco-Seed-109K needs more explanation. The paper says it is the “seed data used to train CodeGen”, but that would be data consisting of Code CoTs, not natural language prompts and CoTs. Moreover, Line 162 says that the seed dataset is 162K examples, not 109K. Why are these numbers different? Where do the natural language CoTs come from? (Is it just that you’ve applied the rest of the Caco procedure to the 162K to generate the CoTs, and applied filtering, thus reducing the size of the dataset?)  \n* Plots and tables should include confidence intervals/error bars. These do not need to be derived from multiple experiments (I agree with the authors this would be too computationally expensive) but they can still be computed over the *dataset*. For example, for GSM8K (with N=1319 questions) and Caco-1.3M-LLaMA3-8B (with p \\= .878) the upper 95% confidence interval is .878 \\+ 1.96 \\* sqrt(p \\* (1 \\- p)) / sqrt(n) \\= 0.896 (see [https://en.wikipedia.org/wiki/Binomial\\_proportion\\_confidence\\_interval](https://en.wikipedia.org/wiki/Binomial_proportion_confidence_interval)).\n\n### Possible Distillation Effect\n\nTo generate the seed Code CoTs, the paper uses a much larger model (Qwen-2.5-72B-Instruct) than the models ultimately finetuned on Caco & benchmarked in Table 1\\. I am worried that because of this, the Caco data pipeline is essentially just implementing a process of distillation from this larger model.\n\nAn important, simple baseline would be to instead sample natural language CoTs from Qwen-2.5-72B-Instruct and use them for finetuning the smaller models. If this results in similar performance as training on Caco (when matched for dataset size), this would suggest that the ability to synthesize new prompts is not providing much benefit—rather, the gains are coming from solutions generated by a larger model.\n\nI would need to see a comparison to this baseline (or something analogous to convince me that the results are not coming from distillation) to consider raising my score.\n\nIdeally, the paper would also include benchmark scores for Qwen-2.5-72B-Instruct for reference.\n\n### Comparison to STaR / ReST-EM\n\nRelated to the above point, but slightly different, is that the Caco procedure can be seen in some sense as a (relatively complicated) iteration of STaR \\[1\\] or ReST-EM \\[2\\], which is a much simpler method. An important baseline would be to include a comparison to a single iteration of self-improvement: (1) generate CoTs from a model (e.g. one of the ones benchmarked), (2) filter the CoTs by correctness, (3) and then finetune the model on the resulting traces.\n\nThis would be an important control because it would again show that the gains are coming from having a higher number of unique prompts to train on, rather than a more generic form of self-improvement. (Note it should be possible to match for dataset size as well by generating multiple samples per prompt.)\n\nI would need to see a comparison to this baseline (or something analogous to convince me that the results are not coming from vanilla self-improvement) to consider raising my score.\n\nRefs:\n\n1. Zelikman, E., Wu, Y., Mu, J., & Goodman, N. (2022). STaR: Bootstrapping reasoning with reasoning. Advances in Neural Information Processing Systems, 35, 15476-15488.  \n2. Singh, A., Co-Reyes, J. D., Agarwal, R., Anand, A., Patil, P., Garcia, X., ... & Fiedel, N. (2023). Beyond human data: Scaling self-training for problem-solving with language models. arXiv preprint arXiv:2312.06585.\n\n### Minor Points\n\nThese are some suggestions for improving the paper (but have not strongly influenced my score):\n\n* Line 66 \\- please make it clear what the “65.4% accuracy” is over (i.e. the six benchmarks).  \n* As far as I can tell, the “problem-level augmentation” described in Figure 3 isn’t mentioned anywhere else in the paper. It should be removed if it’s not part of the paper, or made clear what part of the text this is referring to.  \n* Section 3.4 seems like it could be in the appendix.  \n* Section C.1: What is Caco-100K? It is not described anywhere.  \n* Section C.2: I’m not sure I understand this experiment. Is the idea that when generating the problem solutions on the Caco prompts, you enable thinking, and then train on these longer traces instead? If so, I think this not a very informative experiment without the inclusion of some further baselines (e.g. AIME performance after training on Caco w/o long prompts)\n\nLIMITATIONS:\nThe main unaddressed limitation is the fact that the paper uses a larger model as part of the data generation procedure, but then only evaluates on smaller models. This could result in the improved results being due to a distillation effect, rather than due to the proposed data generation method. This limitation should be addressed.\n\nEDIT: I believe the authors' rebuttal has sufficiently addressed this limitation.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes Caco (Code-Assisted Chain-of-ThOught), a novel framework designed to automate the synthesis of high-quality, verifiable, and diverse instruction-based Chain-of-Thought (CoT) reasoning data through code-driven augmentation. This framework can improve the scalability and generalizability of the code-assisted reasoning method, thereby enhancing the reasoning capabilities of large language models.\n\nThe key contributions are:\n\n1.\tThe authors proposed Caco, a scalable code-assisted CoT and instruction generation framework designed to automate the production of high-quality reasoning training data through code-anchored refinement.\n2.\tThe authors created Caco-1.3M, a large-scale synthetic dataset containing millions of high-quality reasoning traces\n3.\tThrough extensive experiments on multiple mathematical reasoning benchmarks, the authors demonstrate the effectiveness and advantages of Caco over existing approaches.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1.\tHigh-quality framework proposal: The paper proposes a original and novel framework called Caco which effectively addresses the limitations of traditional CoT methods on the code-assisted reasoning problem, such as uncontrolled generation, insufficient quality, and limited diversity in reasoning paths.\n2.\tClear framework description: The paper provides a detailed introduction to the overall Caco framework. Each component of Caco is elaborated with specific methods and processes.\n3.\tProviding high-quality training data: The Caco framework generates a large-scale dataset, Caco-1.3M, which offers high-quality LLM reasoning training data. \n\nWeaknesses\n\n1.\tLack of clarity in the Introduction: The main contributions of the paper are not clearly explained in the introduction.\n2.\tInadequate problem formulation: The paper lacks a clear problem formulation. It is essential to explicitly define and describe the code-assisted reasoning problem, using appropriate notations to help readers understand the whole framework.\n\nQUESTIONS:\n1.\tWhy is your method not evaluated on the AIME dataset?\n2.\tCode-assisted methods primarily address incorrect calculations in reasoning trajectories. Does the Caco framework specifically aim to reduce calculation errors? It would be beneficial to analyze whether improvements arise from other factors, such as Caco's influence on the reasoning patterns and behaviors of LLMs.\n3.\tThe introduction mentions the length inefficiency problem. Have you compared the efficiency metrics, such as response length and generation time, of Caco with those of other baseline methods?\n4.\tSuggestion: Clearly articulate the main contributions of your work, including the dataset generation and any novel mechanisms introduced in Caco.\n5.\tSuggestion: Complete the problem formulation section to enhance clarity.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Caco (Code-Assisted Chain-of-ThOught), a framework for automated generation of verifiable and code CoT data. Unlike traditional CoT methods that suffer from unverifiability issues, Caco leverages code execution to validate reasoning chains and automated back-translation to produce natural language instructions and CoTs.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n- This paper contribute a fully auto-generates 1.3M high-quality instruction-CoT pairs without human labels.\n- The proposed method outperforms prior datasets across multiple math benchmarks.\n\n### Weaknesses\n- This paper is difficult to distinguish from some accepted works [1,2]. Both adopt the same approach of converting mathematical CoTs into code, using Python execution for filtering. This has become a common practice for CoT verification. Moreover, this paper also uses AST-based code validation, resulting in significant overlap in methodlogy.\n- Evaluation limited to math.Despite the claimed generality, experiments are only conducted on math datasets. There is no evidence the framework generalizes to other reasoning domains (e.g., logic puzzles, science QA).\n- Scalability claim lacks resource metrics. While Caco claims to be scalable, the paper does not report data generation time, compute cost, or comparison to other data-efficient methods.\n\n[1] Gao L, Madaan A, Zhou S, et al. Pal: Program-aided language models[C]//International Conference on Machine Learning. PMLR, 2023: 10764-10799.\n[2] [PaD: Program-aided Distillation Can Teach Small Models Reasoning Better than Chain-of-thought Fine-tuning](https://aclanthology.org/2024.naacl-long.142/) (Zhu et al., NAACL 2024)\n\nQUESTIONS:\n1. The proposed method is only evaluated on mathematical and algorithmic tasks. It remains unclear whether Caco's code-assisted CoT generation and filtering pipeline generalizes to other domains (e.g., logic puzzles, scientific reasoning, or commonsense QA). Have the authors considered extending their framework beyond math-centric problems?\n2. The paper does not report the proportion of generated code-CoTs that fail execution or are filtered out due to semantic or structural issues. What is the rejection rate during data generation? Understanding how much noisy or invalid data is discarded would help assess the efficiency and robustness of the pipeline.\n\nLIMITATIONS:\nSee in Questions and Weaknesses", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}