CoRT: Code-integrated Reasoning within Thinking
Authors: name: Chengpeng Li, name: Zhengyang Tang, name: Ziniu Li, name: Mingfeng Xue, name: Keqin Bao, name: Tian Ding, name: Ruoyu Sun, name: Benyou Wang, name: Xiang Wang, name: Junyang Lin, name: Dayiheng Liu

Large Reasoning Models (LRMs) like o1 and DeepSeek-R1 have shown remarkable
progress in natural language reasoning with long chain-of-thought (CoT), yet
they remain inefficient or inaccurate when handling complex mathematical
operations. Addressing these limitations through computational tools (e.g.,
computation libraries and symbolic solvers) is promising, but it introduces a
technical challenge: Code Interpreter (CI) brings external knowledge beyond the
model's internal text representations, thus the direct combination is not
efficient. This paper introduces CoRT, a post-training framework for teaching
LRMs to leverage CI effectively and efficiently. As a first step, we address
the data scarcity issue by synthesizing code-integrated reasoning data through
Hint-Engineering, which strategically inserts different hints at appropriate
positions to optimize LRM-CI interaction. We manually create 30 high-quality
samples, upon which we post-train models ranging from 1.5B to 32B parameters,
with supervised fine-tuning, rejection fine-tuning and reinforcement learning.
Our experimental results demonstrate that Hint-Engineering models achieve 4\%
and 8\% absolute improvements on DeepSeek-R1-Distill-Qwen-32B and
DeepSeek-R1-Distill-Qwen-1.5B respectively, across five challenging
mathematical reasoning datasets. Furthermore, Hint-Engineering models use about
30\% fewer tokens for the 32B model and 50\% fewer tokens for the 1.5B model
compared with the natural language models. The models and code are available at
https://github.com/ChengpengLi1003/CoRT.