NeurIPS 2025

Teaching Language Models to Reason with Tools
Chengpeng Li∗1,2, Zhengyang Tang∗2,3, Ziniu Li∗3,4, Mingfeng Xue2, Keqin Bao1,2, Tian Ding4,
Ruoyu Sun3,4, Benyou Wang3, Xiang Wang1, Junyang Lin2, and Dayiheng Liu†2
1University of Science and Technology of China
2Qwen Team, Alibaba Inc.
3The Chinese University of Hong Kong, Shenzhen
4Shenzhen International Center for Industrial and Applied Mathematics, Shenzhen Research Institute
of Big Data
Abstract
Large reasoning models (LRMs) like OpenAI-o1 have shown impressive capa-
bilities in natural language reasoning. However, these models frequently demon-
strate inefficiencies or inaccuracies when tackling complex mathematical opera-
tions. While integrating computational tools such as Code Interpreters (CIs) offers
a promising solution, it introduces a critical challenge: a conflict between the
model’s internal, probabilistic reasoning and the external, deterministic knowl-
edge provided by the CI, which often leads models to unproductive deliberation.
To overcome this, we introduce CoRT (Code-Optimized Reasoning Training), a
post-training framework designed to teach LRMs to effectively utilize CIs. We
propose Hint-Engineering, a new data synthesis strategy that strategically injects
diverse hints at optimal points within reasoning paths. This approach generates
high-quality, code-integrated reasoning data specifically tailored to optimize LRM-
CI interaction. Using this method, we have synthesized 30 high-quality samples
to post-train models ranging from 1.5B to 32B parameters through supervised
fine-tuning. CoRT further refines the multi-round interleaving of external CI usage
and internal thinking by employing rejection sampling and reinforcement learn-
ing. Our experimental evaluations demonstrate CoRT’s effectiveness, yielding
absolute improvements of 4% and 8% on DeepSeek-R1-Distill-Qwen-32B and
DeepSeek-R1-Distill-Qwen-1.5B, respectively, across five challenging mathemati-
cal reasoning datasets. Moreover, CoRT sign...