DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and
  Verifiable Mathematical Dataset for Advancing Reasoning
Authors: name: Zhiwei He; name: Tian Liang; name: Jiahao Xu; name: Qiuzhi Liu; name: Xingyu Chen; name: Yue Wang; name: Linfeng Song; name: Dian Yu; name: Zhenwen Liang; name: Wenxuan Wang; name: Zhuosheng Zhang; name: Rui Wang; name: Zhaopeng Tu; name: Haitao Mi; name: Dong Yu
Published: 2025-04-15 17:59:51

Abstract: The capacity for complex mathematical reasoning is a key benchmark for
artificial intelligence. While reinforcement learning (RL) applied to LLMs
shows promise, progress is significantly hindered by the lack of large-scale
training data that is sufficiently challenging, possesses verifiable answer
formats suitable for RL, and is free from contamination with evaluation
benchmarks. To address these limitations, we introduce DeepMath-103K, a new,
large-scale dataset comprising approximately 103K mathematical problems,
specifically designed to train advanced reasoning models via RL. DeepMath-103K
is curated through a rigorous pipeline involving source analysis, stringent
decontamination against numerous benchmarks, and filtering for high difficulty
(primarily Levels 5-9), significantly exceeding existing open resources in
challenge. Each problem includes a verifiable final answer, enabling rule-based
RL, and three distinct R1-generated solutions suitable for diverse training
paradigms like supervised fine-tuning or distillation. Spanning a wide range of
mathematical topics, DeepMath-103K promotes the development of generalizable
reasoning. We demonstrate that models trained on DeepMath-103K achieve
significant improvements on challenging mathematical benchmarks, validating its
effectiveness. We release DeepMath-103K publicly to facilitate community
progress in building more capable AI reasoning systems:
https://github.com/zwhe99/DeepMath.