DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and
  Verifiable Mathematical Dataset for Advancing Reasoning
Authors: name: Zhiwei He, name: Tian Liang, name: Jiahao Xu, name: Qiuzhi Liu, name: Xingyu Chen, name: Yue Wang, name: Linfeng Song, name: Dian Yu, name: Zhenwen Liang, name: Wenxuan Wang, name: Zhuosheng Zhang, name: Rui Wang, name: Zhaopeng Tu, name: Haitao Mi, name: Dong Yu

The capacity for complex mathematical reasoning is a key benchmark for
artificial intelligence. While reinforcement learning (RL) applied to LLMs
shows promise, progress is significantly hindered by the lack of large-scale
training data that is sufficiently challenging, possesses verifiable answer
formats suitable for RL, and is free from contamination with evaluation
benchmarks. To address these limitations, we introduce DeepMath-103K, a new,
large-scale dataset comprising approximately 103K mathematical problems,
specifically designed to train advanced reasoning models via RL. DeepMath-103K
is curated through a rigorous pipeline involving source analysis, stringent
decontamination against numerous benchmarks, and filtering for high difficulty
(primarily Levels 5-9), significantly exceeding existing open resources in
challenge. Each problem includes a verifiable final answer, enabling rule-based
RL, and three distinct R1-generated solutions suitable for diverse training
paradigms like supervised fine-tuning or distillation. Spanning a wide range of
mathematical topics, DeepMath-103K promotes the development of generalizable
reasoning. We demonstrate that models trained on DeepMath-103K achieve
significant improvements on challenging mathematical benchmarks, validating its
effectiveness. We release DeepMath-103K publicly to facilitate community
progress in building more capable AI reasoning systems:
https://github.com/zwhe99/DeepMath.