OpenThoughts: Data Recipes for Reasoning Models
Authors: name: Etash Guha; name: Ryan Marten; name: Sedrick Keh; name: Negin Raoof; name: Georgios Smyrnis; name: Hritik Bansal; name: Marianna Nezhurina; name: Jean Mercat; name: Trung Vu; name: Zayne Sprague; name: Ashima Suvarna; name: Benjamin Feuer; name: Liangyu Chen; name: Zaid Khan; name: Eric Frankel; name: Sachin Grover; name: Caroline Choi; name: Niklas Muennighoff; name: Shiye Su; name: Wanjia Zhao; name: John Yang; name: Shreyas Pimpalgaonkar; name: Kartik Sharma; name: Charlie Cheng-Jie Ji; name: Yichuan Deng; name: Sarah Pratt; name: Vivek Ramanujan; name: Jon Saad-Falcon; name: Jeffrey Li; name: Achal Dave; name: Alon Albalak; name: Kushal Arora; name: Blake Wulfe; name: Chinmay Hegde; name: Greg Durrett; name: Sewoong Oh; name: Mohit Bansal; name: Saadia Gabriel; name: Aditya Grover; name: Kai-Wei Chang; name: Vaishaal Shankar; name: Aaron Gokaslan; name: Mike A. Merrill; name: Tatsunori Hashimoto; name: Yejin Choi; name: Jenia Jitsev; name: Reinhard Heckel; name: Maheswaran Sathiamoorthy; name: Alexandros G. Dimakis; name: Ludwig Schmidt
Published: 2025-06-04 17:25:39

Abstract: Reasoning models have made rapid progress on many benchmarks involving math,
code, and science. Yet, there are still many open questions about the best
training recipes for reasoning since state-of-the-art models often rely on
proprietary datasets with little to no public information available. To address
this, the goal of the OpenThoughts project is to create open-source datasets
for training reasoning models. After initial explorations, our OpenThoughts2-1M
dataset led to OpenThinker2-32B, the first model trained on public reasoning
data to match DeepSeek-R1-Distill-32B on standard reasoning benchmarks such as
AIME and LiveCodeBench. We then improve our dataset further by systematically
investigating each step of our data generation pipeline with 1,000+ controlled
experiments, which led to OpenThoughts3. Scaling the pipeline to 1.2M examples
and using QwQ-32B as teacher yields our OpenThinker3-7B model, which achieves
state-of-the-art results: 53% on AIME 2025, 51% on LiveCodeBench 06/24-01/25,
and 54% on GPQA Diamond. All of our datasets and models are available on
https://openthoughts.ai.